Symbolic AI for XAI: Evaluating LFIT Inductive Programming for Fair and Explainable Automatic Recruitment

Alfonso Ortega, Julian Fierrez, Aythami Morales, Zilong Wang School of Engineering, Universidad Autonoma de Madrid {alfonso.ortega,julian.fierrez,aythami.morales}@uam.es, [email protected] Tony Ribeiro Laboratoire des Sciences du Numerique´ de Nantes National Institute of Informatics Japan tony [email protected]

Abstract learning process. A deep review about this question can be found in [2]. Machine learning methods are growing in relevance for Another characteristic of these machine learning algo- biometrics and personal information processing in domains rithms is that they rely on data, and therefore reflect those such as forensics, e-health, recruitment, and e-learning. In data. This could be an advantage in general, but in some these domains, white-box (human-readable) explanations of particular domains it could be an important drawback. Con- systems built on machine learning methods can become cru- sider, for example, automatic recruitment systems, or algo- cial. Inductive Logic Programming (ILP) is a subfield of rithms to authorize financial products. In these domains, symbolic AI aimed to automatically learn declarative the- ethic behavior is mandatory and biased data are unaccept- ories about the process of data. Learning from Interpreta- able. Appropriate measures have to be taken for guaran- tion Transition (LFIT) is an ILP technique that can learn teeing ethical AI behavior sometimes contradictory to the a propositional logic theory equivalent to a given black- possibly biased training data. These questions are receiving box system (under certain conditions). The present work increasing interest [1, 9, 25, 39, 40, 20]. takes a first step to a general methodology to incorporate On the other hand, logic programming is a declarative accurate declarative explanations to classic machine learn- programming paradigm with a high level of abstraction. It ing by checking the viability of LFIT in a specific AI ap- is based on a formal model (first order logic) to represent plication scenario: fair recruitment based on an automatic human knowledge. Inductive Logic Programming (ILP) tool generated with machine learning methods for ranking has been developed for inductively learning logic programs Curricula Vitae that incorporates soft biometric informa- from examples [22]. Roughly speaking, given a collection tion (gender and ethnicity). We show the expressiveness of of positive and negative examples and background knowl- LFIT for this specific problem and propose a scheme that edge, ILP systems learn declarative (symbolic) programs can be applicable to other domains. [24, 6], which could even be noise tolerant [7, 23], that en-

arXiv:2012.00360v1 [cs.AI] 1 Dec 2020 tail all of the positive examples but none of the negative examples. 1. Introduction One of the ILP most promising approaches for us is Learning From Interpretation Transition (LFIT) [30]. Statistical and optimization-based machine learning al- LFIT learns a logic representation (digital twin) of dy- gorithms have achieved great success in various applica- namical complex systems by observing their behavior as tions such as speech recognition [38], image classification a black box under some circumstances. The most general [8], machine translation [43], and so on. Among these ap- of LFIT algorithms is GULA (General Usage LFIT Algo- proaches, deep neural networks have shown the most re- rithm). PRIDE is an approximation to GULA with poly- markable success, especially in speech and image recog- nomial performance. GULA and PRIDE generate a propo- nition. Although deep learning methods usually have good sitional logic program equivalent to the system under con- generalization ability on similarly distributed new data, they sideration. These approaches will be introduced in depth in have some weaknesses including the lack of explanations the following sections. and the poor understandability by humans of the whole Figure 1 shows the architecture of our proposed ap- Classifier seen as black-box system (input/outputs) 1 Input features (variables)

풱 = v1, v2, v3 Output classes (target) = {0,1} v1 binary, v2 and v3 ∈ ℕ

Examples (two inputs 풱퐴 and 풱퐵) Declarative explanation v1 = 0 v1 = 1 (propositional logic fragment) 풱퐴 = v2 = 5 풱퐵 = v2 = 3 LFIT (PRIDE) 3 v = 2 v = 0 Logical equivalent system 3 3 v 0 . (White-box Digital Twin of target(0) :- 1 the Black-box Classifier) target(1) :- v1 1 . 2

Note that the explanation reveals that the output only depends on v1

Figure 1: Architecture of the proposed approach for generating an explanation of a given black-box Classifier (1) using PRIDE (2) with a toy example (3). Note that the resulting explanations generated by PRIDE are in propositional logic. proach for generating white-box explanations using PRIDE automatically generating LISP expressions for given tasks of a given black-box classifier. expressed as pairs input/output. This is, in fact, a typical The main contributions of this work are: machine learning scenario. GP was extended by the use of formal grammars to allow to generate programs in any arbi- • We have proposed a method to provide declarative ex- trary language keeping not only syntactic correctness [27] planations using PRIDE about the classification pro- but also semantic properties [28]. Algorithms expressed cess made by an algorithm automatically learnt by a in any language are declarative versions of the concepts neural architecture in a typical machine learning sce- learnt what makes evolutionary automatic programming al- nario. Our approach guarantees the logical equivalence gorithms machine learners with good explainability. between the explanations and the algorithm with re- spect to the data used to feed PRIDE. It does not de- Declarative programming paradigms (functional, logi- pend on any particular characteristic of the domain, so cal) are as old as computer science and are implemented it could be applied to any problem. in multiple ways, e.g.: LISP [13], Prolog [5], Datalog [11], Haskell [41], and Answer Set Programs (ASP) [10]. • We have checked the expressive power of these ex- Of particular interest for us within declarative paradigms planations by experimenting in a multimodal machine is logic programming, and in particular first order logic pro- learning testbed around automatic recruitment includ- gramming, which is based on the Robinson’s resolution in- ing different biases (by gender and ethnicity). ference rule that automates the reasoning process of de- The rest of the paper is structured as follows: Sec. 2 sum- ducing new clauses from a first order theory [17]. Intro- marizes the related relevant literature. Sec. 3 describes our ducing examples and counter examples and combining this methodology including LFIT, GULA, and PRIDE. Sec. 4 scheme with the ability of extending the initial theory with presents the experimental framework including the datasets new clauses it is possible to automatically induce a new and experiments conducted. Sec. 5 presents our results. Fi- theory that (logically) entails all of the positive examples nally Secs. 6, 7 and 8 respectively discuss our work and but none of the negative examples. The underlying theory describe our conclusions and further research lines. from which the new one emerges is considered background knowledge. This is the hypothesis of Inductive Logic Pro- 2. Related Works: Inductive Programming for gramming (ILP, [21, 6]) that has received a great research XAI effort in the last two decades. Recently, these approaches have been extended to make them noise tolerant (in order Some meta-heuristics approaches (as genetic algorithms) to overcome one of the main drawbacks of ILP vs statisti- have been used to automatically generate programs. Ge- cal/numerical approaches when facing bad-labeled or noisy netic programming (GP) was introduced by Koza [15] for examples [23]). Other declarative paradigms are also compatible with ILP, e.g., MagicHaskeller (that implements [14]) with the functional programming language Haskell, and Inductive Learning of Answer Set Programs (ILASP) [16]. It has been previously mentioned that ILP implies some kind of search in spaces that can become huge. This search can be eased by hybridising with other techniques, e.g., [26] introduces GA-Progol that applies evolutive techniques. Within ILP methods we have identified LFIT as spe- cially relevant for explainable AI (XAI). In the next sec- tion we will describe the fundamentals of LFIT and its PRIDE implementation, which will be tested experimen- tally for XAI in the experiments that will follow.

2.1. Learning From Interpretation Transition (LFIT) Learning From Interpretation Transition (LFIT) [12] has been proposed to automatically construct a model of the dy- namics of a system from the observation of its state transi- tions. Given some raw data, like time-series data of gene expression, a discretization of those data in the form of state Figure 2: Experimental framework: PRIDE is fed with all transitions is assumed. From those state transitions, accord- the data available (train + test) for increasing the accuracy ing to the semantics of the system dynamics, several infer- of the equivalence. In our experiments we consider the clas- ence algorithms modeling the system as a logic program sifier (see [29] for details) as a black box to perform regres- have been proposed. The semantics of a system’s dynamics sion from input resume attributes (atts.) to output labels can indeed differ with regard to the synchronism of its vari- (recruitment scores labelled by human resources experts). ables, the determinism of its evolution and the influence of LFIT gets a digital twin to the neural network providing its history. explainability (as human-readable white-box rules) to the The LFIT framework proposes several modeling and neural network classifier. learning algorithms to tackle those different semantics. To date, the following systems have been tackled: memory- less deterministic systems [12], systems with memory [35], 3. Methods probabilistic systems [19] and their multi-valued extensions 3.1. General Methodology [36, 18]. The work [37] proposes a method that allows to deal with continuous time series data, the abstraction itself Figure 2 graphically describes our proposed approach being learned by the algorithm. to generate explanations using LFIT of a given black-box In [31, 33], LFIT was extended to learn systems dynam- classifier. We can see there our purpose to get declarative ics independently of its update semantics. That extension explanations in parallel (in a kind of white-blox digital twin) relies on a modeling of discrete memory-less multi-valued to a given neural network classifier. In the present work, for systems as logic programs in which each rule represents that our experiments we have used the same neural network and a variable possibly takes some value at the next state, ex- datasets described in [29] but excluding the face images as tending the formalism introduced in [12, 34]. The represen- it is explained in the following sections. tation in [31, 33] is based on annotated logics [4, 3]. Here, 3.2. PRIDE Implementation of LFIT each variable corresponds to a domain of discrete values. In a rule, a literal is an atom annotated with one of these GULA [31, 33] and PRIDE [32] are particular imple- values. It allows to represent annotated atoms simply as mentations of the LFIT framework [12]. In the present sec- classical atoms and thus to remain at a propositional level. tion we introduce notation and describe the fundamentals of This modeling allows to characterize optimal programs in- both methods. dependently of the update semantics. It allows to model the In the following, we denote by N := {0, 1, 2, ...} the set dynamics of a wide range of discrete systems including our of natural numbers, and for all k, n ∈ N, k; n := {i ∈ N | domain of interest in this paper. LFIT can be used to learn k ≤ i ≤ n} is the set of natural numbersJ betweenK k and n an equivalent propositional logic program that provides ex- included. For any set S, the cardinal of S is denoted |S| and planation for each given observation. the power set of S is denoted ℘(S). Let V = {v1,..., vn} be a finite set of n ∈ N vari- The dynamical system we want to learn the rules of is ables, Val the set in which variables take their values and represented by a succession of states as formally given by dom : V → ℘(Val) a function associating a domain to each Definition 3. We also define the “compatibility” of a rule variable. The atoms of MVL (multi-valued logic) are of with a state in Definition 4. the form vval where v ∈ V and val ∈ dom(v). The set of V val such atoms is denoted by Adom = {v ∈ V × Val | val ∈ Definition 3 (Discrete state) A discrete state s on T (resp. dom(v)} for a given set of variables V and a given domain F) of a DMVLP is a function from T (resp. F) to N, i.e. function dom. In the following, we work on specific V and it associates an integer value to each variable in T (resp. dom that we omit to mention when the context makes no F). It can be equivalently represented by the set of atoms V s(v) ambiguity, thus simply writing A for Adom. {v | v ∈ T (resp. F)} and thus we can use classical set T F Example 1 For a system of 3 variables, the typical set operations on it. We write S (resp. S ) to denote the set of variables is V = {a, b, c}. In general, Val = of all discrete states of T (resp. F), and a couple of states (s, s0) ∈ SF × ST is called a transition. N so that domains are sets of natural integers, for in- stance: dom(a) = {0, 1}, dom(b) = {0, 1, 2} and dom(c) = {0, 1, 2, 3} . Thus, the set of all atoms is: Definition 4 (Rule-state matching) Let s ∈ SF . The A = {a0, a1, b0, b1, b2, c0, c1, c2, c3} . MVL rule R matches s, written R u s, if b(R) ⊆ s. A MVL rule R is defined by: val val val The notion of transition in LFIT correspond to a data R = v 0 ← v 1 ∧ · · · ∧ v m (1) 0 1 m sample in the problem we tackle in this paper: a couple fea- vali where ∀i ∈ 0; m , vi ∈ A are atoms in MVL so that tures/targets. When a rule match a state it can be considered every variableJ is mentionedK at most once in the right-hand as a possible explanation to the corresponding observation. part: ∀j, k ∈ 1; m , j 6= k ⇒ vj 6= vk. Intuitively, the rule The final program we want to learn should both: R has the followingJ K meaning: the variable v0 can take the value val0 in the next dynamical step if for each i ∈ 1; m , • match the observations in a complete (all observations J K variable vi has value vali in the current dynamical step. are explained) and correct (no spurious explanation) The atom on the left-hand side of the arrow is called the way; val0 head of R and is denoted h(R) := v0 . The notation var(h(R)) := v denotes the variable that occurs in h(R). 0 • represent only minimal necessary interactions (accord- The conjunction on the right-hand side of the arrow is called ing to Occam’s razor: no overly-complex bodies of the body of R, written b(R) and can be assimilated to the val1 valm rules) set {v1 ,..., vm }; we thus use set operations such as ∈ and ∩ on it. The notation var(b(R)) := {v1, ··· , vm} denotes the set of variables that occurs in b(R). More gen- GULA [31, 33] and PRIDE [32] can produce such pro- erally, for all set of atoms X ⊆ A, we denote var(X) := grams. {v ∈ V | ∃val ∈ dom(v), vval ∈ X} the set of variables Formally, given a set of observations T , GULA [31, 33] appearing in the atoms of X.A multi-valued logic program and PRIDE [32] will learn a set of rules P such that all (MVLP) is a set of MVL rules. observations are explained: ∀(s, s0) ∈ T, ∀vval ∈ s0, ∃R ∈ Definition 1 introduces a domination relation between P,R u s, h(R) = vval. All rules of P are correct w.r.t. rules that defines a partial anti-symmetric ordering. Rules T : ∀R ∈ P, ∀(s1, s2) ∈ T,R u s1 =⇒ ∃(s1, s3) ∈ with the most general bodies dominate the other rules. In T, h(R) ∈ s3 (if T is deterministic, s2 = s3). All rules are practice, these are the rules we are interested in since they minimal w.r.t. F: ∀R ∈ P, ∀R0 ∈ MVLP,R0 correct w.r.t. cover the most general cases. T it holds that R ≤ R0 =⇒ R0 = R. The possible explanations of an observation are the rules Definition 1 (Rule Domination) Let R1, R2 be two MVL that match the feature state of this observation. The body of rules. The rule R1 dominates R2, written R2 ≤ R1 if the rules gives minimal condition over feature variables to h(R1) = h(R2) and b(R1) ⊆ b(R2). obtain its conclusions over a target variable. Multiple rules In [33], the set of variables is divided into two disjoint can match the same feature state, thus multiple explanations subsets: T (for targets) and F (for features). It allows to can be possible. Rules can be weighted by the number of define dynamic MVLP which capture the dynamics of the observations they match to assert their level of confidence. problem we tackle in this paper. Output programs of GULA and PRIDE can also be used in Definition 2 (Dynamic MVLP) Let T ⊂ V and F ⊂ V order to predict and explain from unseen feature states by such that F = V\T .A DMVLP P is a MVLP such that learning additional rules that encode when a target variable ∀R ∈ P, var(h(R)) ∈ T and ∀vval ∈ b(R), v ∈ F. value is not possible as shown in the experiments of [33]. 4. Experimental Framework 4.1. Dataset

For testing the capability of PRIDE to generate explana- tions in machine learning domains we have designed several experiments using the FairCVdb dataset [29]. FairCVdb comprises 24,000 synthetic resume profiles. Each resume includes 12 features (vi) related to the can- didate merits, 2 demographic attributes (gender and three ethnicity groups), and a face photograph. In our experi- ments, we discarded the face image for simplicity (unstruc- tured image data will be explored in future work). Each of Figure 3: Structure of the experimental tests. There are 4 the profiles includes three target scores (T ) generated as a datasets for analysing gender (named g) and ethnicity (e) linear combination of the 12 features: bias separately. Apart from gender and ethnicity there are 12 other input attributes (named from i1 to i12). There is a couple of (biased and unbiased) datasets for each one: gen- 12 X der and ethnicity. We have studied the input attributes by T = β + α · v , (2) i i increasing complexity starting with i1 and i2 and adding i=1 one at each time. So, for each couple we have considered 11 different scenarios (named from s1 to s11). This fig- α where i is a weighting factor for each of the merits (see ure shows their structure (s is included in all s for which i) β = 0 ii) i j [29] for details): unbiased score ( ); gender- i < j). biased scores (β = 0.2 for male and β = 0 for female candidates); and iii) ethnicity-biased scores (β = 0.0, 0.15 and 0.3 for candidates from ethnic groups 1, 2 and 3 respec- 5. Results tively). Thus we intentionally introduce bias in the candi- date scores. From this point on we will simplify the name 5.1. Example of Declarative Explanation of the attributes considering g for gender, e for ethnic group Listing 1 shows a fragment generated with the proposed and i1 to i12 for the rest of input attributes. In addition methods for scenario s1 for gender-biased scores. We have to the bias previously introduced, some other random bias chosen a fragment that fully explains how a CV is scored was introduced relating some attributes and gender to sim- with the value 3 for scenario 1. Scenario 1 takes into ac- ulate real social dynamics. The attributes concerned were count the input attributes gender, education and experience. i3 and i7. Note that merits were generated without bias, as- The first clause (rule), for example, says that if the value of suming an ideal scenario where candidate competencies do a CV for the attribute gender is 1 (female), for education is 5 not depend on their gender of ethnic group. For the current (the highest), and for experience is 3, then this CV receives work we have used only discrete values for each attribute the highest score (3). discretizing one attribute (experience to take values from 0 The resulting explanation is a propositional logic frag- to 5, the higher the better) and the scores (from 0 to 3) that ment equivalent to the classifier for the data seen. It can were real valued in [29]. be also understood as a set of rules with the same behav- ior. From the viewpoint of explainable AI, this resulting 4.2. Experimental Protocol: Towards Declarative fragment can be understood by an expert in the domain and Explanations used to generate new knowledge about the scoring of CVs.

1 We have experimented with PRIDE on the FairCVdb 2 scores(3) :- gender(1), dataset described in the previous section. 3 education(5), 4 experience(3). Figure 3 shows names and explains the scenarios consid- 5 scores(3) :- education(4), ered in our experiments. In [29], researchers demonstrate 6 experience(3). that an automatic recruitment algorithm based on multi- Listing 1: Fragment of explanation for scoring 3 modal machine learning reproduces existing biases in the target functions even if demographic information was not 5.2. Quantitative Results: Identifying Biases available as input (see [29] for details). Our purpose in the experiments is to obtain a declarative explanation capable Our quantitative results are divided in two parts. The first of revealing those biases. part is based on the fact that, in the biased experiments, if gender(0) appears more frequently than gender(1) in the rules, then that would lead to higher scores for gender(0). In the second quantitative experimental part we will show the influence of bias in the distribution of attributes.

5.2.1 Biased attributes in rules

We first define Partial Weight PW as follows. For any pro- i j val0 val1 i gram P and two atoms v0 and v1 , where val0 ∈ val0 i i val0 and val1 ∈ val1, define S = ∀R ∈ P ∧ v0 ∈ h(R) ∧ j 0 val1 val0 v ∈ b(R). Then we have: PW j (v ) = |S|.A 1 val1 0 v1 more accurate PW could be defined, for example, setting different weights for rules with different length. But for our Figure 4: Percentage of the absolute increment (comparing purpose, the frequency is enough. In our analysis, the num- scores with and without bias for ethnicity) of each attribute ber of examples for compared scenarios are consistent. for scenarios s1, s2, s3, s4, s5 and s6 (AIPus1−6,ebs1−6). Depending on PW , we define Global Weight GW The graphs link the points corresponding to all the input valj 1 attributes considered in each scenario. as follows. For any program P and v1 , we have: i val0 i j P j GW val = i PW val (v ) · val . The 1 val0∈val0 1 0 0 v1 v1 GW j is a weighted addition of all the values of the out- val1 v1 put, and the weight, in our case, is the value of scores. This analysis was performed only on scenario s11, com- paring unbiased and gender- and ethnicity-biased scores. We have observed a similar behavior of both parameters: Partial and Global Weights. In unbiased scenarios the distributions of the occurrences of each value could be considered statistically the same (between gender(0) and gender(1) and among ethnicity(0), ethnicity(1) and ethnic- ity(2)). Nevertheless in biased datasets the occurrences of gender(0) and ethnic(0) for higher scores is significantly higher. The maximum difference even triplicates the oc- currences of the other values. Figure 5: AIP For the Global Weights, for example, the maximum dif- us7−11,ebs7−11 ferences in the number of occurrences, without and with bias respectively, for higher scores expressed as % increases We have observed that for both parameters the only from 48.8% to 78.1% for gender(0) while for gender(1) attributes that consistently increase their values are gen- decreases from 51.2% to 21.9%. In the case of ethnicity, der and ethnicity comparing unbiased and gender/ethnicity- it increases from 33.4% to 65.9% for ethnic(0), but de- biased scores. Figures 4 and 5 show AIPus1−11,ebs1−11 for creases from 33.7% to 19.4% for ethnic(1) and from 32.9% each attribute, that is, their values comparing unbiased and to 14.7% for ethnic(2). ethnic-biased scores for all the scenarios from s1 to s11. It is clear that the highest values correspond to the attribute 5.2.2 Distribution of biased attributes ethnicity. Something similar happens for gender. Figures 6 and

We now define freqp1 (a) as the frequency of attribute a in 7 show AIPus1−11,gbs1−11 for each attribute when study- P1. The normalized percentage for input a is: NPp (a) = ing gender-biased scores. It is worth mentioning some dif- P 1 freqp1 (a)/ x∈input freqp1 (x) and the percentage of the ferences in scenarios s9, s10 and s11 regarding attributes absolute increment for each input from unbiased exper- i3 and i7. These apparent anomalies are explained by the iments to its corresponding biased ones is defined as: random bias introduced in the datasets in order to relate

AIPp1,p2 (a) = (freqp1 (a) − freqp2 (a))/freqp2 (a). these attributes with gender when the score is biased. Fig- In this approach we have taken into account all the sce- ure 8 shows NPs11 for all the attributes. It clearly shows narios (from s1 to s11) for both gender and ethnicity. the small relevance of attributes i3 and i7 in the final bi- Figure 6: AIPus1−6,ebs1−6 Figure 8: Normalized percentage of frequency in scenario s11 of each attribute: g, i1 to i11 (NPs11). No bias (blue), Gender-biased scores (red).

• PRIDE can explain what happens in a specific do- main. Our experimental results discover these charac- teristics of the domain:

– Insights into the structure of the datasets. We have seen (and further confirmed with the au- thors of the datasets) some characteristics of the datasets, e.g.: 1) All the attributes are needed for the score. We have learnt the logical ver- sion of the system starting from only two in- put attributes and including one additional at- tribute at a time and we only reached an accu- Figure 7: AIP us7−11,gbs7−11 racy of 100 % when taking into account all of them. This is because removing some attributes ased score. As it is highlighted elsewhere, this capability generates indistinguishable CVs (all the remain- of PRIDE to identify this random indirect perturbation of der attributes have the same value) with differ- other attributes in the bias is a relevant achievement of our ent scores (that correspond to different values in proposal. some of the removed attributes). 2) Gender and ethnicity are not the most relevant attributes for scoring: The number of occurrences of these at- 6. Discussion tributes is much smaller than others in the con- After running the experiments described in the previous ditions of the clauses of the learnt logical pro- sections we can extract the following conclusions. gram. 3) While trying to catch the biases we have discovered that some attributes seem to increase • PRIDE can explain algorithms learnt by neural net- their relevance when the score is biased. For works. The theorems that support the characteristics example, the competence in some specific lan- of PRIDE allow to get a set of propositional clauses guages (attribute i7) seems to be more relevant logically equivalent to the systems observed when fac- when the score has gender bias. After discussing ing the input data provided. In addition, each proposi- with the authors of the datasets, they confirmed a tion has a set of conditions that is minimum. So, once random perturbation of these languages into the the scorer is learnt, PRIDE translates it into a logical biases, that explained our observations. equivalent program. This program is a list of clauses – Biases in the training datasets are detected. We like the one shown in Listing 1. Logical programs are have analysed the relationship between the scores declarative theories that explain the knowledge on a and the specific values of the attributes used to domain. generated the biased data. We have proposed a simple mathematical model based on the ef- the formal model that supports the learning engine us- fective weights of the attributes that concludes ing, for example, ILP based on first order logic. that higher values of the scores correspond to the same specific values of gender (for gender • Adding predictive capability. PRIDE is actually bias) and ethnic group (for ethnicity bias). On the not aimed to predict but to explain (declaratively) by other hand, we have performed an exhaustive se- means of a digital twin of the observed systems. Nev- ries of experiments to analyse the increase of the ertheless, it is not really complicated to extend PRIDE presence of the gender and ethnicity in the condi- functionality to predict. It should be necessary to tions of the clauses of the learnt logical program change the way in which the result is interpreted as (comparing the unbiased and biased versions). a logical program: mainly by adding mechanisms to chose the most promising rule when more than one is Our overall conclusion is that LFIT, and in particular applicable. PRIDE, is able to offer explanations to the algorithm learnt Our plan is to test an extended-to-predict PRIDE ver- in the domain under consideration. The resulting explana- sion to this same domain and compare the result with tion is, as well, expressive enough to catch training biases the classifier generated by deep learning algorithms. in the models learnt with neural networks. • Handling numerical inputs. [29] included as input 7. Conclusions the images of the faces of the owners of the CVs. Al- though some variants to PRIDE are able to cope with The main goal of this paper was to check if ILP (and numerical signals, the huge amount of information as- more specifically LFIT with PRIDE) could be useful to pro- sociated with images implies performance problems. vide declarative explanations in machine learning by neural Images are a typical input format in real deep learn- networks. ing domains. We would like to add some automatic The domain selected for our experiments in this first pre-processing step for extracting discrete information entry to the topic is one in which the explanations of the (such as semantic labels) from input images. We are learned models’ outputs are specially relevant: automatic motivated by the success of systems with similar ap- recruitment algorithms. In this domain, ethic behavior is proaches but different structure like [42]. needed, no spurious biases are allowed. For this purpose, • Measuring the accuracy and performance of the ex- a pack of synthetically generated datasets has been used. planations. As far as the authors know there is no The datasets contain resumes (CVs) used in [29] for testing standard procedure to evaluate and compare different the ability of deep learning approaches to reproduce and re- explainability approaches. We will incorporate in fu- move biases present in the training datasets. In the present ture versions some formal metric. work, different input attributes (including the resume owner merits, gender, and ethnicity) are used to score each CV 9. Acknowledgements automatically using a neural network. Different setups are considered to introduce artificial gender- and ethnicity-bias This work has been supported by projects: PRIMA in the learning process of the neural network. In [29] face (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN images were also used and the relationship between these (H2020-MSCA-ITN-2019-860813), IDEA-FAST (IMI2- pictures and the biases was studied (it seems clear that from 2018-15-853981), BIBECA (RTI2018-101248-B-I00 the face you should be able to deduce the gender and ethnic MINECO/FEDER), RTI2018-095232-B-C22 MINECO, group of a person). Here we have removed images because and Accenture. PRIDE is more efficient with pure discrete information. Our main goal indicated above translates into these two References questions: Is PRIDE expressive enough to explain how the [1] A. Acien, A. Morales, R. Vera-Rodriguez, I. Bartolome, and program learnt by deep-learning approaches works? Does J. Fierrez. Measuring the gender and ethnicity bias in deep PRIDE catch biases in the deep-learning processes? We models for face recognition. In Iberoamerican Congress have given positive answer to both questions. on Pattern Recognition (IBPRIA), pages 584–593. Springer, 2018. 8. Further Research Lines [2] A. Barredo Arrieta, N. D´ıaz Rodr´ıguez, J. Del Ser, A.Bennetot, S. Tabik, A. Barbado, S. Garc´ıa, S. Gil-Lopez, • Increasing understandability. Two possibilities D. Molina, R. Benjamins, R. Chatila, and F. Herrera. Ex- could be considered in the future: 1) to ad hoc post- plainable artificial intelligence (XAI): concepts, taxonomies, process the learnt program for translating it into a more opportunities and challenges toward responsible AI. Infor- abstract form, or 2) to increase the expressive power of mation Fusion, 58:82–115, 2020. [3] H. A Blair and V.S. Subrahmanian. Paraconsistent founda- [21] S. Muggleton. Inductive logic programming. In S. Arikawa, tions for logic programming. Journal of Non-classical Logic, Sh. Goto, S. Ohsuga, and T. Yokomori, editors, Proc. First 5(2):45–73, 1988. Intl. Workshop on Algorithmic Learning Theory, pages 42– [4] H. A. Blair and V.S. Subrahmanian. Paraconsistent logic pro- 62. Springer, October 1990. gramming. Theoretical Computer Science, 68(2):135 – 154, [22] S. Muggleton. Inductive logic programming. New Genera- 1989. tion Computing, 8(4):295–318, 1991. [5] I. Bratko. Prolog Programming for Artificial Intelligence, [23] S. Muggleton, W.-Z. Dai, C. Sammut, A. Tamaddoni- 4th Edition. Addison-Wesley, 2012. Nezhad, J. Wen, and Z.-H. Zhou. Meta-interpretive learning [6] A. Cropper and S. H. Muggleton. Learning efficient logic from noisy images. Machine Learning, 107(7):1097–1118, programs. Machine Learning, 108(7):1063–1083, 2019. 2018. [7] W. Z. Dai, S. H. Muggleton, and Z. H. Zhou. Logical vision: [24] S. H. Muggleton, D. Lin, N. Pahlavi, and A. Tamaddoni- Meta-interpretive learning for simple geometrical concepts. Nezhad. Meta-interpretive learning: application to grammat- 2015. ical inference. Machine Learning, 94(1):25–49, 1994. [8] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. [25] S. Nagpal, M. Singh, R. Singh, M. Vatsa, and N.i K. Ratha. ImageNet: A Large-Scale Hierarchical Image Database. In Deep learning for face recognition: Pride or prejudiced? CVPR, 2009. CoRR, abs/1904.01219, 2019. [9] P. Drozdowski, C. Rathgeb, A. Dantcheva, N. Damer, and [26] A. T. Nezhad. Logic-based machine learning using a C. Busch. Demographic bias in biometrics: A survey on an bounded hypothesis space: the lattice structure, refinement emerging challenge. IEEE Transactions on Technology and operators and a genetic algorithm approach, August 2013. Society, 2020. PhD, Imperial College London. [10] M. Gebser, R. Kaminski, B. Kaufmann, and T. Schaub. An- [27] M. O’Neill and R. Conor. Grammatical Evolution - Evolu- swer Set Solving in Practice. Synthesis Lectures on Artifi- tionary Automatic Programming in an Arbitrary Language, cial Intelligence and Machine Learning. Morgan & Claypool volume 4 of Genetic programming. Kluwer, 2003. Publishers, 2012. [28] A. Ortega, M. de la Cruz, and M. Alfonseca. Christiansen [11] S. S. Huang, T. Jeffrey Green, and B. T. Loo. Datalog and grammar evolution: Grammatical evolution with semantics. emerging applications: an interactive tutorial. In Timos K. IEEE Trans. Evol. Comput., 11(1):77–90, 2007. Sellis, Renee´ J. Miller, Anastasios Kementsietsidis, and Yan- [29] A. Pena,˜ I. Serna, A. Morales, and J. Fierrez. Bias in mul- nis Velegrakis, editors, Proc. of the ACM SIGMOD Intl. timodal AI: testbed for fair automatic recruitment. In Proc. Conf. on Management of Data, pages 1213–1216, June 2011. IEEE/CVF Conf. on Computer Vision and Pattern Recogni- [12] K. Inoue, T. Ribeiro, and C. Sakama. Learning from inter- tion (CVPR) Workshops, pages 129–137, June 2020. pretation transition. Machine Learning, 94(1):51–79, 2014. [30] T. Ribeiro. Studies on learning dynamics of systems from [13] Guy L. Steele Jr. Common LISP: The Language, 2nd Edition. state transitions, 2015. PhD. Digital Pr., 1990. [31] T. Ribeiro, M. Folschette, M. Magnin, O. Roux, and K. In- [14] S. Katayama. Systematic search for lambda expressions. In oue. Learning dynamics with synchronous, asynchronous M. C. J. D. van Eekelen, editor, Revised Selected Papers from and general semantics. In International Conference on In- the Sixth Symposium on Trends in Functional Programming, ductive Logic Programming, pages 118–140. Springer, 2018. volume 6 of Trends in Functional Programming, pages 111– [32] T. Ribeiro, M. Folschette, L. Trilling, N. Glade, K. Inoue, M. 126. Intellect, September 2005. Magnin, and O. Roux. Les enjeux de l’inference´ de modeles` [15] J.R. Koza. Genetic Programming. MIT Press, 1992. dynamiques des systemes` biologiques a` partir de series´ tem- [16] M. Law. Inductive learning of answer set programs., 2018. porelles. In C. Lhoussaine and E. Remy, editors, Approches PhD. Imperial College London. symboliques de la modelisation´ et de l’analyse des systemes` [17] J. W. Lloyd. Foundations of Logic Programming, 2nd Edi- biologiques. ISTE Editions, 2020. In edition. tion. Springer, 1987. [33] T. Ribeiro, M. Folschette, M.and Magnin, and K. Inoue. [18] D. Martınez, G. Alenya, C. Torras, T. Ribeiro, and K. Inoue. Learning any semantics for dynamical systems represented Learning relational dynamics of stochastic domains for plan- by logic programs. working paper or preprint, Sept. 2020. ning. In Proceedings of the 26th International Conference on [34] T. Ribeiro and K. Inoue. Learning prime implicant condi- Automated Planning and Scheduling, 2016. tions from interpretation transition. In Inductive Logic Pro- [19] D. Mart´ınez Mart´ınez, T. Ribeiro, K. Inoue, G. Alenya` Ribas, gramming, pages 108–125. Springer, 2015. and C. Torras. Learning probabilistic action models from [35] T. Ribeiro, M. Magnin, K. Inoue, and C. Sakama. Learning interpretation transitions. In Proceedings of the Technical delayed influences of biological systems. Frontiers in Bio- Communications of the 31st International Conference on engineering and Biotechnology, 2:81, 2015. Logic Programming (ICLP 2015), pages 1–14, 2015. [36] T. Ribeiro, M. Magnin, K. Inoue, and C. Sakama. Learn- [20] A. Morales, J. Fierrez, R. Vera-Rodriguez, and R. Tolosana. ing multi-valued biological models with delayed influence Sensitivenets: Learning agnostic representations with appli- from time-series observations. In 2015 IEEE 14th Inter- cation to face recognition. IEEE Transactions on Pattern national Conference on Machine Learning and Applications Analysis and Machine Intelligence, 2021. (ICMLA), pages 25–31, Dec 2015. [37] T. Ribeiro, S. Tourret, M. Folschette, M. Magnin, D. Borzac- chiello, F. Chinesta, O. Roux, and K. Inoue. Inductive learn- ing from state transitions over continuous domains. In N. Lachiche and C. Vrain, editors, Inductive Logic Program- ming, pages 124–139. Springer, 2018. [38] A. Senior, V. Vanhoucke, P. Nguyen, and T. Sainath. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing magazine, 2012. [39] I. Serna, A. Morales, J. Fierrez, M. Cebrian, N. Obradovich, and I. Rahwan. Algorithmic discrimination: Formulation and exploration in deep learning-based face biometrics. In AAAI Workshop on Artificial Intelligence Safety (SafeAI), February 2020. [40] I. Serna, A. Pena,˜ A.Morales, and J. Fierrez. InsideBias: Measuring bias in deep networks and application to face gen- der biometrics. In IAPR Intl. Conf. on Pattern Recognition (ICPR), January 2021. [41] S. J. Thompson. Haskell - The Craft of Functional Program- ming, 3rd Edition. Addison-Wesley, 2011. [42] D. Varghese and A. Tamaddoni-Nezhad. One-shot rule learn- ing for challenging character recognition. In S. Moschoyian- nis, P. Fodor, J. Vanthienen, D. Inclezan, Ni. Nikolov, F. Mart´ın-Recuerda, and I. Toma, editors, Proc. of the 14th Intl. Rule Challenge, volume 2644 of CEUR Workshop Proceed- ings, pages 10–27, June 2020. [43] Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi, W. Macherey, and J Klingner. Google’s neural machine transla- tion system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016.