An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Data Lev V. Utkin, Andrei V. Konstantinov, and Kirill A. Vishniakov Peter the Great St.Petersburg Polytechnic University St.Petersburg, Russia e-mail:
[email protected],
[email protected],
[email protected] Abstract One of the most popular methods of the machine learning prediction explanation is the SHapley Additive exPlanations method (SHAP). An imprecise SHAP as a modification of the original SHAP is proposed for cases when the class probability distributions are imprecise and represented by sets of distributions. The first idea behind the imprecise SHAP is a new approach for computing the marginal contribution of a feature, which fulfils the important efficiency property of Shapley values. The second idea is an attempt to consider a general approach to calculating and reducing interval-valued Shapley values, which is similar to the idea of reachable probability intervals in the imprecise probability theory. A simple special implementation of the general approach in the form of linear optimization problems is proposed, which is based on using the Kolmogorov-Smirnov distance and imprecise contamination models. Numerical examples with synthetic and real data illustrate the imprecise SHAP. Keywords: interpretable model, XAI, Shapley values, SHAP, Kolmogorov-Smirnov distance, imprecise probability theory. arXiv:2106.09111v1 [cs.LG] 16 Jun 2021 1 Introduction An importance of the machine learning models in many applications and their success in solv- ing many applied problems lead to another problem which may be an obstacle for using the models in areas where their prediction accuracy as well as understanding is crucial, for ex- ample, in medicine, reliability analysis, security, etc.