DOCSLIB.ORG

Convex Nonnegative Matrix Factorization with Missing Data Ronan Hamon, Valentin Emiya, Cédric Févotte

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Convex Nonnegative Matrix Factorization with Missing Data Ronan Hamon, Valentin Emiya, Cédric Févotte Convex nonnegative matrix factorization with missing data Ronan Hamon, Valentin Emiya, Cédric Févotte To cite this version: Ronan Hamon, Valentin Emiya, Cédric Févotte. Convex nonnegative matrix factorization with missing data. IEEE International Workshop on Machine Learning for Signal Processing, Sep 2016, Vietri sul Mare, Salerno, Italy. hal-01346492 HAL Id: hal-01346492 https://hal-amu.archives-ouvertes.fr/hal-01346492 Submitted on 7 Oct 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. 2016 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 13–16, 2016, SALERNO, ITALY CONVEX NONNEGATIVE MATRIX FACTORIZATION WITH MISSING DATA Ronan Hamon, Valentin Emiya˚ Cedric´ Fevotte´ Aix Marseille Univ, CNRS, LIF, Marseille, France CNRS & IRIT, Toulouse, France ABSTRACT example concerns the field of image or audio inpainting [5, 6, 7, 8], where CNMF may improve the current reconstruction Convex nonnegative matrix factorization (CNMF) is a variant techniques. In inpainting of audio spectrograms for example, of nonnegative matrix factorization (NMF) in which the com- setting up the dictionary to be a comprehensive collection of ponents are a convex combination of atoms of a known dic- notes from a specific instrument may guide the factorization tionary. In this contribution, we propose to extend CNMF to toward a realistic and meaningful decomposition, increasing the case where the data matrix and the dictionary have miss- the quality of the reconstruction of the missing data. In this ing entries. After a formulation of the problem in this context contribution, we also consider the case where the dictionary of missing data, we propose a majorization-minimization al- may have missing coefficients itself. gorithm for the solving of the optimization problem incurred. Experimental results with synthetic data and audio spectro- The paper is organized as follows. Section 2 formulates grams highlight an improvement of the performance of re- CNMF in the presence of missing entries in the data ma- construction with respect to standard NMF. The performance trix and in the dictionary. Section 3 describes the proposed gap is particularly significant when the task of reconstruction majorization-minimization (MM) algorithm. Sections 4 and 5 becomes arduous, e.g. when the ratio of missing data is high, report experimental results with synthetic data and audio the noise is steep, or the complexity of data is high. spectrograms. Index Terms— matrix factorization, nonnegativity, low- rankness, matrix completion, spectrogram inpainting 2. CONVEX NONNEGATIVE MATRIX FACTORIZATION WITH MISSING DATA 1. INTRODUCTION 2.1. Notations and definitions Convex NMF (CNMF) [1] is a special case of nonnegative For any integer N, the integer set t1; 2;:::;Nu is denoted by matrix factorization (NMF) [2], in which the matrix of com- rNs. The coefficients of a matrix A P RMˆN are denoted ponents is constrained to be a linear combination of atoms by either amn or rAsmn. The element-wise matrix product, A of a known dictionary. The term “convex” refers to the con- matrix division and matrix power are denoted by A:B, B straint of the linear combination, where the combination co- and A.γ , respectively where A and B are matrices with same efficients forming each component are nonnegative and sum dimensions and γ is a scalar. 0 and 1 denote vectors or ma- to 1. Compared to the fully unsupervised NMF setting, the trices composed of zeros and ones, respectively, with dimen- use of known atoms is a source of supervision that may guide sions that can be deduced from the context. The element-wise learning based on this additional data: in particular, an inter- negation of a binary matrix M is denoted by M¯ fi 1 ´ M. esting case of CNMF consists in auto-encoding the data them- selves, by defining the atoms as the data matrix. CNMF has 2.2. NMF and Convex NMF been of interest in a number of contexts, such as clustering, F ˆN data analysis, face recognition, or music transcription [1, 3]. NMF consists in approximating a data matrix V P R` as It is also related to the self-expressive dictionary-based repre- F ˆK the product WH of two nonnegative matrices W P R` sentation proposed in [4]. KˆN and H P R` . Often, K ă min pF; Nq, such that WH An issue that has not yet been addressed is when the is a low-rank approximation of V. Every sample vn, the n- data matrix has missing coefficients. Such an extension of th column of V, is thus decomposed as a linear combination CNMF is worth being considered, as it opens the way to F of K elementary components or patterns w1;:::; wK P R`, data-reconstruction settings with nonnegative low-rank con- the columns of W. The coefficients of the linear combination straints, which covers several relevant applications. One are given by the n-th column hn of H. ˚This work was supported by ANR JCJC program MAD (ANR- 14- In [9] and [10], algorithms have been proposed for the un- CE27-0002). supervised estimation of W and H from V, by minimization 978-1-5090-0746-2/16/$31.00 c 2016 IEEE of the cost function Dβ pV|WHq “ fn dβ pvfn|rWHsfnq, Algorithm 1 CNMF with missing data o where dβ px|yq is the β-divergence defined as: Require: V, S , M , M , β ř V S Initialize S, L, H with random nonnegative values 1 β β β´1 βpβ´1q x ` pβ ´ 1q y ´ βxy loop Update S: $ ` for β P Rz t0; 1˘u dβ px|yq fi (1) ' x o 'x log y ´ x ` y for β “ 1 S Ð M :S ` (5) &' S x ´ log x ´ 1 for β “ 0 .γpβq y y M : pSLHq:pβ´2q :V pLHqT ' M :S: V ' S ´ :pβ´1q ¯ T When ill-defined,% we set by convention dβp0|0q “ 0. ¨ M : pSLHq pLHq ˛ V CNMF is a variant of NMF in which W “ SL. S “ ˝ ´ ¯ ‚ F ˆP rs1;:::; sP s P R` is a nonnegative matrix of atoms and Update L: P ˆK L “ rl1;:::; lK s P R` is the so-called labeling matrix. .γpβq T :pβ´2q T Each dictionary element wk is thus equal to Slk, with usu- S M : pSLHq :V H L Ð L: V (6) ally P K, and the data is in the end decomposed as :pβ´1q ¡¡ ¨ ST´ M : pSLHq ¯HT ˛ V “ SLH. The scale indeterminacy between L and H may V ˝ ´ ¯ ‚ be lifted by imposing }lk}1 “ 1, in which case wk is pre- Update H: cisely a convex combination of the elements of the subspace .γpβq S. CNMF can be related to the so-called archetypal analysis pSLqT M : pSLHq:pβ´2q :V [11], but without considering any nonnegativity constraint. H Ð H: V (7) ¨ pSLqT´ M : pSLHq:pβ´1q ¯ ˛ The use of known examples in S can then be seen as a V source of supervision that guides learning. A special case of ˝ ´ ¯ ‚ CNMF is obtained by setting S “ V, thus auto-encoding Rescale L and H: the data as VLH. This particular case is studied in depth @k P rKs ; h Ð}l } ˆ h (8) in [1]. In this paper, we consider the general case for S, with k k 1 k or without missing data. lk lk Ð (9) }lk}1 2.3. Convex NMF with missing data end loop We assume that some coefficients in V and S may be missing. return S, L, H Let Ă rF s ˆ rNs be a set of pairs of indices that locates V the observed coefficients in V: pf; nq P iff vfn is known. Similarly, let Ă rF s ˆ rP s be a set of pairsV of indices that 3. PROPOSED ALGORITHM locates the observedS coefficients in S. The use of sets and may be reformulated equivalently by defining maskingV ma- 3.1. General description of the algorithm S F ˆN F ˆP trices MV P t0; 1u and MS P t0; 1u from and Algorithm 1 extends the algorithm proposed in [9] for com- as V S plete CNMF with the β-divergence to the case of missing en- tries in V or S. The algorithm is a block-coordinate descent 1 if pf; nq P procedure in which each block is one the three matrix factors. rMV sfn fi V @pf; nq P rF s ˆ rNs (2) #0 otherwise The updates of each block/factor is obtained via majorization- minimization (MM), a classic procedure that consists in itera- 1 if pf; pq P tively minimizing a tight upper bound (called auxiliary func- rMS sfp fi S @pf; pq P rF s ˆ rP s (3) #0 otherwise tion) of the objective function. In the present setting, the MM procedure leads to multiplicative updates, characteristic A major goal in this paper is to estimate L, H and the of many NMF algorithms, that automatically preserve non- missing entries in S, given the partially observed data matrix negativity given positive initialization. V. Denoting by So the set of observed/known dictionary ma- trix coefficients, our aim is to minimize the objective function 3.2. Detailed updates C pS; L; Hq fi Dβ pMV :V|MV :SLHq (4) We consider the optimization of C pS; L; Hq with respect to each of its three arguments individually, using MM.
Load more

Recommended publications
  • Imputets: Time Series Missing Value Imputation in R by Steffen Moritz and Thomas Bartz-Beielstein
    CONTRIBUTED RESEARCH ARTICLE 1 imputeTS: Time Series Missing Value Imputation in R by Steffen Moritz and Thomas Bartz-Beielstein Abstract The imputeTS package specializes on univariate time series imputation. It offers multiple state-of-the-art imputation algorithm implementations along with plotting functions for time series missing data statistics. While imputation in general is a well-known problem and widely covered by R packages, finding packages able to fill missing values in univariate time series is more complicated. The reason for this lies in the fact, that most imputation algorithms rely on inter-attribute correlations, while univariate time series imputation instead needs to employ time dependencies. This paper provides an introduction to the imputeTS package and its provided algorithms and tools. Furthermore, it gives a short overview about univariate time series imputation in R. Introduction In almost every domain from industry (Billinton et al., 1996) to biology (Bar-Joseph et al., 2003), finance (Taylor, 2007) up to social science (Gottman, 1981) different time series data are measured. While the recorded datasets itself may be different, one common problem are missing values. Many analysis methods require missing values to be replaced with reasonable values up-front. In statistics this process of replacing missing values is called imputation. Time series imputation thereby is a special sub-field in the imputation research area. Most popular techniques like Multiple Imputation (Rubin, 1987), Expectation-Maximization (Dempster et al., 1977), Nearest Neighbor (Vacek and Ashikaga, 1980) and Hot Deck (Ford, 1983) rely on inter-attribute correlations to estimate values for the missing data. Since univariate time series do not possess more than one attribute, these algorithms cannot be applied directly.
    [Show full text]
  • Uncertainty-Aware Variational-Recurrent Imputation Network for Clinical Time Series Ahmad Wisnu Mulyadi, Eunji Jun, and Heung-Il Suk, Member, IEEE
    UNDER REVIEW 1 Uncertainty-Aware Variational-Recurrent Imputation Network for Clinical Time Series Ahmad Wisnu Mulyadi, Eunji Jun, and Heung-Il Suk, Member, IEEE Abstract—Electronic health records (EHR) consist of longitudi- data yields poor performance at high missing rates and also nal clinical observations portrayed with sparsity, irregularity, and requires modeling separately for the distinct dataset. In fact, high-dimensionality, which become major obstacles in drawing those missing values reflect the decisions made by health-care reliable downstream clinical outcomes. Although there exist great numbers of imputation methods to tackle these issues, most of providers [5]. Therefore, the missing values and their patterns them ignore correlated features, temporal dynamics and entirely contain information regarding a patient’s health status [5] and set aside the uncertainty. Since the missing value estimates involve correlate with the outcomes or target labels [6]. Thus, we the risk of being inaccurate, it is appropriate for the method to resort to the imputation approach by exploiting those missing handle the less certain information differently than the reliable patterns to improve the prediction of the clinical outcomes as data. In that regard, we can use the uncertainties in estimating the missing values as the fidelity score to be further utilized to the downstream task. alleviate the risk of biased missing value estimates. In this work, There exist numerous strategies for imputing missing values we propose a novel variational-recurrent imputation network, in the literature. Generally, imputation methods can be clas- which unifies an imputation and a prediction network by taking sified into a deterministic or stochastic approach, depending into account the correlated features, temporal dynamics, as well on the use of randomness [7].
    [Show full text]
  • Potential Sources of Missing Data in a Meta-Analysis
    SMG advanced workshop, Cardiff, March 4-5, 2010 PotentialPotential sourcessources ofof missingmissing datadata inin aa meta-analysismeta-analysis Missing data • Studies not found • Outcome not reported Julian Higgins • Outcome partially reported (e.g. missing SD or SE) MRC Biostatistics Unit Cambridge, UK • Extra information required for meta-analysis (e.g. imputed correlation coefficient) with thanks to Ian White, Fred Wolf, Angela Wood, Alex Sutton • Missing participants • No information on study characteristic for heterogeneity analysis ConceptsConcepts inin missingmissing datadata ConceptsConcepts inin missingmissing datadata 1. ‘Missing completely at random’ 2. ‘Missing at random’ • As if missing observation was randomly picked • Missingness depends on things you know about, and • Missingness unrelated to the true (missing) value not on the missing data themselves • e.g. – (genuinely) accidental deletion of a file •e.g. – observations measured on a random sample – older people more likely to drop out (irrespective of – statistics not reported due to ignorance of their the effect of treatment) importance(?) – recent cluster trials more likely to report intraclass correlation • OK (unbiased) to analyse just the data available, but – experimental treatment more likely to cause drop- sample size is reduced - which may not be ideal out (independent of therapeutic effect) • Not very common! • Usually surmountable StrategiesStrategies forfor dealingdealing withwith missingmissing datadata ConceptsConcepts inin missingmissing datadata •
    [Show full text]
  • Multiple Imputation: a Review of Practical and Theoretical Findings Jared S
    Submitted to Statistical Science Multiple Imputation: A Review of Practical and Theoretical Findings Jared S. Murray∗ University of Texas at Austin Abstract. Multiple imputation is a straightforward method for handling missing data in a principled fashion. This paper presents an overview of multiple imputation, including important theoretical results and their practical implications for generating and using multiple imputations. A review of strategies for generating imputations follows, including recent developments in flexible joint modeling and sequential regres- sion/chained equations/fully conditional specification approaches. Fi- nally, we compare and contrast different methods for generating impu- tations on a range of criteria before identifying promising avenues for future research. Key words and phrases: missing data, proper imputation, congeniality, chained equations, fully conditional specification, sequential regression multivariate imputation. 1. INTRODUCTION Multiple imputation (MI) (Rubin, 1987) is a simple but powerful method for dealing with missing data. MI as originally conceived proceeds in two stages: A data disseminator creates a small number of completed datasets by filling in the missing values with samples from an imputation model. Analysts compute their estimates in each completed dataset and combine them using simple rules to get pooled estimates and standard errors that incorporate the additional variability due to the missing data. MI was originally developed for settings in which statistical agencies or other data disseminators provide multiply imputed databases to distinct end-users. arXiv:1801.04058v1 [stat.ME] 12 Jan 2018 There are a number of benefits to MI in this setting: The disseminator can support approximately valid inference for a wide range of potential analyses with a small set of imputations, and the burden of dealing with the missing data is on the imputer rather than the analyst.
    [Show full text]
  • Within-Industry Productivity Dispersion and Imputation for Missing Data
    Imputation in U.S. Manufacturing Data and Implications for Within-Industry Productivity Dispersion ∗ T. Kirk Whitey, Jerome P. Reiter**, and Amil Petrin*** y Center for Economic Studies, U.S. Census Bureau ** Duke University *** University of Minnesota,Twin Cities and NBER Abstract In the U.S. Census Bureau’s 2002 and 2007 manufacturing data, respectively, 79% and 73% of observa­ tions have imputed data for at least one variable used to compute total factor productivity. The Bureau imputes for missing values using methods known to result in underestimation of variability and potential bias in multivariate inferences. We present an alternative strategy for handling the missing data based on multiple imputation via sequences of classification and regression trees. We use our imputations and the Bureau’s imputations to estimate within-industry productivity dispersions for every manufacturing industry. The results suggest that there is even more within-industry productivity dispersion in U.S. man­ ufacturing than previous research has indicated. We also estimate relationships between plant exit, pro­ ductivity, prices, and demand shocks. For these estimands, we find the results are substantively robust to using an alternative imputation strategy. ∗Some of the research in this paper was conducted while the first author was a Census Bureau employee. Any opinions and conclusions expressed herein are those of the authors and do not necessarily represent the views of the U.S. Census Bureau. All results have been reviewed to ensure that no confidential information is disclosed. Corresponding author: T. Kirk White. Email: [email protected]. 1 1 Introduction Nearly all economic surveys suffer from item nonresponse, i.e., respondents answer some questions but not others.
    [Show full text]
  • Dealing with Missing Standard Deviation and Mean Values in Meta-Analysis of Continuous Outcomes: a Systematic Review Christopher J
    Weir et al. BMC Medical Research Methodology (2018) 18:25 https://doi.org/10.1186/s12874-018-0483-0 RESEARCHARTICLE Open Access Dealing with missing standard deviation and mean values in meta-analysis of continuous outcomes: a systematic review Christopher J. Weir1*, Isabella Butcher1, Valentina Assi1, Stephanie C. Lewis1, Gordon D. Murray1, Peter Langhorne2 and Marian C. Brady3 Abstract Background: Rigorous, informative meta-analyses rely on availability of appropriate summary statistics or individual participant data. For continuous outcomes, especially those with naturally skewed distributions, summary information on the mean or variability often goes unreported. While full reporting of original trial data is the ideal, we sought to identify methods for handling unreported mean or variability summary statistics in meta-analysis. Methods: We undertook two systematic literature reviews to identify methodological approaches used to deal with missing mean or variability summary statistics. Five electronic databases were searched, in addition to the Cochrane Colloquium abstract books and the Cochrane Statistics Methods Group mailing list archive. We also conducted cited reference searching and emailed topic experts to identify recent methodological developments. Details recorded included the description of the method, the information required to implement the method, any underlying assumptions and whether the method could be readily applied in standard statistical software. We provided a summary description of the methods identified, illustrating selected methods in example meta-analysis scenarios. Results: For missing standard deviations (SDs), following screening of 503 articles, fifteen methods were identified in addition to those reported in a previous review. These included Bayesian hierarchical modelling at the meta-analysis level; summary statistic level imputation based on observed SD values from other trials in the meta-analysis; a practical approximation based on the range; and algebraic estimation of the SD based on other summary statistics.
    [Show full text]
  • Missing Value Imputation Using Feature-Specific Generative
    IFGAN: Missing Value Imputation using Feature-specific Generative Adversarial Networks Wei Qiu* Yangsibo Huang* Quanzheng Li Yuanpei College College of Computer Science and Technology Department of Radiology Peking University Zhejiang University Massachusetts General Hospital Beijing, China Hangzhou, China Boston, MA [email protected] [email protected] [email protected] Abstract—Missing value imputation is a challenging and well- established state-of-the-art performance in many applications. researched topic in data mining. In this paper, we propose When it comes to the task of imputation, deep architectures IFGAN, a missing value imputation algorithm based on Feature- have also shown greater promise than classical ones in recent specific Generative Adversarial Networks (GAN). Our idea is intuitive yet effective: a feature-specific generator is trained studies [7]–[9], due to their capability to automatically learn to impute missing values, while a discriminator is expected to latent representations and inter-variable associations. Nonethe- distinguish the imputed values from observed ones. The proposed less, these work all formulated data imputation as a mapping architecture is capable of handling different data types, data from a missing matrix to a complete one, which ignores the distributions, missing mechanisms, and missing rates. It also difference in data type and statistical importance between improves post-imputation analysis by preserving inter-feature correlations. We empirically show on several real-life datasets features and makes the model unable to learn feature-specific that IFGAN outperforms current state-of-the-art algorithm un- representations. der various missing conditions. To overcome the limitations of classical models and to Index Terms—Data preprocessing, Missing value imputation, incorporate feature-specific representations into deep learning Deep adversarial network frameworks, we propose IFGAN, a feature-specific generative adversarial network for missing data imputation.
    [Show full text]
  • Should a Normal Imputation Model Be Modified to Impute Skewed Variables?
    Sociological Methods and Research, 2013, 42(1), 105-138 Should a Normal Imputation Model Be Modified to Impute Skewed Variables? Paul T. von Hippel Abstract (169 words) Researchers often impute continuous variables under an assumption of normality, yet many incomplete variables are skewed. We find that imputing skewed continuous variables under a normal model can lead to bias; the bias is usually mild for popular estimands such as means, standard deviations, and linear regression coefficients, but the bias can be severe for more shape- dependent estimands such as percentiles or the coefficient of skewness. We test several methods for adapting a normal imputation model to accommodate skewness, including methods that transform, truncate, or censor (round) normally imputed values, as well as methods that impute values from a quadratic or truncated regression. None of these modifications reliably reduces the biases of the normal model, and some modifications can make the biases much worse. We conclude that, if one has to impute a skewed variable under a normal model, it is usually safest to do so without modifications—unless you are more interested in estimating percentiles and shape that in estimated means, variance, and regressions. In the conclusion, we briefly discuss promising developments in the area of continuous imputation models that do not assume normality. Key words: missing data; missing values; incomplete data; regression; transformation; normalization; multiple imputation; imputation Paul T. von Hippel is Assistant Professor, LBJ School of Public Affairs, University of Texas, 2315 Red River, Box Y, Austin, TX 78712, [email protected]. I thank Ray Koopman for help with the calculations in Section 2.2.
    [Show full text]
  • Supervised Nonnegative Matrix Factorization to Predict ICU
    Supervised Nonnegative Matrix Factorization to Predict ICU Mortality Risk Guoqing Chao Chengsheng Mao Fei Wang Yuan Zhao Feinberg School of Medicine Feinberg School of Medicine Weill Cornell Medicine Feinberg School of Medicine Northwestern University Northwestern University Cornell University Northwestern University Chicago, U.S Chicago, U.S New York, U.S Chicago, U.S [email protected] [email protected] [email protected] [email protected] Yuan Luo Feinberg School of Medicine Northwestern University Chicago, U.S [email protected] Abstract—ICU mortality risk prediction is a tough yet im- In this paper, we describe a supervised nonnegative ma- portant task. On one hand, due to the complex temporal data trix factorization (NMF) algorithm that performs predictive collected, it is difficult to identify the effective features and modeling by the exploration of atomic and the higher-order interpret them easily; on the other hand, good prediction can help clinicians take timely actions to prevent the mortality. These features jointly. We automate the mining of higher-order fea- correspond to the interpretability and accuracy problems. Most tures by converting data into graph representation. Moreover, existing methods lack of the interpretability, but recently Sub- supervised latent group identification reduces dimensionality graph Augmented Nonnegative Matrix Factorization (SANMF) of different feature types for patients, and simultaneously has been successfully applied to time series data to provide a group temporal trends to form effective features for patient path to interpret the features well. Therefore, we adopted this approach as the backbone to analyze the patient data. One outcome prediction. Applications on patient physiological time limitation of the original SANMF method is its poor prediction series [1] show significant performance improvements from ability due to its unsupervised nature.
    [Show full text]
  • Optimal Recovery of Missing Values for Non-Negative Matrix Factorization Rebecca Chen and Lav R
    bioRxiv preprint doi: https://doi.org/10.1101/647560; this version posted May 24, 2019. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 1 Optimal Recovery of Missing Values for Non-negative Matrix Factorization Rebecca Chen and Lav R. Varshney Abstract We extend the approximation-theoretic technique of optimal recovery to the setting of imputing missing values in clustered data, specifically for non-negative matrix factorization (NMF), and develop an implementable algorithm. Under certain geometric conditions, we prove tight upper bounds on NMF relative error, which is the first bound of this type for missing values. We also give probabilistic bounds for the same geometric assumptions. Experiments on image data and biological data show that this theoretically-grounded technique performs as well as or better than other imputation techniques that account for local structure. Index Terms imputation, missing values, non-negative matrix factorization, optimal recovery I. INTRODUCTION ATRIX factorization is commonly used for clustering and dimensionality reduction in computational biology, imaging, M and other fields. Non-negative matrix factorization (NMF) is particularly favored by biologists because non-negativity constraints preclude negative values that are difficult to interpret in biological processes [2], [3]. NMF of gene expression matrices can discover cell groups and lower-dimensional manifolds in gene counts for different cell types. Due to physical and biological limitations of DNA- and RNA-sequencing techniques, however, gene-expression matrices are usually incomplete and matrix imputation is often necessary before further analysis [3].
    [Show full text]
  • From Predictive Methods to Missing Data Imputation: an Optimization Approach
    Journal of Machine Learning Research 18 (2018) 1-39 Submitted 2/17; Revised 3/18; Published 4/18 From Predictive Methods to Missing Data Imputation: An Optimization Approach Dimitris Bertsimas [email protected] Colin Pawlowski [email protected] Ying Daisy Zhuo [email protected] Sloan School of Management and Operations Research Center Massachusetts Institute of Technology Cambridge, MA 02139 Editor: Francis Bach Abstract Missing data is a common problem in real-world settings and for this reason has attracted significant attention in the statistical literature. We propose a flexible framework based on formal optimization to impute missing data with mixed continuous and categorical variables. This framework can readily incorporate various predictive models including K- nearest neighbors, support vector machines, and decision tree based methods, and can be adapted for multiple imputation. We derive fast first-order methods that obtain high quality solutions in seconds following a general imputation algorithm opt.impute presented in this paper. We demonstrate that our proposed method improves out-of-sample accuracy in large-scale computational experiments across a sample of 84 data sets taken from the UCI Machine Learning Repository. In all scenarios of missing at random mechanisms and various missing percentages, opt.impute produces the best overall imputation in most data sets benchmarked against five other methods: mean impute, K-nearest neighbors, iterative knn, Bayesian PCA, and predictive-mean matching, with an average reduction in mean absolute error of 8.3% against the best cross-validated benchmark method. Moreover, opt.impute leads to improved out-of-sample performance of learning algorithms trained using the imputed data, demonstrated by computational experiments on 10 downstream tasks.
    [Show full text]
  • Missing Data Imputation Using Optimal Transport
    Missing Data Imputation using Optimal Transport Boris Muzellec 1 Julie Josse 2 3 * Claire Boyer 4 Marco Cuturi 5 1 Abstract with plausible values, are very appealing as they allow to both get a guess for the missing entries as well as to per- Missing data is a crucial issue when applying ma- form (with care) downstream machine learning methods on chine learning algorithms to real-world datasets. the completed data. Efficient methods include, among oth- Starting from the simple assumption that two ers, methods based on low-rank assumptions (Hastie et al., batches extracted randomly from the same dataset 2015), iterative random forests (Stekhoven & Buhlmann, should share the same distribution, we leverage 2011) and imputation using variational autoencoders (Mattei optimal transport distances to quantify that crite- & Frellsen, 2019; Ivanov et al., 2019). A desirable prop- rion and turn it into a loss function to impute miss- erty for imputation methods is that they should preserve the ing data values. We propose practical methods to joint and marginal distributions of the data. Non-parametric minimize these losses using end-to-end learning, Bayesian strategies (Murray & Reiter, 2016) or recent ap- that can exploit or not parametric assumptions proaches based on generative adversarial networks (Yoon on the underlying distributions of values. We et al., 2018) are attempts in this direction. However, they evaluate our methods on datasets from the UCI can be quite cumbersome to implement in practice. repository, in MCAR, MAR and MNAR settings. These experiments show that OT-based methods We argue in this work that the optimal transport (OT) tool- match or out-perform state-of-the-art imputation box constitutes a natural, sound and straightforward alter- methods, even for high percentages of missing native.
    [Show full text]