CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Algorithmic Challenges in the CPAMMS Project Wilfried Gansterer, Andreas Janecek Research Lab Computational Applications and Technologies, University of Vienna January 31, 2008 Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Outline 1 CPAMMS The CPAMMS Project CPAMMS - CHARMM 2 KD and DM 3 Feature Selection 4 Machine Learning 5 Efforts and Tools Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools The CPAMMS Project Aims Computing Paradigms and Algorithms for Molecular Modeling and Simulation Applications in Chemistry Molecular Biology Pharmacy Focus on Methodological questions in CS and SC Development of innovative methods and computational technologies and on their application Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools The CPAMMS Project Participating Institutes Faculty of Computer Science Research Lab Computational Technologies and Applications Institute of Scientific Computing Faculty of Chemistry Institute for Theoretical Chemistry Department of Biomolecular Structural Chemistry Faculty of Life Sciences Department of Medicinal Chemistry Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools The CPAMMS Project Research Efforts Middleware ⇒ Talk Sigi Benkner Applications Quantum Chemistry on Grids ⇒ Talk Mathias Ruckenbauer In-silico Screening ⇒ Talk Gerhard F. Ecker Distributed Molecular Dynamics Simulation ⇒ ... Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools CPAMMS - CHARMM CHARMM (Chemistry at HARvard Macromolecular Mechanics) General Scalable Molecular Dynamics Simulation and Analysis (evolved over 20+ years) Free energy calculation Replicated data model F77/F95 code + MPI Simulation studies of ionic liquids Long-term equilibrium simulation (>100ns) Force calculation via Particle Mesh Ewald System size: > 104 atoms Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools CPAMMS - CHARMM CHARMM - Platform & Activities Computing platform Sun Fire Cluster (72 Sun Fire X4100 = 288 Cores, Infiniband Interconnect) Ongoing work: Improving scalability Optimizing calculation of atomic interactions Optimal usage of MPI collective communication on Infiniband Comparison with other codes (NAMD, GROMACS) Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools CPAMMS - CHARMM Research Efforts Middleware ⇒ Talk Sigi Benkner Applications Quantum Chemistry on Grids ⇒ Talk Mathias Ruckenbauer In-silico Screening ⇒ Talk Gerhard F. Ecker Distributed Molecular Dynamics Simulation ⇒ ... Algorithms Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Outline 1 CPAMMS 2 KD and DM Predictive QSAR Modeling Steps in the Knowlegde Discovery Process 3 Feature Selection 4 Machine Learning 5 Efforts and Tools Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Predictive QSAR Modeling QSAR Quantitative Structure Activity Relationship Quantitative representations of molecular structures ...encoded in terms of information-preserving descriptor values Pharmacological or biological activity ⇒ Expression describing the beneficial or adverse effects of a drug in an organism Very general: Activity = f (physiochemical and/or structural properties) Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Predictive QSAR Modeling QSAR Workflow Descriptor calculation Chemical structure Descriptor numbers Classification or Regression Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Predictive QSAR Modeling Chemical Descriptors “A chemical descriptor is the final result of a logical and mathematical procedure which transforms chemical information encoded within a symbolic representation of a molecule into an useful number or the result of some standardized experiment.” 1 Physical-chemical properties Similarity Principle Compounds with similar chemical structures (i.e., descriptor similarity) usually possess similar physicochemical properties and biological activities 1 http://www.qsarworld.com/insilico-chemistry-chemical-descriptors.php Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Predictive QSAR Modeling Chemical Descriptors Huge amount of obtainable chemical descriptor (> 3 000) ⇒ 1D, 2D, 3D, 4D, molecular weight, volume, solubility, lipophilicity,. Descriptors are computed using structural codes ⇒ Example: SMILES Code Simplified Molecular Input Line Entry Specification ⇒ N(CCC=C1c2c(CCc3c1cccc3)cccc2)(C)C Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Steps in the Knowlegde Discovery Process Overview Feature Extraction Feature Selection Data Mining Post-processing Information Data Pre-processing Feature Collection Feature Subset Selection Supervised and Filtering Patterns Feature Computation Unsupervised Visualization Normalization Dimensionality Reduction Machine Learning Pattern Interpretation Discretization Algorithms ... Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Steps in the Knowlegde Discovery Process Feature Extraction Feature Extraction Feature Selection Data Mining Post-processing Information Data Pre-processing Feature Collection Feature Subset Selection Supervised and Filtering Patterns Feature Computation Unsupervised Visualization Normalization Dimensionality Reduction Machine Learning Pattern Interpretation Discretization Algorithms ... Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Steps in the Knowlegde Discovery Process Feature Extraction 1. Data collection Collection of pre-classified data ⇒ Literature, NCI (national cancer institute), . Collection of unclassified data ⇒ Compound libraries e.g., SPECS, ChemDiv, . 2. Extract structural code e.g., SMILES, .sdf, .mol, . 3. Input for software packages to compute descriptors Commercial examples: MOE, Adriana, Dragon Non-commercial examples: JOELib Self developed descriptors Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Steps in the Knowlegde Discovery Process Feature Extraction Normalization Descriptor values may have different scales Mean shifting + scaling ⇒ Mean = 0 ⇒ Standard deviation = 1 Discretization Process of transferring continuous numbers into discrete counterparts Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Steps in the Knowlegde Discovery Process Feature Selection Feature Extraction Feature Selection Data Mining Post-processing Information Data Pre-processing Feature Collection Feature Subset Selection Supervised and Filtering Patterns Feature Computation Unsupervised Visualization Normalization Dimensionality Reduction Machine Learning Pattern Interpretation Discretization Algorithms ... Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Steps in the Knowlegde Discovery Process Feature Selection Automatic Feature Selection (FS) and Dimensionality Reduction (DR) ⇒ See later Intuitive FS methods 2 Deletion of descriptors with low “information” content ⇒ E.g., Descriptors, that show more than 80% zero values Deletion of descriptors with low variance ⇒ variance ≤ 0.5 2 Huang, J., et al., Identifying P-Glycoprotein Substrates Using a Support Vector Machine Optimized by a Particle Swarm. J. Chem. Inf. Model., 2007. 47(4): p. 1638-1647. Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts and Tools Steps in the Knowlegde Discovery Process Data Mining Feature Extraction Feature Selection Data Mining Post-processing Information Data Pre-processing Feature Collection Feature Subset Selection Supervised and Filtering Patterns Feature Computation Unsupervised Visualization Normalization Dimensionality Reduction Machine Learning Pattern Interpretation Discretization Algorithms ... Wilfried Gansterer, Andreas Janecek Workshop on Campus Grids and Scientific Applications CPAMMS KD and DM Feature Selection Machine Learning Efforts