DOCSLIB.ORG

Python Data Science Essentials Table of Contents

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Python Data Science Essentials Table of Contents Python Data Science Essentials Table of Contents Python Data Science Essentials Credits About the Authors About the Reviewers www.PacktPub.com Support files, eBooks, discount offers, and more Why subscribe? Free access for Packt account holders Preface What this book covers What you need for this book Who this book is for Conventions Reader feedback Customer support Downloading the example code Errata Piracy Questions 1. First Steps Introducing data science and Python Installing Python Python 2 or Python 3? Step-by-step installation A glance at the essential Python packages NumPy SciPy pandas Scikit-learn IPython Matplotlib Statsmodels Beautiful Soup NetworkX NLTK Gensim PyPy The installation of packages Package upgrades Scientific distributions Anaconda Enthought Canopy PythonXY WinPython Introducing IPython The IPython Notebook Datasets and code used in the book Scikit-learn toy datasets The MLdata.org public repository LIBSVM data examples Loading data directly from CSV or text files Scikit-learn sample generators Summary 2. Data Munging The data science process Data loading and preprocessing with pandas Fast and easy data loading Dealing with problematic data Dealing with big datasets Accessing other data formats Data preprocessing Data selection Working with categorical and textual data A special type of data – text Data processing with NumPy NumPy's n-dimensional array The basics of NumPy ndarray objects Creating NumPy arrays From lists to unidimensional arrays Controlling the memory size Heterogeneous lists From lists to multidimensional arrays Resizing arrays Arrays derived from NumPy functions Getting an array directly from a file Extracting data from pandas NumPy fast operation and computations Matrix operations Slicing and indexing with NumPy arrays Stacking NumPy arrays Summary 3. The Data Science Pipeline Introducing EDA Feature creation Dimensionality reduction The covariance matrix Principal Component Analysis (PCA) A variation of PCA for big data – RandomizedPCA Latent Factor Analysis (LFA) Linear Discriminant Analysis (LDA) Latent Semantical Analysis (LSA) Independent Component Analysis (ICA) Kernel PCA Restricted Boltzmann Machine (RBM) The detection and treatment of outliers Univariate outlier detection EllipticEnvelope OneClassSVM Scoring functions Multilabel classification Binary classification Regression Testing and validating Cross-validation Using cross-validation iterators Sampling and bootstrapping Hyper-parameters' optimization Building custom scoring functions Reducing the grid search runtime Feature selection Univariate selection Recursive elimination Stability and L1-based selection Summary 4. Machine Learning Linear and logistic regression Naive Bayes The k-Nearest Neighbors Advanced nonlinear algorithms SVM for classification SVM for regression Tuning SVM Ensemble strategies Pasting by random samples Bagging with weak ensembles Random Subspaces and Random Patches Sequences of models – AdaBoost Gradient tree boosting (GTB) Dealing with big data Creating some big datasets as examples Scalability with volume Keeping up with velocity Dealing with variety A quick overview of Stochastic Gradient Descent (SGD) A peek into Natural Language Processing (NLP) Word tokenization Stemming Word Tagging Named Entity Recognition (NER) Stopwords A complete data science example – text classification An overview of unsupervised learning Summary 5. Social Network Analysis Introduction to graph theory Graph algorithms Graph loading, dumping, and sampling Summary 6. Visualization Introducing the basics of matplotlib Curve plotting Using panels Scatterplots Histograms Bar graphs Image visualization Selected graphical examples with pandas Boxplots and histograms Scatterplots Parallel coordinates Advanced data learning representation Learning curves Validation curves Feature importance GBT partial dependence plot Summary Index Python Data Science Essentials Python Data Science Essentials Copyright © 2015 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: April 2015 Production reference: 1240415 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78528-042-9 www.packtpub.com Credits Authors Alberto Boschetti Luca Massaron Reviewers Robert Dempsey Daniel Frimer Kevin Markham Alberto Gonzalez Paje Bastiaan Sjardin Michele Usuelli Zacharias Voulgaris, PhD Commissioning Editor Julian Ursell Acquisition Editor Subho Gupta Content Development Editor Merwyn D'souza Technical Editor Namrata Patil Copy Editor Vedangi Narvekar Project Coordinator Neha Bhatnagar Proofreaders Simran Bhogal Faye Coulman Safis Editing Dan McMahon Indexer Priya Sane Production Coordinator Komal Ramchandani Cover Work Komal Ramchandani About the Authors Alberto Boschetti is a data scientist with expertise in signal processing and statistics. He holds a PhD in telecommunication engineering and currently lives and works in London. In his work projects, he faces challenges involving natural language processing (NLP), machine learning, and probabilistic graph models everyday. He is very passionate about his job and he always tries to stay updated on the latest developments in data science technologies by attending meetups, conferences, and other events. I would like to thank my family, my friends, and my colleagues. Also, a big thanks to the open source community. Luca Massaron is a data scientist and marketing research director who specializes in multivariate statistical analysis, machine learning, and customer insight, with over a decade of experience in solving real-world problems and generating value for stakeholders by applying reasoning, statistics, data mining, and algorithms. From being a pioneer of web audience analysis in Italy to achieving the rank of a top 10 Kaggler, he has always been passionate about everything regarding data and analysis and about demonstrating the potentiality of data-driven knowledge discovery to both experts and nonexperts. Favoring simplicity over unnecessary sophistication, he believes that a lot can be achieved in data science by understanding its essentials. To Yukiko and Amelia, for their loving patience. "Roads go ever ever on, under cloud and under star, yet feet that wandering have gone turn at last to home afar". About the Reviewers Robert Dempsey is an experienced leader and technology professional specializing in delivering solutions and products to solve tough business challenges. His experience in forming and leading agile teams, combined with more than 14 years of experience in the field of technology, enables him to solve complex problems while always keeping the bottom line in mind. Robert has founded and built three start-ups in technology and marketing, developed and sold two online applications, consulted Fortune 500 and Inc. 500 companies, and spoken nationally and internationally on software development and agile project management. He is currently the head of data operations at ARPC, an econometrics firm based in Washington, DC. In addition, he's the founder of Data Wranglers DC, a group dedicated to improving the craft of data wrangling, as well as a board member of Data Community DC. In addition to spending time with his growing family, Robert geeks out on Raspberry Pis and Arduinos and automates most of his life with the help of hardware and software. Daniel Frimer has been an advocate for the Python language for 2 years now. With a degree in applied and computational math sciences from the University of Washington, he has spearheaded various automation projects in the Python language involving natural language processing, data munging, and web scraping. In his side projects, he has dived into a deep analysis of NFL and NBA player statistics for his fantasy sports teams. Daniel has recently started working in SaaS at a private company for online health insurance shopping called Array Health, in support of day-to-day data analysis and the perfection of the integration between consumers, employers, and insurers. He has also worked with data-centric teams at Amazon, Starbucks, and Atlas International. Kevin Markham is a computer engineer, a data science instructor for General Assembly in Washington, DC, and the cofounder of Causetown, an online cause marketing platform for small businesses. He is passionate about teaching data science and machine learning and enjoys both Python and R. He founded Data School (http://dataschool.io) in order to provide in-depth educational resources that are accessible to data science novices. He has an active YouTube channel (http://youtube.com/dataschool) and can also be found on Twitter (@justmarkham). Alberto Gonzalez Paje is an economist specializing
Load more

Recommended publications
  • Practical Machine Learning with Python
    Practical Machine Learning with Python A Problem-Solver’s Guide to Building Real-World Intelligent Systems — Dipanjan Sarkar Raghav Bali Tushar Sharma Practical Machine Learning with Python A Problem-Solver’s Guide to Building Real-World Intelligent Systems Dipanjan Sarkar Raghav Bali Tushar Sharma Practical Machine Learning with Python Dipanjan Sarkar Raghav Bali Bangalore, Karnataka, India Bangalore, Karnataka, India Tushar Sharma Bangalore, Karnataka, India ISBN-13 (pbk): 978-1-4842-3206-4 ISBN-13 (electronic): 978-1-4842-3207-1 https://doi.org/10.1007/978-1-4842-3207-1 Library of Congress Control Number: 2017963290 Copyright © 2018 by Dipanjan Sarkar, Raghav Bali and Tushar Sharma This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.
    [Show full text]
  • A Dataset Simulation Package for Strong Gravitational Lensing
    deeplenstronomy: A dataset simulation package for strong gravitational lensing Robert Morgan∗1, 2, Brian Nord3, 4, Simon Birrer5, Joshua Yao-Yu Lin6, and Jason Poh4 1 University of Wisconsin-Madison 2 Legacy Survey of Space and Time Data Science Fellowship Program 3 Fermi National Accelerator Laboratory 4 University of Chicago 5 Stanford University 6 DOI: 10.21105/joss.02854 University of Illinois Urbana-Champaign Software • Review Background • Repository • Archive Astronomical observations and statistical modeling permit the high-fidelity analysis of strong gravitational lensing (SL) systems, which display an astronomical phenomenon in Submitted: 10 November 2020 which light from a distant object is deflected by the gravitational field of another object Published: 04 February 2021 along its path to the observer. These systems are of great scientific interest because they License provide information about multiple astrophysical and cosmological phenomena, including Authors of papers retain copyright the nature of dark matter, the expansion rate of the Universe, and characteristics of galaxy and release the work under a Cre- populations. They also serve as standing tests of the theory of General Relativity and ative Commons Attribution 4.0 In- modified theories of gravity. ternational License (CC BY 4.0). Traditional searches for SL systems have involved time- and effort-intensive visual or manual inspection of images by humans to identify characteristic features — like arcs, particular color combinations, and object orientations. However, a comprehensive search using the traditional approach is prohibitively expensive for large numbers of images, like those in cosmological surveys — e.g., the Sloan Digital Sky Survey (York et al. 2000), the Dark Energy Survey (Abbott et al.
    [Show full text]
  • A Python Toolbox for Ensemble Learning and Visualisation Benjamin Guedj, Bhargav Srinivasa Desikan
    Pycobra: A Python Toolbox for Ensemble Learning and Visualisation Benjamin Guedj, Bhargav Srinivasa Desikan To cite this version: Benjamin Guedj, Bhargav Srinivasa Desikan. Pycobra: A Python Toolbox for Ensemble Learning and Visualisation. Journal of Machine Learning Research, Microtome Publishing, 2018, 18, pp.1 - 5. hal-01514059v3 HAL Id: hal-01514059 https://hal.inria.fr/hal-01514059v3 Submitted on 20 Jun 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Distributed under a Creative Commons Attribution - NonCommercial - ShareAlike| 4.0 International License Journal of Machine Learning Research 18 (2018) 1-5 Submitted 4/17; Revised 3/18; Published 4/18 Pycobra: A Python Toolbox for Ensemble Learning and Visualisation Benjamin Guedj [email protected] Modal project-team, Lille - Nord Europe research center Inria, France Bhargav Srinivasa Desikan [email protected] Modal project-team, Lille - Nord Europe research center Inria, France Editor: Geoff Holmes Abstract We introduce pycobra, a Python library devoted to ensemble learning (regression and clas- sification) and visualisation. Its main assets are the implementation of several ensemble learning algorithms, a flexible and generic interface to compare and blend any existing machine learning algorithm available in Python libraries (as long as a predict method is given), and visualisation tools such as Voronoi tessellations.
    [Show full text]
  • A Python Toolbox for Ensemble Learning and Visualisation
    Journal of Machine Learning Research 18 (2018) 1-5 Submitted 4/17; Revised 3/18; Published 4/18 Pycobra: A Python Toolbox for Ensemble Learning and Visualisation Benjamin Guedj [email protected] Modal project-team, Lille - Nord Europe research center Inria, France Bhargav Srinivasa Desikan [email protected] Modal project-team, Lille - Nord Europe research center Inria, France Editor: Geoff Holmes Abstract We introduce pycobra, a Python library devoted to ensemble learning (regression and clas- sification) and visualisation. Its main assets are the implementation of several ensemble learning algorithms, a flexible and generic interface to compare and blend any existing machine learning algorithm available in Python libraries (as long as a predict method is given), and visualisation tools such as Voronoi tessellations. pycobra is fully scikit-learn compatible and is released under the MIT open-source license. pycobra can be down- loaded from the Python Package Index (PyPi) and Machine Learning Open Source Software (MLOSS). The current version (along with Jupyter notebooks, extensive documentation, and continuous integration tests) is available at https://github.com/bhargavvader/pycobra and official documentation website is https://modal.lille.inria.fr/pycobra. Keywords: ensemble methods, machine learning, Voronoi tesselation, Python, open source software 1. Introduction Combined statistical procedures, also known as ensemble or aggregation methods, are very popular in Machine Learning competitions { the celebrated Netflix Challenge (as are re- peatedly Kaggle competitions) was won by an ensemble technique (see for example Bell and Koren, 2007, for a discussion). In a nutshell, ensemble methods combine different predictors (each trained on the same dataset) to improve performance.
    [Show full text]