Mastering Python High Performance
Total Page:16
File Type:pdf, Size:1020Kb
[ 1 ] www.allitebooks.com Mastering Python High Performance Measure, optimize, and improve the performance of your Python code with this easy-to-follow guide Fernando Doglio BIRMINGHAM - MUMBAI www.allitebooks.com Mastering Python High Performance Copyright © 2015 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: September 2015 Production reference: 1030915 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78398-930-0 www.packtpub.com www.allitebooks.com Credits Author Project Coordinator Fernando Doglio Milton Dsouza Reviewers Proofreader Erik Allik Safis Editing Mike Driscoll Enrique Escribano Indexer Mariammal Chettiyar Mosudi Isiaka Graphics Commissioning Editor Sheetal Aute Kunal Parikh Production Coordinator Acquisition Editors Arvindkumar Gupta Vivek Anantharaman Richard Brookes-Bland Cover Work Arvindkumar Gupta Content Development Editors Akashdeep Kundu Rashmi Suvarna Technical Editor Vijin Boricha Copy Editors Relin Hedly Karuna Narayanan www.allitebooks.com About the Author Fernando Doglio has been working as a web developer for the past 10 years. During that time, he shifted his focus to the Web and grabbed the opportunity of working with most of the leading technologies, such as PHP, Ruby on Rails, MySQL, Python, Node.js, AngularJS, AJAX, REST APIs, and so on. In his spare time, Fernando likes to tinker and learn new things. This is why his GitHub account keeps getting new repos every month. He's also a big open source supporter and tries to win the support of new people with the help of his website, lookingforpullrequests.com. You can reach him on Twitter at @deleteman123. When he is not programming, he spends time with his family. I'd like to thank my lovely wife for putting up with me and the long hours I spent writing this book; this book would not have been possible without her continued support. I would also like to thank my two sons. Without them, this book would've been finished months earlier. Finally, I'd like to thank the reviewers and editors. They helped me get this book in shape and achieve the quality level that you deserve. www.allitebooks.com About the Reviewers Erik Allik is a self-taught multilingual, multiparadigm full-stack software engineer. He started programming at the age of 14. Since then, Erik has been working with many programming languages (both imperative and functional) and various web and non-web-related technologies. He has worked primarily with Python, Scala, and JavaScript. Erik is currently focusing on applying Haskell and other innovative functional programming techniques in various industries and leveraging the power of a mathematical approach and formalism in the wild. www.allitebooks.com Mike Driscoll has been programming in Python since 2006. He enjoys writing about Python on his blog at http://www.blog.pythonlibrary.org/. Mike has coauthored Core Python refcard for DZone. He recently authored Python 101 and was a technical reviewer for the following books by Packt Publishing: • Python 3 Object-Oriented Programming • Python 2.6 Graphics Cookbook • Tkinter GUI Application Development Hotshot I would like to thank my beautiful wife, Evangeline, for supporting me throughout. I would also like to thank my friends and family for all their help. Also, thank you Jesus Christ for taking good care of me. www.allitebooks.com Enrique Escribano lives in Chicago and is working as a software engineer at Nokia. Although he is just 23 years old, he holds a master's of computer science degree from IIT (Chicago) and a master's of science degree in telecommunication engineering from ETSIT-UPM (Madrid). Enrique has also worked as a software engineer at KeepCoding and as a developer intern at Telefonica, SA, the most important Spanish tech company. He is an expert in Java and Python and is proficient in using C/C++. Most of his projects involve working with cloud-based technologies, such as AWS, GAE, Hadoop, and so on. Enrique is also working on an open source research project based on security with software-defined networking (SDN) with professor Dong Jin at IIT Security Lab. You can find more information about Enrique on his personal website at enriquescribano.com. You can also reach him on LinkedIn at linkedin.com/in/enriqueescribano. I would like to thank my parents, Lucio and Carmen, for all the unconditional support they have provided me with over the years. They allowed me to be as ambitious as I wanted. Without them, I may never have gotten to where I am today. I would like to thank my siblings, Francisco and Marta. Being the eldest brother is challenging, but you both keep inspiring me everyday. Lastly, I would also like to thank Paula for always being my main inspiration and motivation since the very first day. I am so fortunate to have her in my life. www.allitebooks.com Mosudi Isiaka is a graduate in electrical and computer engineering from the Federal University of Technology Minna, Niger State, Nigeria. He demonstrates excellent skills in numerous aspects of information and communication technology. From a simple network to a mid-level complex network scenario of no less than one thousand workstations (Microsoft Windows 7, Microsoft Windows Vista, and Microsoft Windows XP), along with a Microsoft Windows 2008 Server R2 Active Directory domain controller deployed in more than a single location, Mosudi has extensive experience in implementing and managing a local area network. He has successfully set up a data center infrastructure, VPN, WAN link optimization, firewall and intrusion detection system, web/e-mail hosting control panel, OpenNMS network management application, and so on. Mosudi has the ability to use open source software and applications to achieve enterprise-level network management solutions in scenarios that cover a virtual private network (VPN), IP PBX, cloud computing, clustering, virtualization, routing, high availability, customized firewall with advanced web filtering, network load balancing, failover and link aggregation for multiple Internet access solutions, traffic engineering, collaboration suits, network-attached storage (NAS), Linux systems administration, virtual networking and computing. He is currently employed as a data center manager at One Network Ltd., Nigeria. Mosudi also works with ServerAfrica(http://www.serverafrica.com) as a managing consultant (technicals). You can find more information about him at http://www.mioemi.com. You can also reach him at http://ng.linkedin.com/pub/isiaka-mosudi/1b/7a2/936/. I would like to thank my amiable wife, Mosudi Efundayo Coker, for her moral support. Also, many thanks to my colleague, Oyebode Micheal Tosin, for his timely reminders and technical suggestions during the reviewing process. www.allitebooks.com www.PacktPub.com Support files, eBooks, discount offers, and more For support files and downloads related to your book, please visit www.PacktPub.com. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks. TM https://www2.packtpub.com/books/subscription/packtlib Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books. Why subscribe? • Fully searchable across every book published by Packt • Copy and paste, print, and bookmark content • On demand and accessible via a web browser Free access for Packt account holders If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access. www.allitebooks.com Table of Contents Preface v Chapter 1: Profiling 101 1 What is profiling? 2 Event-based profiling 2 Statistical profiling 5 The importance of profiling 6 What can we profile? 8 Execution time 8 Where are the bottlenecks? 10 Memory consumption and memory leaks 11 The risk of premature optimization 15 Running time complexity 15 Constant time – O(1) 16 Linear time – O(n) 16 Logarithmic time – O(log n) 17 Linearithmic time – O(nlog n) 18 Factorial time – O(n!) 18 Quadratic time – O(n^) 19 Profiling best practices 22 Build a regression-test suite 22 Mind your code 22 Be patient 22 Gather as much data as you can 23 Preprocess