DOCSLIB.ORG

2.5 Python Virtual Machines

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
2.5 Python Virtual Machines Shannon, Mark (2011) The construction of high-performance virtual machines for dynamic languages. PhD thesis. http://theses.gla.ac.uk/2975/ Copyright and moral rights for this thesis are retained by the author A copy can be downloaded for personal non-commercial research or study, without prior permission or charge This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the Author The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the Author When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given Glasgow Theses Service http://theses.gla.ac.uk/ [email protected] The Construction of High-Performance Virtual Machines for Dynamic Languages Mark Shannon, MSc. Submitted for the Degree of Doctor of Philosophy School of Computing Science College of Science and Engineering University of Glasgow November 2011 2 Abstract Dynamic languages, such as Python and Ruby, have become more widely used over the past decade. Despite this, the standard virtual machines for these lan- guages have disappointing performance. These virtual machines are slow, not be- cause methods for achieving better performance are unknown, but because their implementation is hard. What makes the implementation of high-performance virtual machines difficult is not that they are large pieces of software, but that there are fundamental and complex interdependencies between their components. In order to work together correctly, the interpreter, just-in-time compiler, garbage collector and library must all conform to the same precise low-level protocols. In this dissertation I describe a method for constructing virtual machines for dy- namic languages, and explain how to design a virtual machine toolkit by building it around an abstract machine. The design and implementation of such a toolkit, the Glasgow Virtual Machine Toolkit, is described. The Glasgow Virtual Machine Toolkit automatically generates a just-in-time compiler, integrates precise garbage collection into the virtual machine, and automatically manages the complex inter- dependencies between all the virtual machine components. Two different virtual machines have been constructed using the GVMT. One is a minimal implementation of Scheme; which was implemented in under three weeks to demonstrate that toolkits like the GVMT can enable the easy construc- tion of virtual machines. The second, the HotPy VM for Python, is a high- performance virtual machine; it demonstrates that a virtual machine built with a toolkit can be fast and that the use of a toolkit does not overly constrain the high-level design. Evaluation shows that HotPy outperforms the standard Python interpreter, CPython, by a large margin, and has performance on a par with PyPy, the fastest Python VM currently available. Contents 1 Introduction 11 1.1 Virtual Machines .......................... 12 1.2 Dynamic Languages ........................ 12 1.3 The Problem ............................. 13 1.4 Thesis ................................ 14 1.5 Contributions ............................ 14 1.6 Outline ............................... 15 2 Virtual Machines 17 2.1 A Little History ........................... 17 2.2 Interpreters ............................. 19 2.3 Garbage Collection ......................... 24 2.4 Optimisation for Dynamic Languages ............... 31 2.5 Python Virtual Machines ...................... 33 2.6 Other Interpreted Languages and their VMs ............ 36 2.7 Self-Interpreters ........................... 43 2.8 Multi-Threading and Dynamic Languages ............. 43 2.9 Conclusion ............................. 44 3 Abstract Machine Based Toolkits 45 3.1 Introduction ............................. 45 3.2 The Essential Features of a Virtual Machine ............ 47 3.3 An Abstract Machine for Virtual Machines ............. 48 3.4 Optimisation in VMs for Dynamic Languages ........... 52 3.5 When to use the abstract machine approach? ............ 54 3.6 Alternative Approaches to Building VMs ............. 55 3.7 Related Work ............................ 56 3.8 Conclusions ............................. 57 4 The Glasgow Virtual Machine Toolkit 59 4.1 Overview .............................. 59 4.2 The Abstract Machine ........................ 61 4.3 Front-End Tools ........................... 66 4.4 Back-End Tools ........................... 71 4.5 Translating GVMT Abstract Machine Code to Real Machine Code 73 2 4.6 Memory Management in the GVMT ................ 78 4.7 Locks ................................ 86 4.8 Concurrency and Garbage Collection ................ 88 4.9 Comparison of PyPy and GVMT .................. 90 4.10 The GVMT Scheme Example Implementation ........... 91 4.11 Conclusions ............................. 92 5 HotPy, ANew VM for Python 95 5.1 Introduction ............................. 95 5.2 The HotPy VMModel ....................... 96 5.3 Design of the HotPy VM ...................... 99 5.4 Tracing and Traces ......................... 104 5.5 Optimisation of Traces ....................... 108 5.6 Specialisation ............................ 111 5.7 Deferred Object Creation ...................... 113 5.8 Further Optimisations ........................ 119 5.9 De-Optimisation .......................... 120 5.10 An Example ............................. 120 5.11 Deviations from the Design of CPython .............. 125 5.12 Dictionaries ............................. 126 5.13 Related Work ............................ 130 5.14 Conclusion ............................. 131 6 Results and Evaluation 133 6.1 Introduction ............................. 133 6.2 Utility of the GVMT and Toolkits in General ........... 133 6.3 Performance of the GVMT Scheme VM .............. 134 6.4 Comparison of Unladen Swallow, the PyPy VM, and HotPy ... 135 6.5 Aspects of Virtual Machine Performance .............. 143 6.6 Memory Usage ........................... 146 6.7 Effect of Garbage Collection .................... 149 6.8 Potential for Further Optimisation ................. 150 6.9 Conclusions ............................. 152 7 Conclusions 155 7.1 Review of the Thesis ........................ 155 7.2 Significant Results ......................... 156 7.3 Dissertation Summary ....................... 156 7.4 Future Work ............................. 158 7.5 In Closing .............................. 160 A The GVMT Abstract Machine Instruction Set 162 BThe GVMT Abstract Machine Language Grammar 197 C Python Attribute Lookup Semantics 200 3 C.1 Definitions .............................. 200 C.2 Lookup Algorithm ......................... 201 D Surrogate Functions 203 D.1 The __new__ method for tuple ................... 203 D.2 The __call__ method for type .................... 204 D.3 The Binary Operator ........................ 204 EThe HotPy Virtual Machine Bytecodes 205 E.1 Base Instructions .......................... 205 E.2 Instructions Required for Tracing .................. 213 E.3 Specialised Instructions ....................... 216 E.4 D.O.C. Instructions ......................... 221 E.5 Super Instructions .......................... 222 F Results 224 Bibliography 228 4 List of Tables 2.1 Main Python Implementations ................... 34 2.2 Main Ruby Implementations .................... 39 4.1 GVMTTypes ............................ 64 6.1 Unoptimised Interpreters. Short Benchmarks ........... 139 6.2 Unoptimised Interpreters. Medium Benchmarks .......... 139 6.3 Full VM. Short Benchmarks .................... 140 6.4 Full VM. Medium Benchmarks ................... 140 6.5 Full VM. Long Benchmarks .................... 141 6.6 Optimised Interpreters. Short Benchmarks ............. 141 6.7 Optimised Interpreters. Medium Benchmarks ........... 141 6.8 Interpreter vs. Compiler. Short Benchmarks ............ 142 6.9 Interpreter vs. Compiler. Medium Benchmarks .......... 142 6.10 Interpreter vs. Compiler. Long Benchmarks ............ 142 6.11 HotPy(C) Performance Permutations. Speeds Relative to CPython 144 6.12 HotPy(Py) Performance Permutations. Speeds Relative to CPython 144 6.13 Speed Up Due to Adding Specialiser; HotPy(C)........... 144 6.14 Speed Up Due to Adding D.O.C.; HotPy(C)............. 145 6.15 Speed Up Due to Adding Compiler; HotPy(C)........... 145 6.16 Speed Up Due to Adding Specialiser; HotPy(Py).......... 145 6.17 Speed Up Due to Adding D.O.C.; HotPy(Py)............ 145 6.18 Speed Up Due to Adding Compiler; HotPy(Py)........... 146 6.19 Memory Usage ........................... 147 6.20 CPython GC percentages ...................... 149 6.21 Theoretical CPython Speedups ................... 149 F.1 Timings (in seconds); short benchmarks .............. 225 F.2 Timings (in seconds); medium benchmarks ............ 226 F.3 Timings (in seconds); long benchmarks .............. 227 5 List of Figures 2.1 Token Threaded Code – Stack-based program for A+B ...... 20 2.2 Switch Threaded Code – Stack-based program for A+B ...... 20 2.3 Switch Threaded Implementation in C ............... 21 2.4 Direct Threaded Code – Stack-based program for A+B ...... 21 2.5 Indirect Threaded Code – Stack-based program for A+B ..... 21 2.6 Subroutine Threaded Code – Stack-based program for A+B .... 22 2.7 Call Threaded Code – Stack-based program for A+B ....... 22 2.8 Memory Cycle ........................... 24 2.9 Uncollectable cycle for Reference Counting ............ 27 2.10 Self VM Tag Formats ........................ 29 3.1 Generalised Bytecode Optimiser
Load more

Recommended publications
  • Learning to Program in Perl
    Learning to Program in Perl by Graham J Ellis Languages of the Web Learning to Program in Perl version 1.7 Written by Graham Ellis [email protected] Design by Lisa Ellis Well House Consultants, Ltd. 404, The Spa, Melksham, Wiltshire SN12 6QL England +44 (0) 1225 708 225 (phone) +44 (0) 1225 707 126 (fax) Find us on the World Wide Web at: http://www.wellho.net Or contact us at: [email protected] Copyright © 2003 by Well House Consultants, Ltd. Printed in Great Britain. Printing History May 1999 1.0 First Edition February 2000 1.1 Minor additions June 2000 1.2 Compliation of modules October 2000 1.3 Name change, revisions April 2002 1.4 Added modules September 2002 1.5 Added modules January 2003 1.6 Updated modules February 2003 1.7 Updated modules This manual was printed on 21 May 2003. Notice of Rights All rights reserved. No part of this manual, including interior design, may be reproduced or translated into any language in any form, or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise, without prior written permission of Well House Consultants except in the case of brief quotations embodied in critical articles and reviews. For more information on getting permission for reprints and excerpts, contact Graham Ellis at Well House Consultants. This manual is subject to the condition that it shall not, by way of trade or otherwise, be lent, sold, hired out or otherwise circulated without the publisher's prior consent, incomplete nor in any form of binding or cover other than in which it is published and without a similar condition including this condition being imposed on the subsequent receiver.
    [Show full text]
  • Hacker Public Radio
    hpr0001 :: Introduction to HPR hpr0002 :: Customization the Lost Reason hpr0003 :: Lost Haycon Audio Aired on 2007-12-31 and hosted by StankDawg Aired on 2008-01-01 and hosted by deepgeek Aired on 2008-01-02 and hosted by Morgellon StankDawg and Enigma talk about what HPR is and how someone can contribute deepgeek talks about Customization being the lost reason in switching from Morgellon and others traipse around in the woods geocaching at midnight windows to linux Customization docdroppers article hpr0004 :: Firefox Profiles hpr0005 :: Database 101 Part 1 hpr0006 :: Part 15 Broadcasting Aired on 2008-01-03 and hosted by Peter Aired on 2008-01-06 and hosted by StankDawg as part of the Database 101 series. Aired on 2008-01-08 and hosted by dosman Peter explains how to move firefox profiles from machine to machine 1st part of the Database 101 series with Stankdawg dosman and zach from the packetsniffers talk about Part 15 Broadcasting Part 15 broadcasting resources SSTRAN AMT3000 part 15 transmitter hpr0007 :: Orwell Rolled over in his grave hpr0009 :: This old Hack 4 hpr0008 :: Asus EePC Aired on 2008-01-09 and hosted by deepgeek Aired on 2008-01-10 and hosted by fawkesfyre as part of the This Old Hack series. Aired on 2008-01-10 and hosted by Mubix deepgeek reviews a film Part 4 of the series this old hack Mubix and Redanthrax discuss the EEpc hpr0010 :: The Linux Boot Process Part 1 hpr0011 :: dd_rhelp hpr0012 :: Xen Aired on 2008-01-13 and hosted by Dann as part of the The Linux Boot Process series.
    [Show full text]
  • Ironpython in Action
    IronPytho IN ACTION Michael J. Foord Christian Muirhead FOREWORD BY JIM HUGUNIN MANNING IronPython in Action Download at Boykma.Com Licensed to Deborah Christiansen <[email protected]> Download at Boykma.Com Licensed to Deborah Christiansen <[email protected]> IronPython in Action MICHAEL J. FOORD CHRISTIAN MUIRHEAD MANNING Greenwich (74° w. long.) Download at Boykma.Com Licensed to Deborah Christiansen <[email protected]> For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. Sound View Court 3B fax: (609) 877-8256 Greenwich, CT 06830 email: [email protected] ©2009 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15% recycled and processed without the use of elemental chlorine.
    [Show full text]
  • A Practical Solution for Scripting Language Compilers
    A Practical Solution for Scripting Language Compilers Paul Biggar, Edsko de Vries, David Gregg Department of Computer Science, Trinity College Dublin, Dublin 2, Ireland Abstract Although scripting languages are becoming increasingly popular, even mature script- ing language implementations remain interpreted. Several compilers and reimplemen- tations have been attempted, generally focusing on performance. Based on our survey of these reimplementations, we determine that there are three important features of scripting languages that are difficult to compile or reimplement. Since scripting languages are defined primarily through the semantics of their original implementations, they often change semantics between releases. They provide C APIs, used both for foreign-function interfaces and to write third-party extensions. These APIs typically have tight integration with the original implementation, and are used to providelarge standard libraries, which are difficult to re-use, and costly to reimplement. Finally, they support run-time code generation. These features make the important goal of correctness difficult to achieve for compilers and reimplementations. We present a technique to support these features in an ahead-of-time compiler for PHP. Our technique uses the original PHP implementation through the provided C API, both in our compiler, and in our generated code. We support all of these impor- tant scripting language features, particularly focusing on the correctness of compiled programs. Additionally, our approach allows us to automatically support limited fu- ture language changes. We present a discussion and performance evaluation of this technique. Key words: Compiler, Scripting Language 1. Motivation Although scripting languages1 are becoming increasingly popular, most scripting language implementations remain interpreted. Typically, these implementations are slow, between one and two orders of magnitude slower than C.
    [Show full text]
  • 2016 8Th International Conference on Cyber Conflict: Cyber Power
    2016 8th International Conference on Cyber Conflict: Cyber Power N.Pissanidis, H.Rõigas, M.Veenendaal (Eds.) 31 MAY - 03 JUNE 2016, TALLINN, ESTONIA 2016 8TH International ConFerence on CYBER ConFlict: CYBER POWER Copyright © 2016 by NATO CCD COE Publications. All rights reserved. IEEE Catalog Number: CFP1626N-PRT ISBN (print): 978-9949-9544-8-3 ISBN (pdf): 978-9949-9544-9-0 CopyriGHT AND Reprint Permissions No part of this publication may be reprinted, reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of the NATO Cooperative Cyber Defence Centre of Excellence ([email protected]). This restriction does not apply to making digital or hard copies of this publication for internal use within NATO, and for personal or educational use when for non-profit or non-commercial purposes, providing that copies bear this notice and a full citation on the first page as follows: [Article author(s)], [full article title] 2016 8th International Conference on Cyber Conflict: Cyber Power N.Pissanidis, H.Rõigas, M.Veenendaal (Eds.) 2016 © NATO CCD COE Publications PrinteD copies OF THIS PUBlication are availaBLE From: NATO CCD COE Publications Filtri tee 12, 10132 Tallinn, Estonia Phone: +372 717 6800 Fax: +372 717 6308 E-mail: [email protected] Web: www.ccdcoe.org Head of publishing: Jaanika Rannu Layout: Jaakko Matsalu LEGAL NOTICE: This publication contains opinions of the respective authors only. They do not necessarily reflect the policy or the opinion of NATO CCD COE, NATO, or any agency or any government.
    [Show full text]
  • A Java Virtual Machine Extended to Run Parrot Bytecode
    A JAVA VIRTUAL MACHINE EXTENDED TO RUN PARROT BYTECODE A thesis submitted to the University of Manchester for the degree of Master of Science in the Faculty of Engineering and Physical Sciences 2006 By Martin Dahl School of Computer Science Contents Abstract 8 Declaration 9 Copyright 10 Acknowledgements 11 1 Introduction 12 1.1 Objectives and Motivation . 13 1.2 Related Work . 13 1.2.1 Parrot . 14 1.2.2 Pugs . 14 1.2.3 PearColator . 14 1.3 Organisation of the Thesis . 15 2 The Parrot Project 16 2.1 Perl 6 . 17 2.1.1 Design . 17 2.1.2 Perl 6 Internals . 18 2.1.3 Pugs . 19 2.1.4 Parrot Perl 6 Compiler . 19 2.2 Virtual Machine . 20 2.2.1 Design . 20 2.2.2 Architecture . 21 2.2.3 Registers . 22 2.2.4 Instruction Set . 23 2.2.5 Parrot Assembly Language and Intermediate Representation 24 2 2.3 Parrot Magic Cookies . 24 2.3.1 Core PMCs . 24 2.3.2 Other PMCs . 27 2.4 Compiler suite . 27 2.4.1 Perl 6 . 27 2.4.2 Other languages . 27 2.5 Summary . 28 3 Parrot Bytecode Format 30 3.1 Header . 31 3.2 Bytecode Format 1 . 33 3.2.1 Directory Segment . 34 3.2.2 Constant Table Segment . 35 3.2.3 Bytecode Segment . 38 3.2.4 Debug Segment . 38 3.2.5 Fixup Segment . 39 3.3 Summary . 40 4 Parakeet 41 4.1 Overview . 42 4.2 Jikes RVM . 43 4.3 PearColator .
    [Show full text]
  • Threat Modeling and Circumvention of Internet Censorship by David Fifield
    Threat modeling and circumvention of Internet censorship By David Fifield A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor J.D. Tygar, Chair Professor Deirdre Mulligan Professor Vern Paxson Fall 2017 1 Abstract Threat modeling and circumvention of Internet censorship by David Fifield Doctor of Philosophy in Computer Science University of California, Berkeley Professor J.D. Tygar, Chair Research on Internet censorship is hampered by poor models of censor behavior. Censor models guide the development of circumvention systems, so it is important to get them right. A censor model should be understood not just as a set of capabilities|such as the ability to monitor network traffic—but as a set of priorities constrained by resource limitations. My research addresses the twin themes of modeling and circumvention. With a grounding in empirical research, I build up an abstract model of the circumvention problem and examine how to adapt it to concrete censorship challenges. I describe the results of experiments on censors that probe their strengths and weaknesses; specifically, on the subject of active probing to discover proxy servers, and on delays in their reaction to changes in circumvention. I present two circumvention designs: domain fronting, which derives its resistance to blocking from the censor's reluctance to block other useful services; and Snowflake, based on quickly changing peer-to-peer proxy servers. I hope to change the perception that the circumvention problem is a cat-and-mouse game that affords only incremental and temporary advancements.
    [Show full text]
  • LAMP and the REST Architecture Step by Step Analysis of Best Practice
    LAMP and the REST Architecture Step by step analysis of best practice Santiago Gala High Sierra Technology S.L.U. Minimalistic design using a Resource Oriented Architecture What is a Software Platform (Ray Ozzie ) ...a relevant and ubiquitous common service abstraction Creates value by leveraging participants (e- cosystem) Hardware developers (for OS level platforms) Software developers Content developers Purchasers Administrators Users Platform Evolution Early stage: not “good enough” solution differentiation, innovation, value flows Later: modular architecture, commoditiza- tion, cloning no premium, just speed to market and cost driven The platform effect - ossification, followed by cloning - is how Chris- tensen-style modularity comes to exist in the software industry. What begins as a value-laden proprietary platform becomes a replaceable component over time, and the most successful of these components finally define the units of exchange that power commodity networks. ( David Stutz ) Platform Evolution (II) Example: PostScript Adobe Apple LaserWriter Aldus Pagemaker Desktop Publishing Linotype imagesetters NeWS (Display PostScript) OS X standards (XSL-FO -> PDF, Scribus, OOo) Software Architecture ...an abstraction of the runtime elements of a software system during some phase of its oper- ation. A system may be composed of many lev- els of abstraction and many phases of opera- tion, each with its own software architecture. Roy Fielding (REST) What is Architecture? Way to partition a system in components
    [Show full text]
  • Graceful Language Extensions and Interfaces
    Graceful Language Extensions and Interfaces by Michael Homer A thesis submitted to the Victoria University of Wellington in fulfilment of the requirements for the degree of Doctor of Philosophy Victoria University of Wellington 2014 Abstract Grace is a programming language under development aimed at ed- ucation. Grace is object-oriented, imperative, and block-structured, and intended for use in first- and second-year object-oriented programming courses. We present a number of language features we have designed for Grace and implemented in our self-hosted compiler. We describe the design of a pattern-matching system with object-oriented structure and minimal extension to the language. We give a design for an object-based module system, which we use to build dialects, a means of extending and restricting the language available to the programmer, and of implementing domain-specific languages. We show a visual programming interface that melds visual editing (à la Scratch) with textual editing, and that uses our dialect system, and we give the results of a user experiment we performed to evaluate the usability of our interface. ii ii Acknowledgments The author wishes to acknowledge: • James Noble and David Pearce, his supervisors; • Andrew P. Black and Kim B. Bruce, the other designers of Grace; • Timothy Jones, a coauthor on a paper forming part of this thesis and contributor to Minigrace; • Amy Ruskin, Richard Yannow, and Jameson McCowan, coauthors on other papers; • Daniel Gibbs, Jan Larres, Scott Weston, Bart Jacobs, Charlie Paucard, and Alex Sandilands, other contributors to Minigrace; • Gilad Bracha, Matthias Felleisen, and the other (anonymous) review- ers of papers forming part of this thesis; • the participants in his user study; • David Streader, John Grundy, and Laurence Tratt, examiners of the thesis; • and Alexandra Donnison, Amy Chard, Juanri Barnard, Roma Kla- paukh, and Timothy Jones, for providing feedback on drafts of this thesis.
    [Show full text]
  • Proseminar Python - Python Bindings
    Proseminar Python - Python Bindings Sven Fischer Student an der TU-Dresden [email protected] Abstract Diese Arbeit beschaftigt¨ sich damit, einen Einblick in die Benut- zung von externen Bibliotheken in Python zu geben und fuhrt¨ den Leser in das Schreiben von eigenen Erweiterungen ein. Daruber¨ hinaus wird darauf eingegangen, wie Projekte in anderen Program- miersprachen Python benutzen konnen.¨ Weiterhin wird kurz auf verschiedene andere Moglichkeiten¨ eingegangen, Python oder Py- thons Benutzung zu erweitern und zu verandern:¨ durch Kompilati- on, Mischen“ mit oder Ubersetzen¨ in anderen Sprachen, oder spe- ” zielle Interpreter. Abbildung 1. Vergleich der Moglichkeiten¨ Python zu erwei- tern (links) und Python einzubetten (rechts). Categories and Subject Descriptors D.3.3 [Programming Lan- guages]: Language Constructs and Features—Modules, packages 1. Daten von C nach Python konvertieren General Terms Languages, Documentation 2. Python Funktion mit konvertierten Werten aufrufen Keywords Python, Extension, Embedding, Library 3. Ergebnis von Python zuruck¨ nach C konvertieren 1. Einfuhrung¨ Diese Konvertierung ist auch das großte¨ Hindernis beim Ver- 1 Python wird mit einer umfangreichen Standartbibliothek ausgelie- knupfen¨ von Python und C , zumindest ist es mit einigem Aufwand fert. Trotzdem gibt es Grunde,¨ warum man Python um verschiedene verbunden. externe Bibliotheken erweitern mochte.¨ Ein Beispiel dafur¨ ware¨ die Es gibt einige Projekte, die sich mit dem Verandern¨ und Erwei- ¨ Geschwindigkeit, die bei Python als interpretierter Sprache nicht tern der Sprache Python beschaftigen.¨ Einen kurzen Uberblick uber¨ immer den Anforderungen entspricht. Weiterhin gibt es genugend¨ einige Moglichkeiten¨ gebe ich in Abschnitt4. Dort gehe ich auf Software von Drittanbietern, welche man in Python nutzbar ma- zwei Python-Interpreter neben dem in der Standard-Distribution chen will - ohne sie in Python zu ubersetzen.¨ Darauf gehe ich in enthaltenen ein und zeige Moglichkeiten¨ auf, Python in andere Abschnitt2 ein.
    [Show full text]
  • Efficient Use of Python on the Clusters
    Efficient use of Python on the clusters Ariel Lozano CÉCI training November 21, 2018 Outline I Analyze our code with profiling tools: I cpu: cProfile, line_profiler, kernprof I memory: memory_profiler, mprof I Being a highly abstract dynamically typed language, how to make a more efficient use of hardware internals? I Numpy and Scipy ecosystem (mainly wrappers to C/Fortran compiled code) I binding to compiled code: interfaces between python and compiled modules I compiling: tools to compile python code I parallelism: modules to exploit multicores Sieve of eratostenes Algorithm to find all prime numbers up to any given limit. Ex: Find all the prime numbers less than or equal to 25: I 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Cross out every number displaced by 2 after 2 up to the limit: I 23 45 67 89 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Move to next n non crossed, cross out each non crossed number displaced by n: I 23 45 67 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 I 2 3 45 67 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 The remaining numbers non crossed in the list are all the primes below limit. 2 Trivial optimization: jump directlyp to n to start crossing out. Then, n must loop only up to limit.
    [Show full text]
  • CIS 192: Lecture 12 Deploying Apps and Concurrency
    CIS 192: Lecture 12 Deploying Apps and Concurrency Lili Dworkin University of Pennsylvania Good Question from Way Back I All HTTP requests have 1) URL, 2) headers, 3) body I GET requests: parameters sent in URL I POST requests: parameters sent in body Can GET requests have a body? StackOverflow's response: \Yes, you can send a request body with GET but it should not have any meaning. If you give it meaning by parsing it on the server and changing your response based on its contents you're violating the HTTP/1.1 spec." Good Question from Last Week What is the difference between jsonify and json.dumps? def jsonify(*args, **kwargs): if __debug__: _assert_have_json() return current_app.response_class(json.dumps(dict(* args, **kwargs), indent=None if request.is_xhr else 2), mimetype='application/json') I jsonify returns a Response object I jsonify automatically sets content-type header I jsonify also sets the indentation Review Find a partner! Deploying Apps I We've been running Flask apps locally on a builtin development server I When you're ready to go public, you need to deploy to a production server I Easiest option: use one hosted by someone else! I We'll use Heroku, a platform as a service (PaaS) that makes it easy to deploy apps in a variety of languages Heroku Prerequisites: I Virtualenv (creates standalone Python environments) I Heroku toolbox I Heroku command-line client I Git (for version control and pushing to Heroku) Virtualenv I Allows us to create a virtual Python environment I Unique, isolated environment for each project I Use case: different versions of packages for different projects Virtualenv How to use it? prompt$ pip install virtualenv Now navigate to your project directory: prompt$ virtualenv --no-site-packages venv prompt$ source venv/bin/activate (<name>)prompt$ pip install Flask gunicorn (<name>)prompt$ deactivate prompt% Heroku Toolbox Once you make a Heroku account, install the Heroku toolbox.
    [Show full text]