Automated Program Transformation for Improving Software Quality

Total Page:16

File Type:pdf, Size:1020Kb

Automated Program Transformation for Improving Software Quality Automated Program Transformation for Improving Software Quality Rijnard van Tonder CMU-ISR-19-101 October 2019 Institute for Software Research School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Claire Le Goues, Chair Christian Kästner Jan Hoffmann Manuel Fähndrich, Facebook, Inc. Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Software Engineering. Copyright 2019 Rijnard van Tonder This work is partially supported under National Science Foundation grant numbers CCF-1750116 and CCF-1563797, and a Facebook Testing and Verification research award. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring corporation, institution, the U.S. government, or any other entity. Keywords: syntax, transformation, parsers, rewriting, crash bucketing, fuzzing, bug triage, program transformation, automated bug fixing, automated program repair, separation logic, static analysis, program analysis Abstract Software bugs are not going away. Millions of dollars and thousands of developer-hours are spent finding bugs, debugging the root cause, writing a patch, and reviewing fixes. Automated techniques like static analysis and dynamic fuzz testing have a proven track record for cutting costs and improving software quality. More recently, advances in automated program repair have matured and see nascent adoption in industry. Despite the value of these approaches, automated techniques do not come for free: they must approximate, both theoretically and in the interest of practicality. For example, static analyzers suffer false positives, and automatically produced patches may be insufficiently precise to fix a bug. Such limitations continue to impose substantial human effort amid the benefits of automation. Software development activities revolve around changing code. Thus, per- forming and reasoning about program change has extensive bearing on the effectiveness of automated techniques. From this perspective, we develop new automated techniques for changing programs to improve analysis behavior, and, correspondingly, use automated reasoning and analysis to specialize program changes for automated program repair. We present the first evidence that automated program transformation, program analysis, and program repair are interrelated and cooperative. We first show that automated program transfor- mation leads to higher quality static analysis (by reducing false positives) and dynamic fuzz testing (by reducing duplicate bug reports). We then show how high-quality static analyses can feed into and enable automated program repair, and how automated repair can circle back to further improve static analysis (e.g., by revealing more true positive bugs). The thesis is that automated syn- tactic and semantic search and application of program transformations enables efficient, scalable, and unassisted techniques for improving the effectiveness of existing program analyses and end-to-end repair of real-world programs. We show that these techniques are effective compared to current approaches in the respective domains of static analysis, dynamic fuzz testing, and program repair. We demonstrate relevance and real-world applicability by evaluating on large, popular, and active projects across multiple languages. Our vision for this work is that new capabilities and techniques for automated program transformation foster effective ways to automate burdensome human effort and reasoning incurred by limitations in program analysis and repair. iv Acknowledgments I’m incredibly grateful to my advisor, Claire, for taking me as her student. My first impression of Claire was that she would make a good advisor. Over time I realized that she’s exceptional. I admire the focus and tact with which she’s helped steer my work; always through keen questions and candid feedback. She’s been my advocate and collaborator, and a valuable source of optimism. Our interactions had a special sense of levity that I’ll always remember. I thank my committee members, Christian, Jan, and Manuel. Their engage- ment was excellent and I greatly value their time and feedback. Their input has raised the quality of this dissertation. I want to acknowledge the friendship and support of the following people during my studies: Arné, Nelius, Robert, Gerdus, Dominic, Nam, and Ivan. Then also those in the research group: Chris, Zack, Deby, Mau, Afsoon, Cody, and Jeremy. I thank my family for their unceasing encouragement, prayers, and love: my father, Gerhard, my mother, Riana, and my two sisters, Monique and Karli. I’m especially grateful for the decisions and sacrifices that my parents made years and years ago—I realize now that those choices made it possible for me to accomplish what I have today. None of what I accomplish matters without acknowledging Jesus Christ, who grants me all that I have. “Now, for you: So what? The question I would leave you with is: Have you ever, as an adult, faced up to the question of whether the claims of Jesus Christ are true? And that’s the most important question I can ask you.” — Fred Brooks, “Last Blast”, 2015. vi Contents List of Terms xv Preface 1 1 Introduction3 1.1 Scope and Success Criteria...........................7 1.1.1 Lightweight Declarative Syntax Transformation...........7 1.1.2 Tailoring Programs for Static Analysis................8 1.1.3 Tailoring Programs under Dynamic Fuzz Testing...........9 1.1.4 Augmenting Static Analysis for Automated Program Repair.... 10 1.2 Contributions.................................. 11 1.3 Outline...................................... 13 2 Lightweight Declarative Syntax Transformation 15 2.1 Introduction................................... 16 2.2 Motivating Example.............................. 19 2.3 Dyck-Extended Languages........................... 23 2.4 The Rewrite Process.............................. 26 2.4.1 Declarative Matching: The Intuition................. 26 2.4.2 Parser Parser Combinators for Matching............... 29 2.4.3 Match Rules............................... 33 vii 2.4.4 The Rewrite Phase........................... 34 2.5 Evaluation.................................... 35 2.5.1 Real, Large Scale Multi-Language Rewriting............. 37 2.5.2 Comparison with Existing Tools.................... 46 2.5.3 Discussion................................ 49 2.6 Related Work.................................. 51 2.7 Summary.................................... 53 3 Tailoring Programs for Static Analysis 55 3.1 Introduction................................... 55 3.2 Motivation.................................... 59 3.3 Approach.................................... 62 3.3.1 Match Template Behavior....................... 63 3.3.2 Tailoring Programs for Analysis.................... 63 3.4 Evaluation.................................... 67 3.4.1 Experimental Setup........................... 69 3.4.2 Experimental Results.......................... 71 3.4.3 Discussion................................ 82 3.5 Related Work.................................. 84 3.6 Summary.................................... 86 4 Semantic Crash Bucketing 87 4.1 Introduction................................... 87 4.2 Motivating Example.............................. 91 4.3 Semantic Crash Bucketing........................... 94 4.3.1 Problem Formulation.......................... 95 4.3.2 Detecting Duplicates.......................... 97 4.3.3 Semantic Crash Bucketing Procedure................. 98 viii 4.4 Generating Approximate Fixes......................... 100 4.4.1 Tb Production............................... 100 4.4.2 Null Dereferences............................ 101 4.4.3 Buffer Overflows............................ 103 4.5 Experimental Design.............................. 105 4.5.1 Bugs with Ground Truth........................ 106 4.5.2 Crash Corpus Generation....................... 108 4.5.3 Evaluating Fuzzers........................... 108 4.6 Experimental Results.............................. 111 4.6.1 Overall Results............................. 111 4.6.2 Project-specific Results......................... 113 4.7 Discussion.................................... 114 4.8 Related Work.................................. 116 4.9 Summary.................................... 118 5 End-to-End Static Automated Program Repair 119 5.1 Introduction................................... 120 5.2 Preliminaries.................................. 125 5.2.1 Program Model and Assertion Language............... 126 5.2.2 Frame Inference............................. 128 5.2.3 Finding Bugs Using Separation Logic................. 129 5.3 Repair with Separation Logic......................... 130 5.3.1 Formulating Repair........................... 131 5.3.2 Searching for Repairs.......................... 133 5.3.3 Applying Repairs: from Logic to Programs.............. 136 5.4 Evaluation.................................... 140 5.4.1 Setup.................................. 141 ix 5.4.2 Repair Results............................. 144 5.4.3 Patch Quality.............................. 145 5.4.4 Fixing by Semantic Effects....................... 147 5.5 Limitations and Discussion........................... 149 5.6 Related Work.................................. 150 5.7 Summary.................................... 152 6 Conclusion and Future Work 153 6.1 Summary.................................... 153 6.2 Open Questions and Research Pursuits.................... 155 6.2.1 Extending Automated
Recommended publications
  • 16 Inspiring Women Engineers to Watch
    Hackbright Academy Hackbright Academy is the leading software engineering school for women founded in San Francisco in 2012. The academy graduates more female engineers than UC Berkeley and Stanford each year. https://hackbrightacademy.com 16 Inspiring Women Engineers To Watch Women's engineering school Hackbright Academy is excited to share some updates from graduates of the software engineering fellowship. Check out what these 16 women are doing now at their companies - and what languages, frameworks, databases and other technologies these engineers use on the job! Software Engineer, Aclima Tiffany Williams is a software engineer at Aclima, where she builds software tools to ingest, process and manage city-scale environmental data sets enabled by Aclima’s sensor networks. Follow her on Twitter at @twilliamsphd. Technologies: Python, SQL, Cassandra, MariaDB, Docker, Kubernetes, Google Cloud Software Engineer, Eventbrite 1 / 16 Hackbright Academy Hackbright Academy is the leading software engineering school for women founded in San Francisco in 2012. The academy graduates more female engineers than UC Berkeley and Stanford each year. https://hackbrightacademy.com Maggie Shine works on backend and frontend application development to make buying a ticket on Eventbrite a great experience. In 2014, she helped build a WiFi-enabled basal body temperature fertility tracking device at a hardware hackathon. Follow her on Twitter at @magksh. Technologies: Python, Django, Celery, MySQL, Redis, Backbone, Marionette, React, Sass User Experience Engineer, GoDaddy 2 / 16 Hackbright Academy Hackbright Academy is the leading software engineering school for women founded in San Francisco in 2012. The academy graduates more female engineers than UC Berkeley and Stanford each year.
    [Show full text]
  • Magento on HHVM Speeding up Your Webshop with a Drop-In PHP Replacement
    Magento on HHVM Speeding up your webshop with a drop-in PHP replacement. Daniel Sloof [email protected] What is HHVM? ● HipHop Virtual Machine ● Created by engineers at Facebook ● Essentially a reimplementation of PHP ● Originally translated PHP to C++, now translates PHP to bytecode ● Just-in-time compiler, turning generated bytecode into machine code ● In some cases 5 to 10 times faster than regular PHP So what’s the problem? ● HHVM not entirely compatible with PHP ● Magento’s PHP triggering many of these incompatibilities ● Choosing between ○ Forking Magento to work around HHVM ○ Fixing issues within the extensive HHVM C++ codebase Resulted in... fixing HHVM ● Already over 100 commits fixing Magento related HHVM bugs; ○ SimpleXML (majority of bugfixes) ○ sessions ○ number_format ○ __get and __set ○ many more... ● Most of these fixes already merged back into the official (github) repository ● Community Edition running (relatively) stable! Benchmarks Before we go to the results... ● Magento 1.8 with sample data ● Standard Apache2 / php-fpm / MySQL stack (with APC opcode cache). ● Standard HHVM configuration (repo-authoritative mode disabled, JIT enabled) ● Repo-authoritative mode has potential to increase performance by a large margin ● Tool of choice: siege Benchmarks: Response time Average across 50 requests Benchmarks: Transaction rate While increasing siege concurrency until avg. response time ~2 seconds What about <insert caching mechanism here>? ● HHVM does not get in the way ● Dynamic content still needs to be generated ● Replaces PHP - not Varnish, Redis, FPC, Block Cache, etc. ● As long as you are burning CPU cycles (always), you will benefit from HHVM ● Think about speeding up indexing, order placement, routing, etc.
    [Show full text]
  • The Scrabble Player's Handbook Is Available for Free Download At
    The Scrabble Player's Handbook is available for free download at www.scrabbleplayershandbook.com 1 Contents Introduction 3 Meet The Team 5 What's Different About Competitive Scrabble? 10 How To Play Good Scrabble 11 The Words 14 What Is Scrabble? 16 Scoring Well 21 Understanding Rack Leaves 32 Word Learning 35 The First Move 46 Tile Tracking 50 Time Management 54 Exchanging 58 Phoneys 64 Set-Ups 65 Open and Closed Boards 68 The Endgame 75 Playing Style 85 How To Play Amazing Scrabble 94 The Luck Element 98 The Game Behind The Game 99 Starting Out in Competitive Play 101 Quackle 103 Zyzzyva 109 Internet Scrabble Club 115 Aerolith 117 Scrabble by Phone 119 Books 121 Scrabble Variants 123 Scrabble Around The World 125 Playing Equipment 127 Glossary 128 Appendix 133 Rules Governing Word Inclusion 133 Two-letter words 137 Three-letter words 140 SCRABBLE® is a registered trademark. All intellectual property rights in and to the game are owned in the U.S.A. by Hasbro Inc., in Canada by Hasbro Canada Inc. and throughout the rest of the world by J.W. Spear & Sons Ltd. of Maidenhead SL6 4UB, England, a subsidiary of Mattel Inc. Mattel and Spear are not affiliated with Hasbro or Hasbro Canada. The Scrabble Player's Handbook is available free of charge. There is no copyright on the contents and readers are encouraged to distribute the book in PDF or printed form to all who would benefit from it. Please respect our work by retaining the footer on every page and by refraining from reproducing any part of this book for financial gain.
    [Show full text]
  • Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK
    1 From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK Abstract—This paper1 describes some of the challenges and research questions that target the most productive intersection opportunities when deploying static and dynamic analysis at we have yet witnessed: that between exciting, intellectually scale, drawing on the authors’ experience with the Infer and challenging science, and real-world deployment impact. Sapienz Technologies at Facebook, each of which started life as a research-led start-up that was subsequently deployed at scale, Many industrialists have perhaps tended to regard it unlikely impacting billions of people worldwide. that much academic work will prove relevant to their most The paper identifies open problems that have yet to receive pressing industrial concerns. On the other hand, it is not significant attention from the scientific community, yet which uncommon for academic and scientific researchers to believe have potential for profound real world impact, formulating these that most of the problems faced by industrialists are either as research questions that, we believe, are ripe for exploration and that would make excellent topics for research projects. boring, tedious or scientifically uninteresting. This sociological phenomenon has led to a great deal of miscommunication between the academic and industrial sectors. I. INTRODUCTION We hope that we can make a small contribution by focusing on the intersection of challenging and interesting scientific How do we transition research on static and dynamic problems with pressing industrial deployment needs. Our aim analysis techniques from the testing and verification research is to move the debate beyond relatively unhelpful observations communities to industrial practice? Many have asked this we have typically encountered in, for example, conference question, and others related to it.
    [Show full text]
  • Nástroje Pro Sjednocení Datových Zdrojů Projektu Gloffer Tools for Unification of Data Sources Project Gloffer
    VŠB – Technická univerzita Ostrava Fakulta elektrotechniky a informatiky Katedra informatiky Nástroje pro sjednocení datových zdrojů projektu Gloffer Tools for unification of data sources project Gloffer 2018 Bc. Jakub Malchárek Rád bych poděkoval panu Ing. Radoslavu Fasugovi, Ph.D. za odbornou pomoc a konzultaci při zpracování této diplomové práce a cenné rady v průběhu implementace. Abstrakt V této diplomové práci se zabývám analýzou dostupných technologií pro implementaci webo- vého portálu Gloffer. Jsou zde popsány databáze (MySQL, Redis, MongoDB, Aerospike, Apache HBase, Apache Cassandra, Google Bigtable, Memcached), vyhledávače (Solr, Lucene, Elastic Search), webové servery (Apache HTTP server, Apache Tomcat), zprostředkovatelé zpráv (Rab- bit MQ), distribuované výpočetní technologie (Apache Hadoop) a vývojové technologie (PHP 7, Nette Framework, Java, Spring Framework). Cílem je nejen popis těchto technologií, ale také ná- vrh a implementace rozhraní pro sjednocení datových zdrojů projektu Gloffer v programovacím jazyce Java s využitím Spring Frameworku. Výstupem práce je inteligentní nástroj zpřístupňující data z více datových zdrojů. Závěr práce obsahuje výkonové testování vyvinutého nástroje. Klíčová slova: Aerospike, Apache Cassandra, Apache Hadoop, Apache HBase, Apache HTTP server, Apache Tomcat, aplikační rozhraní, datové zdroje, Elastic Search, fulltext, Google Bi- gtable, index, Java, Lucene, Memcached, MongoDB, MySQL, Nette Framework, PHP, Rabbit MQ, Redis, REST, Solr, Spring Framework Abstract In this diploma thesis I deal with analysis of the available technologies for implementation of the Gloffer web portal. There are described databases (MySQL, Redis, MongoDB, Aerospike, Apache HBase, Apache Cassandra, Google Bigtable, Memcached), search engines (Solr, Lucene, Elastic Search), web servers (Apache HTTP server, Apache Tomcat), message brokers (Rabbit MQ), distributed computing technologies (Apache Hadoop) and develop technologies (PHP 7, Nette Framework, Java, Spring Framework).
    [Show full text]
  • Artificial Intelligence for Understanding Large and Complex
    Artificial Intelligence for Understanding Large and Complex Datacenters by Pengfei Zheng Department of Computer Science Duke University Date: Approved: Benjamin C. Lee, Advisor Bruce M. Maggs Jeffrey S. Chase Jun Yang Dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Computer Science in the Graduate School of Duke University 2020 Abstract Artificial Intelligence for Understanding Large and Complex Datacenters by Pengfei Zheng Department of Computer Science Duke University Date: Approved: Benjamin C. Lee, Advisor Bruce M. Maggs Jeffrey S. Chase Jun Yang An abstract of a dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Computer Science in the Graduate School of Duke University 2020 Copyright © 2020 by Pengfei Zheng All rights reserved except the rights granted by the Creative Commons Attribution-Noncommercial Licence Abstract As the democratization of global-scale web applications and cloud computing, under- standing the performance of a live production datacenter becomes a prerequisite for making strategic decisions related to datacenter design and optimization. Advances in monitoring, tracing, and profiling large, complex systems provide rich datasets and establish a rigorous foundation for performance understanding and reasoning. But the sheer volume and complexity of collected data challenges existing techniques, which rely heavily on human intervention, expert knowledge, and simple statistics. In this dissertation, we address this challenge using artificial intelligence and make the case for two important problems, datacenter performance diagnosis and datacenter workload characterization. The first thrust of this dissertation is the use of statistical causal inference and Bayesian probabilistic model for datacenter straggler diagnosis.
    [Show full text]
  • Facebook Messenger Engineering
    SED 1037 Transcript EPISODE 1037 [INTRODUCTION] [00:00:00] JM: Facebook Messenger is a chat application that millions of people use every day to talk to each other. Over time, Messenger has grown to include group chats, video chats, animations, facial filters, stories and many more features. Messenger is a tool for utility as well as for entertainment. Messengers used on both mobile and desktop, but the size of the mobile application is particularly important. There are many users who are on devices that do not have much storage space. As Messenger has accumulated features, the iOS codebase has grown larger and larger. Several generations of Facebook engineers have rotated through the company with responsibility of working on Facebook Messenger, and that has led to different ways of managing information within the same codebase. The iOS codebase had room for improvement and Project LightSpeed was a project within Facebook that had the goal of making Messenger on iOS much smaller. Mohsen Agsen and is an engineer with Facebook and he joins the show to talk about the process of rewriting the Messenger app. This is a great deep dive into how to rewrite a mission- critical iOS application, and this team became very large at a certain point within Facebook. It's a great story and I hope you enjoy it as well. [SPONSOR MESSAGE] [00:01:27] JM: When I’m building a new product, G2i is the company that I call on to help me find a developer who can build the first version of my product. G2i is a hiring platform run by engineers that matches you with React, React Native, GraphQL and mobile engineers who you can trust.
    [Show full text]
  • Unicorn: a System for Searching the Social Graph
    Unicorn: A System for Searching the Social Graph Michael Curtiss, Iain Becker, Tudor Bosman, Sergey Doroshenko, Lucian Grijincu, Tom Jackson, Sandhya Kunnatur, Soren Lassen, Philip Pronin, Sriram Sankar, Guanghao Shen, Gintaras Woss, Chao Yang, Ning Zhang Facebook, Inc. ABSTRACT rative of the evolution of Unicorn's architecture, as well as Unicorn is an online, in-memory social graph-aware index- documentation for the major features and components of ing system designed to search trillions of edges between tens the system. of billions of users and entities on thousands of commodity To the best of our knowledge, no other online graph re- servers. Unicorn is based on standard concepts in informa- trieval system has ever been built with the scale of Unicorn tion retrieval, but it includes features to promote results in terms of both data volume and query volume. The sys- with good social proximity. It also supports queries that re- tem serves tens of billions of nodes and trillions of edges quire multiple round-trips to leaves in order to retrieve ob- at scale while accounting for per-edge privacy, and it must jects that are more than one edge away from source nodes. also support realtime updates for all edges and nodes while Unicorn is designed to answer billions of queries per day at serving billions of daily queries at low latencies. latencies in the hundreds of milliseconds, and it serves as an This paper includes three main contributions: infrastructural building block for Facebook's Graph Search • We describe how we applied common information re- product. In this paper, we describe the data model and trieval architectural concepts to the domain of the so- query language supported by Unicorn.
    [Show full text]
  • Ting-Yuan Hsia (408) 707-2897 | [email protected]| HPs:// HPs:// Education Santa Clara Univeristy, Santa Clara, CA, USA Sep
    Ting-Yuan Hsia (408) 707-2897 | [email protected]| hps://www.linkedin.com/in/ly2314| hps://www.ly2314.cc Education Santa Clara Univeristy, Santa Clara, CA, USA Sep. 2017 - Jun. 2019 Master of Science in Computer Science and Engineering GPA: 3.63 / 4 • Related Courses: Algorithm, Operating Systems, Data Mining, Cryptology, Computer Networks, Distributed Systems National Taiwan University, Taipei, Taiwan Sep. 2014 - Jun. 2016 Master of Science, Department of Electrical Engineering • Master thesis: “Scheduling-Aware Data Prefetching Based on Spark Framework”. • Related Courses: Machine Learning, Artificial Intelligence, Fault Tolerant Computing, Network and Computer Security National Taiwan University, Taipei, Taiwan Sep. 2010 - Jun. 2014 Bachelor of Science in Engineering, Department of Electrical Engineering • Related Courses: Data Structure and Programming Experience Software Development Engineer Jul. 2019 - Present NetApp, Inc. Sunnyvale, CA, USA • Develop and maintain ONTAP data protection technology including SnapMirror, SnapDiff, Volume Move. Software Engineer Intern Jun. 2018 - Sep. 2018 Facebook, Inc. Menlo Park, CA, USA • Developed a cache service from scratch for an internal system and it reduced page loading time by 60%. • The cache service was implemented in C++ and contains a Thrift interface which can be queried from PHP, C++ and Python clients. • Modified both back-end service and frontend user interface for pagination capability. Software Engineer Jan. 2013 - Jul. 2016 Zuvio Inc. Taipei, Taiwan • Developed and maintained client-side products including PowerPoint Add-ins and desktop applications from scratch using C#, XAML, WPF and VSTO. Teaching Assistant Feb. 2016 - Jul. 2016 National Taiwan University, Department of Electrical Engineering Taipei, Taiwan • Graded and assisted students in EE 4052, Computer Programming.
    [Show full text]
  • Dmon: Efficient Detection and Correction of Data Locality
    DMon: Efficient Detection and Correction of Data Locality Problems Using Selective Profiling Tanvir Ahmed Khan and Ian Neal, University of Michigan; Gilles Pokam, Intel Corporation; Barzan Mozafari and Baris Kasikci, University of Michigan https://www.usenix.org/conference/osdi21/presentation/khan This paper is included in the Proceedings of the 15th USENIX Symposium on Operating Systems Design and Implementation. July 14–16, 2021 978-1-939133-22-9 Open access to the Proceedings of the 15th USENIX Symposium on Operating Systems Design and Implementation is sponsored by USENIX. DMon: Efficient Detection and Correction of Data Locality Problems Using Selective Profiling Tanvir Ahmed Khan Ian Neal Gilles Pokam Barzan Mozafari University of Michigan University of Michigan Intel Corporation University of Michigan Baris Kasikci University of Michigan Abstract cally at run time. In fact, as we (§6.2) and others [2,15,20,27] Poor data locality hurts an application’s performance. While demonstrate, compiler-based techniques can sometimes even compiler-based techniques have been proposed to improve hurt performance when the assumptions made by those heuris- data locality, they depend on heuristics, which can sometimes tics do not hold in practice. hurt performance. Therefore, developers typically find data To overcome the limitations of static optimizations, the locality issues via dynamic profiling and repair them manually. systems community has invested substantial effort in devel- Alas, existing profiling techniques incur high overhead when oping dynamic profiling tools [28,38, 57,97, 102]. Dynamic used to identify data locality problems and cannot be deployed profilers are capable of gathering detailed and more accurate in production, where programs may exhibit previously-unseen execution information, which a developer can use to identify performance problems.
    [Show full text]
  • Racerd: Compositional Static Race Detection
    RacerD: Compositional Static Race Detection SAM BLACKSHEAR, Facebook, USA NIKOS GOROGIANNIS, Facebook, UK and Middlesex University London, UK PETER W. O’HEARN, Facebook, UK and University College London, UK ILYA SERGEY∗, Yale-NUS College, Singapore and University College London, UK Automatic static detection of data races is one of the most basic problems in reasoning about concurrency. We present RacerD—a static program analysis for detecting data races in Java programs which is fast, can scale to large code, and has proven effective in an industrial software engineering scenario. To our knowledge, RacerD is the first inter-procedural, compositional data race detector which has been empirically shownto have non-trivial precision and impact. Due to its compositionality, it can analyze code changes quickly, and this allows it to perform continuous reasoning about a large, rapidly changing codebase as part of deployment within a continuous integration ecosystem. In contrast to previous static race detectors, its design favors reporting high-confidence bugs over ensuring their absence. RacerD has been in deployment for over a year at Facebook, where it has flagged over 2500 issues that have been fixed by developers before reaching production. It has been important in enabling the development of new code as well as fixing old code: it helped support the conversion of part of the main Facebook Android app from a single-threaded to a multi-threaded architecture. In this paper we describe RacerD’s design, implementation, deployment and impact. CCS Concepts: • Theory of computation → Program analysis; • Software and its engineering → Concurrent programming structures; Additional Key Words and Phrases: Concurrency, Static Analysis, Race Freedom ACM Reference Format: Sam Blackshear, Nikos Gorogiannis, Peter W.
    [Show full text]
  • PHP Optimization Using Hip Hop Virtual Machine Chaitali Tambe, Pramod Pawar, Dashrath Mane Vivekanand Education Society Institute of Technology, Chembur
    International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 4 Issue 6, June 2015 PHP Optimization Using Hip Hop Virtual Machine Chaitali Tambe, Pramod Pawar, Dashrath Mane Vivekanand education society institute of technology, Chembur Abstract— HipHop compiler is PHP based compiler created facebook.com. PHP applications that were developed by facebook in order to optimize the php code and save using HipHop terminology resulted in increased speed of resources on the facebook server. HPHPc translates the php application. Several drawbacks of standalone php code into executable c++ code. Later facebook realized that applications were eliminated by using HPHPc, one of HPHPc had some snags which included resource consumption and parallel maintenance of HPHP and which was performance. HPHPi. In order to overcome the drawbacks of HPHPc A. Limitation of HPHPc: HipHop virtual machine was developed in 2011 which is a process virtual machine based on just-in-time (JIT) HPHPc did not fully support the PHP language, compilation, serving as an execution engine for PHP and including the create_function() and eval() Hack programming languages. HHVM provides complete constructs. support for the entire PHP language. This paper shows how HHVM is superior to other compilers like PHP, PHP 7 It involved a specific time and resource consuming and how various framework uses HHVM to optimize deployment process that required a bigger than php. In addition the lock-down approach shows the 1 GB binary to be compiled and distributed to behavior of each framework with hhvm and show how many servers in short orders. performance optimization can be obtained for heavy In addition, maintaining HPHPc and HPHPi in websites.
    [Show full text]