DOCSLIB.ORG

Code Coverage at Google

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Code Coverage at Google Code Coverage at Google Marko Ivanković Goran Petrović René Just Gordon Fraser [email protected] [email protected] [email protected] [email protected] Google Switzerland GmbH Google Switzerland GmbH University of Washington University of Passau Zurich, Switzerland Zurich, Switzerland Seattle, WA, USA Passau, Germany ABSTRACT For example, Piwowarski et al. [25] mention that IBM performed Code coverage is a measure of the degree to which a test suite code coverage measurements in the late 1960s, and Elmendorf [13] exercises a software system. Although coverage is well established provided a reasonably robust strategy for using branch coverage in in software engineering research, deployment in industry is often testing operating systems as early as 1969. inhibited by the perceived usefulness and the computational costs While code coverage is commonly used in software engineering of analyzing coverage at scale. At Google, coverage information is research, its effectiveness at improving testing in practice is still computed for one billion lines of code daily, for seven programming a matter of debate [14, 18] and its adoption in industry is not uni- languages. A key aspect of making coverage information actionable versal. A large survey at IBM on their use of code coverage [25] is to apply it at the level of changesets and code review. revealed that once code coverage was adopted, it led to an increase This paper describes Google’s code coverage infrastructure and in test suite quality; however, ease of use and scalability are primary how the computed code coverage information is visualized and factors in whether code coverage is adopted in the first place. Exist- used. It also describes the challenges and solutions for adopting ing industry reports on code coverage are typically based on code code coverage at scale. To study how code coverage is adopted and bases of manageable size with one or two programming languages. perceived by developers, this paper analyzes adoption rates, error However, even at that scale, these reports raise concerns about the rates, and average code coverage ratios over a five-year period, cost-effectiveness of code coverage [20]. In particular, they raise and it reports on 512 responses, received from surveying 3000 concerns that both the machine resources required to compute cov- developers. Finally, this paper provides concrete suggestions for erage as well as the developer time required to process the results how to implement and use code coverage in an industrial setting. are too costly compared to the increase in test suite quality. Addressing and overcoming these concerns, Google has spent CCS CONCEPTS more than a decade refining its coverage infrastructure and imple- menting and validating the academic approaches. Google’s infras- • Software and its engineering → Software configuration man- tructure supports seven programming languages and scales to a agement and version control systems; Software testing and codebase of one billion lines of code that receives tens of thousands debugging; Empirical software validation; Collaboration in soft- commits per day. The paper details Google’s code coverage infras- ware development. tructure and discusses the many technical challenges and design KEYWORDS decisions. This coverage infrastructure is integrated at multiple points in the development workflow. This paper describes this inte- coverage, test infrastructure, industrial study gration, and reports on the adoption and perceived usefulness of ACM Reference Format: code coverage at Google, by analyzing five years of historical data Marko Ivanković, Goran Petrović, René Just, and Gordon Fraser. 2019. Code and the 512 responses, received from surveying 3000 developers. Coverage at Google. In Proceedings of the 27th ACM Joint European Software In summary, this paper makes the following contributions: Engineering Conference and Symposium on the Foundations of Software En- gineering (ESEC/FSE ’19), August 26–30, 2019, Tallinn, Estonia. ACM, New • It demonstrates that it is possible to implement a code cov- York, NY, USA,9 pages. https://doi.org/10.1145/3338906.3340459 erage infrastructure based on existing, well-established li- braries, seamlessly integrate it into the development work- 1 INTRODUCTION flow, and scale it to an industry-scale code base. Code coverage is a well established concept in computer science, • It details Google’s infrastructure for continuous code cover- and code coverage criteria such as statement coverage, branch age computation and how code coverage is visualized. coverage, and modified condition/decision coverage (MC/DC [11]) • It details adoption rates, error rates, and average code cover- are well-known measures for test suite adequacy. For example, age ratios over a five-year period. MC/DC-adequacy is required for safety-critical systems (RTCA • DO-178). Code coverage has been known and applied for decades. It reports on the perceived usefulness of code coverage, ana- lyzing 512 responses from surveyed developers at Google. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed The remainder of this paper is structured as follows. Section2 for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. provides background information and terminology. Section3 details For all other uses, contact the owner/author(s). Google’s code coverage infrastructure. Section4 describes the adop- ESEC/FSE ’19, August 26–30, 2019, Tallinn, Estonia tion of code coverage and its usefulness, as perceived by Google © 2019 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-5572-8/19/08. developers. Section5 discusses related work. Finally, Section6 con- https://doi.org/10.1145/3338906.3340459 cludes and discusses future work. 955 ESEC/FSE ’19, August 26–30, 2019, Tallinn, Estonia Marko Ivanković, Goran Petrović, René Just, and Gordon Fraser Java Go C++ Python CL 1 CL 3 CL 4 CL 5 100 Pass change 80 CL 2 CL 2 CL 2 CL 2 Fail CL 2 S1 S2 S3 S4 S5 60 mail submit Figure 1: The life cycle of a changelist (CL). 40 Percentage of lines 20 2 BACKGROUND 0 Code coverage is a well established concept, but also a broad term that requires a more precise definition when applied in practice. This section defines important terms and concepts used throughout 1 2 3 4 5 the paper. Number of statements 2.1 Projects and Changelists Figure 2: Distribution of number of statements per line for C++, Java, Go, and Python in Google’s codebase. The following two concepts are required to understand the devel- Multi-statement lines are predominantly for loops and idiomatic returns. opment workflow at Google and its integration of coverage: Lines with more than 5 statements are removed for clarity; in total, they contribute less than 1%. • Project: A project is a collection of code that the project own- ers declare in a project definition file. The project definition 2.2 Code Coverage contains a set of (i.e., patterns describing a set code paths Google’s code coverage infrastructure measures line coverage, of files) and a set of test suites and organizational metadata which is the percentage of lines of source code executed by a set such as a contact email address. of tests. Note that blank lines and comments do not count as lines • Changelist: A changelist is an atomic update to Google’s of code, and hence do not contribute towards the denominator. Be- centralized version control system. It consists of a list of files, cause Google enforces style guides for all major languages [15], line the operations to be performed on these files, and possibly coverage strongly correlates with statement coverage in Google’s the file contents to be modified or added. codebase. Figure2 gives the distribution of number of statements Figure1 illustrates the life cycle of a changelist using an example. per line. Most lines contain a single statement, and lines with more CL 1 represents the starting state of the codebase. Suppose that at than one statement predominantly contain loops (initialization, this point a developer issues a change command, which creates a condition, update) and non-trivial, nested return statements. new changelist, CL 2. Initially, this new changelist is in a “pending” For C++, Java, and Python, Google’s code coverage infrastruc- state and the developer can edit its contents. As the developer edits ture also measures branch coverage, which is the percentage of CL 2, it progresses through so called “snapshots” (represented in branches of conditional statements (e.g., if statements) executed by the diagram as S1 to S5). Once the developer is satisfied with the a set of tests. Branch coverage is equivalent to edge coverage for contents of a changelist, they issue a mail command which initiates all conditional edges in the control flow graph [27]. the code review process. At this point, automated analyses includ- This paper distinguishes between two coverage scopes: ing code coverage are started and a notification is emailed to the • Project coverage: Line coverage of a project’s test suites reviewers. The changelist is now said to be “in review”. Through over that project’s code paths. Project coverage is computed the review process the contents may change, leading to different once per day for a given project. snapshots. When the reviewers and the author approve of a change- • Changelist coverage: The union of line coverage of all test list, the author issues a submit command. In the example above, suites of all projects that are affected by the changelist—that this command first merges CL 2 with the current submitted state is, all projects whose code paths appear in the changelist diff. of the codebase (CL 4) and, if the merge is successful, runs more Changelist coverage is computed one or more times for a automated tests.
Load more

Recommended publications
  • Using the GNU Compiler Collection (GCC)
    Using the GNU Compiler Collection (GCC) Using the GNU Compiler Collection by Richard M. Stallman and the GCC Developer Community Last updated 23 May 2004 for GCC 3.4.6 For GCC Version 3.4.6 Published by: GNU Press Website: www.gnupress.org a division of the General: [email protected] Free Software Foundation Orders: [email protected] 59 Temple Place Suite 330 Tel 617-542-5942 Boston, MA 02111-1307 USA Fax 617-542-2652 Last printed October 2003 for GCC 3.3.1. Printed copies are available for $45 each. Copyright c 1988, 1989, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with the Invariant Sections being \GNU General Public License" and \Funding Free Software", the Front-Cover texts being (a) (see below), and with the Back-Cover Texts being (b) (see below). A copy of the license is included in the section entitled \GNU Free Documentation License". (a) The FSF's Front-Cover Text is: A GNU Manual (b) The FSF's Back-Cover Text is: You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development. i Short Contents Introduction ...................................... 1 1 Programming Languages Supported by GCC ............ 3 2 Language Standards Supported by GCC ............... 5 3 GCC Command Options .........................
    [Show full text]
  • Software Engineer Is .Jusiak.Net Kr Is @Jusiak.Net (0) 791-384-1386
    Krzysztof (Kris) Jusiak Software Engineer http://kr is .jusiak.net kr is @jusiak.net (0) 791-384-1386 Education 2005 - 2010 Wroclaw University of Technology Wroclaw (Poland) MSc in Computer Science, specialised in Software Engineering (Top grade) Employment 2013 – Present King London (United Kingdom) Game/Software Developer (Mobile) King is a worldwide leader in casual games with more than 30 billion games played per month globally. We are a leading interactive entertainment company for the mobile world. Our mission is to provide highly engaging content to our audience to match their mobile lifestyles: anywhere, anytime, through any platform and on any device. Software Developer in a scrum team responsible for development and releases of mobile games played by millions active daily players. 90% of my job is focused on developing software using C++14 standard for different platforms such as, iOS, Android and Facebook (HTML5 - Emscripten). My core role is to provide high quality features and review/refactor already implemented. I am also involved in recruitment process by being a technical expert on interviews. I have done a lot of improvements to the projects I have been involved in, such as, performance/compile times optimizations, introduction of static analysis tools as well as a replacement of Service Locator pattern by Dependency Injection. • Implementing and releasing multi-platform games played by more than 100 millions active daily players • Reduced time to render the textures by 10% by changing loading files caching mechanism • Implemented an integration test framework which eliminated commonly reoccurring issues 2009 – 2013 Nokia Networks Wroclaw (Poland) Software Engineer Nokia Networks is the world’s specialist in mobile broadband, which helps enable end users to do more than ever before with the world’s most efficient mobile networks, the intelligence to maximize their value and the services to make it all work together.
    [Show full text]
  • Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK
    1 From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK Abstract—This paper1 describes some of the challenges and research questions that target the most productive intersection opportunities when deploying static and dynamic analysis at we have yet witnessed: that between exciting, intellectually scale, drawing on the authors’ experience with the Infer and challenging science, and real-world deployment impact. Sapienz Technologies at Facebook, each of which started life as a research-led start-up that was subsequently deployed at scale, Many industrialists have perhaps tended to regard it unlikely impacting billions of people worldwide. that much academic work will prove relevant to their most The paper identifies open problems that have yet to receive pressing industrial concerns. On the other hand, it is not significant attention from the scientific community, yet which uncommon for academic and scientific researchers to believe have potential for profound real world impact, formulating these that most of the problems faced by industrialists are either as research questions that, we believe, are ripe for exploration and that would make excellent topics for research projects. boring, tedious or scientifically uninteresting. This sociological phenomenon has led to a great deal of miscommunication between the academic and industrial sectors. I. INTRODUCTION We hope that we can make a small contribution by focusing on the intersection of challenging and interesting scientific How do we transition research on static and dynamic problems with pressing industrial deployment needs. Our aim analysis techniques from the testing and verification research is to move the debate beyond relatively unhelpful observations communities to industrial practice? Many have asked this we have typically encountered in, for example, conference question, and others related to it.
    [Show full text]
  • Addresssanitizer + Code Coverage Kostya Serebryany, Google Eurollvm 2014 New and Shiny -Fprofile-Instr-Generate
    AddressSanitizer + Code Coverage Kostya Serebryany, Google EuroLLVM 2014 New and shiny -fprofile-instr-generate ● Coming this year ● Fast BB-level code coverage ● Increment a counter per every (*) BB ○ Possible contention on counters ● Creates special non-code sections ○ Counters ○ Function names, line numbers Meanwhile: ASanCoverage ● Tiny prototype-ish thing: ○ Part of AddressSanitizer ○ 30 lines in LLVM, 100 in run-time ● Function- or BB- level coverage ○ Booleans only, not counters ○ No contention ○ No extra sections in the binary At compile time: if (!*BB_Guard) { __sanitizer_cov(); *BB_Guard = 1; } At run time void __sanitizer_cov() { Record(GET_CALLER_PC()); } At exit time ● For every binary/DSO in the process: ○ Dump observed PCs in a separate file as 4-byte offsets At analysis time ● Compare/Merge using 20 lines of python ● Symbolize using regular DWARF % cat cov.c int main() { } % clang -g -fsanitize=address -mllvm -asan-coverage=1 cov. c % ASAN_OPTIONS=coverage=1 ./a.out % wc -c *sancov 4 a.out.15751.sancov % sancov.py print a.out.15751.sancov sancov.py: read 1 PCs from a.out.15751.sancov sancov.py: 1 files merged; 1 PCs total 0x4850b7 % sancov.py print *.sancov | llvm-symbolizer --obj=a.out main /tmp/cov.c:1:0 Fuzzing with coverage feedback ● Test corpus: N random tests ● Randomly mutate random test ○ If new BB is covered -- add this test to the corpus ● Many new bugs in well fuzzed projects! Feedback from our customers ● Speed is paramount ● Binary size is important ○ Permanent & temporary storage, tmps, I/O ○ Stripping non-code
    [Show full text]
  • Selenium Python Bindings Release 2
    Selenium Python Bindings Release 2 Baiju Muthukadan Sep 03, 2021 Contents 1 Installation 3 1.1 Introduction...............................................3 1.2 Installing Python bindings for Selenium.................................3 1.3 Instructions for Windows users.....................................3 1.4 Installing from Git sources........................................4 1.5 Drivers..................................................4 1.6 Downloading Selenium server......................................4 2 Getting Started 7 2.1 Simple Usage...............................................7 2.2 Example Explained............................................7 2.3 Using Selenium to write tests......................................8 2.4 Walkthrough of the example.......................................9 2.5 Using Selenium with remote WebDriver................................. 10 3 Navigating 13 3.1 Interacting with the page......................................... 13 3.2 Filling in forms.............................................. 14 3.3 Drag and drop.............................................. 15 3.4 Moving between windows and frames.................................. 15 3.5 Popup dialogs.............................................. 16 3.6 Navigation: history and location..................................... 16 3.7 Cookies.................................................. 16 4 Locating Elements 17 4.1 Locating by Id.............................................. 18 4.2 Locating by Name............................................ 18 4.3
    [Show full text]
  • N $NUM GPUS Python Train.Py
    Neural Network concurrency Tal Ben-Nun and Torsten Hoefler, Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis, 2018, Data Parallelism vs Model Parallelism Hardware and Libraries ● It is not only a matter of computational power: ○ CPU (MKL-DNN) ○ GPU (cuDNN) ○ FGPA ○ TPU ● Input/Output matter ○ SSD ○ Parallel file system (if you run parallel algorithm) ● Communication and interconnection too, if you are running in distributed mode ○ MPI ○ gRPC +verbs (RDMA) ○ NCCL Install TensorFlow from Source [~]$ wget https://github.com/.../bazel-0.15.2-installer-linux-x86_64.sh [~]$ ./bazel-0.15.2-installer-linux-x86_64.sh --prefix=... [~]$ wget https://github.com/tensorflow/tensorflow/archive/v1.10.0.tar.gz ... [~]$ python3 -m venv $TF_INSTALL_DIR [~]$ source $TF_INSTALL_DIR/bin/activate [~]$ pip3 install numpy wheel [~]$ ./configure ... [~]$ bazel build --config=mkl/cuda \ //tensorflow/tools/pip_package:build_pip_package [~]$ bazel-bin/tensorflow/tools/pip_package/build_pip_package $WHEELREPO [~]$ pip3 install $WHEELREPO/$WHL --ignore-installed [~]$ pip3 install keras horovod ... Input pipeline If using accelerators like GPU, pipeline tha data load exploiting the CPU with the computation on GPU The tf.data API helps to build flexible and efficient input pipelines Optimizing for CPU ● Built from source with all of the instructions supported by the target CPU and the MKL-DNN option for Intel® CPU. ● Adjust thread pools ○ intra_op_parallelism_threads: Nodes that can use multiple threads to parallelize their execution
    [Show full text]
  • Erik Lindahl
    HPC Software Engineering Erik Lindahl XSEDE/PRACE/RIKEN/SciNet HPC Summer School Ostrava, CZ, 2018 Extra Q & A session: Here (or outside at the tables) 12.45 today! “The application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software, and the study of these approaches, that is, the application of engineering to software.” Experiences from 20 years of GROMACS development The GROMACS picture until early 2011 Source code repository: • Simulation hardware project, turned software CVS • Early development based on our own needs • Turned GPL in 2001, LGPL in 2012 Build Chain: • Organic growth of development Automake/Autoconf/libtool • Roughly 10-15 core developers Bug Tracking: • Another 15-20 active contributors Bugzilla • Currently 3,076,420 lines of C++11 code (“C++11”) • Over the years we have used Fortran, C, Assembly Testing: • Lots of old code. Lots of new code. Lots of complicated (read: bad) code written by scientists Software Scientist engineer • Trained in physics, • Trained in CS/software chemistry, etc. • Care about their problem • Care about their code • Care about short-term • Care about long-term deadlines maintenance • New code = asset • New code = liability • Writes more code than • Reads much more code she reads than she writes Without proper software engineering, we are building a technical debt that sooner or later will have to be paid. “Technical Debt is a wonderful metaphor developed by Ward Cunningham to help us think about this problem. In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt.
    [Show full text]
  • Blau Mavi Blue
    4 / 2014 Quartierzeitung für das Untere Kleinbasel Mahalle Gazetesi Aşağ Küçükbasel için www.mozaikzeitung.ch Novine za cˇetvrt donji Mali Bazel Blau Mavi Blue Bilder, Stimmungen, Töned i t Resimler, Farkli sessler, d i Tonlart visions, moods, sounds e ˇ ivotu. Bazelu pricˇa21 o svom Z Foto: Jum Soon Kim Spezial:Jedna Familija u malom HOLZKOMPETENZ BACK INT NACH MASS BAL NCNNCECE Ihre Wunschvorstellung. Unser ALEXANDER-TECHNIK Handwerk. Resultat: Möbel Christina Stahlberger und Holzkonstruktionen, die dipl. Lehrerin für Alexander-Technik SVLAT M_000278 Matthäusstrasse 7, 4057 Basel Sie ein Leben lang begleiten. +41 (0)77 411 99 89 Unsere Spezialgebiete sind [email protected] I www.back-into-balance.com Haus- und Zimmertüren, Schränke, Küchen und Bade- Ed. Borer AG · Schreinerei · Wiesenstrasse 10 · 4057 Basel zimmermöbel sowie Repara- T 061 631 11 15 · F 061 631 11 26 · [email protected] turen und Restaurationen. M_000236 M_000196 Stadtteilsekretariat Kleinbasel M_000028 Darf ich hier Für Fragen, Anliegen und Probleme betreffend: • Wohnlichkeit und Zusammenleben grillieren? • Mitwirkung der Quartierbevölkerung Öffnungszeiten: Mo, Di und Do, 15 – 18.30 h Klybeckstrasse 61, 4057 Basel Tel: 061 681 84 44, Email: [email protected] www.stadtteilsekretariatebasel.ch M_000024 Wir danken unserer Kundschaft Offenburgerstrasse 41, CH-4057 Basel 061 5 54 23 33 Täglich bis 22.00 Uhr geöffnet! Täglich frische Produkte und Bio-Produkte! 365 Tage im Jahr, auch zwischen Weihnacht und Neujahr offen! Lebensmittel- und Getränkemarkt Wir bieten stets beste Qualität und freuen uns auf Ihren Besuch. Gratis-Lieferdienst für ältere Menschen M_000280 Öffnungszeiten: Montag 11.00–22.00 Uhr Dienstag–Sonntag und Feiertage: 8.00–22.00 Uhr Feldbergstrasse 32, 4057 Basel, Telefon 061 693 00 55 M_000006 M_000049 LACHENMEIER.CH SCHREINEREI konstruiert.
    [Show full text]
  • How Google Tests Software
    How Google Tests Software James Whittaker Jason Arbon Jeff Carollo • nTr Addison-Wesley Upper Saddle River, NJ • Boston • Indianapolis • San Francisco New York • Toronto • Montreal • London • Munich • Paris • Madrid Capetown • Sydney • Tokyo • Singapore • Mexico City ' ' ^ : Г: С : '. , Table of Contents Foreword by Alberto Savoia Xlll Foreword by Patrick Copeland xvii Preface ХХШ Chapter 1 Introduction to Google Software Testing 1 Quality^Test 5 Roles 6 Organizational Structure 8 Crawl, Walk, Run 10 Types of Tests 12 Chapter 2 The Software Engineer in Test 15 The Life of an SET 17 Development and Test Workflow 17 Who Are These SETs Anyway? 22 The Early Phase of a Project 22 Team Structure 24 Design Docs 25 Interfaces and Protocols 27 Automation Planning 28 Testability 29 SET Workflow: An Example 32 Test Execution 40 Test Size Definitions 41 Use of Test Sizes in Shared Infrastructure 44 Benefits of Test Sizes 46 Test Runtime Requirements 48 Case 1: Change in Common Library 52 Test Certified 54 An Interview with the Founders of the Test Certified Program 57 Interviewing SETs 62 An Interview with Tool Developer Ted Mao 68 An Interview with Web Driver Creator Simon Stewart 70 How Google Tests Software Chapter 3 The Test Engineer 75 A User-Facing Test Role 75 The Life of а ТЕ 76 Test Planning 79 Risk 97 Life of a Test Case 108 Life of a Bug 113 Recruiting TEs 127 Test Leadership at Google 134 Maintenance Mode Testing 137 Quality Bots Experiment 141 BITE Experiment 153 Google Test Analytics 163 Free Testing Workflow 169 External Vendors 173
    [Show full text]
  • Installing and Running Tensorflow
    Installing and Running Tensorflow DOWNLOAD AND INSTALLATION INSTRUCTIONS TensorFlow is now distributed under an Apache v2 open source license on GitHub. STEP 1. INSTALL NVIDIA CUDA To use TensorFlow with NVIDIA GPUs, the first step is to install the CUDA Toolkit. STEP 2. INSTALL NVIDIA CUDNN Once the CUDA Toolkit is installed, download cuDNN v5.1 Library for Linux (note that you will need to register for the Accelerated Computing Developer Program). Once downloaded, uncompress the files and copy them into the CUDA Toolkit directory (assumed here to be in /usr/local/cuda/): $ sudo tar -xvf cudnn-8.0-linux-x64-v5.1-rc.tgz -C /usr/local STEP 3. INSTALL AND UPGRADE PIP TensorFlow itself can be installed using the pip package manager. First, make sure that your system has pip installed and updated: $ sudo apt-get install python-pip python-dev $ pip install --upgrade pip STEP 4. INSTALL BAZEL To build TensorFlow from source, the Bazel build system must first be installed as follows. Full details are available here. $ sudo apt-get install software-properties-common swig $ sudo add-apt-repository ppa:webupd8team/java $ sudo apt-get update $ sudo apt-get install oracle-java8-installer $ echo "deb http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list $ curl https://storage.googleapis.com/bazel-apt/doc/apt-key.pub.gpg | sudo apt-key add - $ sudo apt-get update $ sudo apt-get install bazel STEP 5. INSTALL TENSORFLOW To obtain the best performance with TensorFlow we recommend building it from source. First, clone the TensorFlow source code repository: $ git clone https://github.com/tensorflow/tensorflow $ cd tensorflow $ git reset --hard 70de76e Then run the configure script as follows: $ ./configure Please specify the location of python.
    [Show full text]
  • Googletest - Tips and Frequently-Asked Questions About Google C++ Testing Framework - Google C++ Testing Framework - Google Project Hosting
    8/23/13 FAQ - googletest - Tips and Frequently-Asked Questions about Google C++ Testing Framework - Google C++ Testing Framework - Google Project Hosting My favorites ▼ | Sign in googletest Google C++ Testing Framework Search projects Project Home Downloads Wiki Issues Source Search Current pages for Search FAQ Tips and Frequently­Asked Questions about Google C++ Testing Framework Updated Jul 7, 2013 by [email protected] Why should I use Google Test instead of my favorite C++ testing framework? I'm getting warnings when compiling Google Test. Would you fix them? Why should not test case names and test names contain underscore? Why is it not recommended to install a pre­compiled copy of Google Test (for example, into /usr/local)? How do I generate 64­bit binaries on Windows (using Visual Studio 2008)? Can I use Google Test on MinGW? Why does Google Test support EXPECT_EQ(NULL, ptr) and ASSERT_EQ(NULL, ptr) but not EXPECT_NE(NULL, ptr) and ASSERT_NE(NULL, ptr)? Does Google Test support running tests in parallel? Why don't Google Test run the tests in different threads to speed things up? Why aren't Google Test assertions implemented using exceptions? Why do we use two different macros for tests with and without fixtures? Why don't we use structs as test fixtures? Why are death tests implemented as assertions instead of using a test runner? My death test modifies some state, but the change seems lost after the death test finishes. Why? The compiler complains about "undefined references" to some static const member variables, but I did define them in the class body.
    [Show full text]
  • Read the Thesis
    Making compiler tests based on compiler source code coverage Jelle Witsen Elias [email protected] January 19, 2021, 33 pages Academic supervisor: Clemens Grelck, [email protected] Daily supervisors: Jos´eLuis March Cabrelles, [email protected] Remi van Veen, [email protected] Host organisation: Solid Sands, www.solidsands.nl Universiteit van Amsterdam Faculteit der Natuurwetenschappen, Wiskunde en Informatica Master Software Engineering http://www.software-engineering-amsterdam.nl Abstract Compilers are important elements in software infrastructure. It is therefore important that these complex pieces of software are tested. We found a way of making C compiler tests that is different from the usual approach of doing this. With our new process we base new tests on their coverage of compiler source code, instead of constructing them by solely looking at the rules defined in language standards. This way, we manage to make tests with constructions that were overlooked when building our existing compiler test suites. We built a tool that takes source code of a C application as input, along with a C compiler test suite. The tool outputs pieces of code which, when compiled, cover parts of compiler source code that the test suite does not yet cover. With this output, we can construct new tests for inclusion in our existing test suite. Our process is as follows: we take an open source C application and compile its sources with an open source compiler. This compiler has been instrumented to enable source code coverage analysis. After compilation, we compare the coverage of compiler source code of the application with coverage of our existing test suites.
    [Show full text]