Génération Automatique De La Documentation D'un Code

Total Page:16

File Type:pdf, Size:1020Kb

Génération Automatique De La Documentation D'un Code Introduction Doxygen Sphinx Remarques R´ef´erences G´en´erationautomatique de la documentation d'un code Anne Cadiou Laboratoire de M´ecaniquedes Fluides et d'Acoustique Ateliers et S´eminairesPour l'Informatique et le Calcul Scientifique PMCS2I - LMFA Vendredi 4 juin 2021 1/39 Introduction Doxygen Sphinx Remarques R´ef´erences Motivation Pourquoi g´en´ererautomatiquement une documentation ? • destin´e`amieux comprendre de un code, en faciliter la lecture et le d´eveloppement • les lignes de codes parlent par elles-m^emedu comment, mais il est parfois n´ecessaired'ajouter un commentaire sur le pourquoi • des commentaires sont pr´esentsdans le code, pour expliquer ou justifier les lignes de codes • ´ecrireun document `apart du code est difficile `amaintenir `ajour • avoir un syst`emequi extrait les commentaires du code permet aussi d'am´eliorer la qualit´ede ces commenaires • extraire une documentation du code permet aussi de tenir compte de la structure du code Un syst`emede g´en´erationautomatique de documentation `apartir d'un code extrait des informations `apartir des sources et les compile en une documentation qui ´evolue automatiquement avec le code. 2/39 Introduction Doxygen Sphinx Remarques R´ef´erences Outils existants tr`esnombreux https://en.wikipedia.org/wiki/Comparison of documentation generators cela d´epend • de leur licence • du langage avec lequel le code `adocumenter est ´ecrit • de leurs int´egrationdans d'´eventuelsIDE • de leurs formats de sortie • de la taille du code `adocumenter • des syst`emesd'exploitation support´es,etc. Outils les plus r´epandus en C, C++, FORTRAN, Python : • Doxygen • Sphinx Certains sont d´edi´es`aun langage, par exemple pydoctor (rempla¸cant d'Epydoc), pdoc ou pydoc pour Python, ROBODoc, FORD pour FORTRAN, etc. 3/39 Introduction Doxygen Sphinx Remarques R´ef´erences Doxygen (1997-) derni`ereversion 2021 Langages support´es C++, C, Python, FORTRAN, Objective-C, C#, PHP, Java, IDL, VHDL, D Bas´esur un ensemble de balises `aajouter aux sources Formats de sortie • documentation `apartir des sources document´ees: HTML, LATEX, RTF (MS-Word), PS, PDF, XML, Unix man pages • extraction de la structure du code pour des sources document´eesou non. g´en`ereles graphes de d´ependances, les h´eritagesde classes, etc. • fonctionne sous Mac OS X, Linux, Windows • Eclipse, VS Code, vim (:Dox), • sous licence GPL 4/39 Introduction Doxygen Sphinx Remarques R´ef´erences Installation sous ubuntu sudo apt-get install doxygen doxygen-gui doxygen-doc Pour ceux qui aiment les interfaces graphiques : sudo apt-get install graphviz qui se lance avec la commande doxywizard & Doxygen se lance en ligne de commande par doxygen 5/39 Introduction Doxygen Sphinx Remarques R´ef´erences Balises C ou C++ FORTRAN Python /*! !< """ * \brief !! \brief \brief * \value !! \value \value */ !! """ ou ou /** !< Pas vraiment optimal * \brief !< \brief pour Python * \value !< \value */ !< (d'autres styles existent) Commentaires priv´es // sur une ligne ! ligne exclue de la doc automatique /* * sur * plusieurs lignes */ 6/39 Introduction Doxygen Sphinx Remarques R´ef´erences Commandes Pr´efix´eespar @ ou n Principales commandes : Permet aussi d'´ecriredu LATEXsimplifi´e • nfile nom du fichier d´ecrit • nff nfg pour les formules • nclass math´ematiques • • nfunc nmainpage page principale • • ndef nom de la macro nsection • • nparam param`etrespass´es nsubsection • nbrief br`evedescription • ndetails • nauthor • ndate • nversion • ncopyright 7/39 Introduction Doxygen Sphinx Remarques R´ef´erences Emplacement Par d´efaut,les commentaires r´ecup´er´espar Doxygen doivent ^etreplac´esavant la structure `adocumenter. Exemple : /** * \brief The function bar. * * \details This function does something. * * \param[in] a Description of parameter a. * \param[out] b Description of the parameter b. * \param[in,out] c Description of the parameter c. * * \return The error return code of the function. * */ errcode_t bar(int a, int b, int c) { /** More detailed description inside the code */ } 8/39 Introduction Doxygen Sphinx Remarques R´ef´erences Utilisation Premi`ere´etape : g´en´ererle fichier de configuration (Doxyfile par d´efaut) doxygen -g Deuxi`eme´etape : variables `arenseigner PROJECT_NAME = Nom du code INPUT = Localisation des sources du code USE_MATHJAX = YES # permet d'inserer des formules de math en \LaTeX GENERATE_XML = YES # permet l'interface avec Sphinx utile pour le FORTRAN EXTRACT_ALL = YES OPTIMIZE_FOR_FORTRAN = YES utile pour les graphes d'appel CALL_GRAPH = YES CALLER_GRAPH = YES Derni`ere´etape : g´en´ererla documentation doxygen Doxyfile 9/39 Introduction Doxygen Sphinx Remarques R´ef´erences Exemple en FORTRAN Codes source project/ src/ burgers.f90 Makefile mexact.f90 mio.f90 mscheme.f90 msolver.f90 bin/ 10/39 Introduction Doxygen Sphinx Remarques R´ef´erences Ajout de balises de commentaires program burgers !> @mainpage Solve Burger's equations !! @author Anne Cadiou !! @date 19/04/2021 !! !! @details !! solve the Burgers'1D equation !! @f\[ !!\frac{\partialu}{\partialt}+u\frac{\partialu}{\partialx}=\nu\frac{\ partial^2u}{\partialx^2} !! @f\] !! !! with either one of these schemes !!- time explicit Euler and second order space centered !!- semi-implicit Crank-Nicolson !! use mexact use msolver use mscheme use mio implicit none integer, parameter:: dp = kind(0.D0) (...) 11/39 Introduction Doxygen Sphinx Remarques R´ef´erences G´en´erationdu fichier de configuration Cr´eationd'un r´epertoire o`ug´en´ererla documentation mkdir -p docs En ligne de commande doxygen -g Avec l'interface graphique doxywizard & 12/39 Introduction Doxygen Sphinx Remarques R´ef´erences 13/39 Introduction Doxygen Sphinx Remarques R´ef´erences 14/39 Introduction Doxygen Sphinx Remarques R´ef´erences 15/39 Introduction Doxygen Sphinx Remarques R´ef´erences 16/39 Introduction Doxygen Sphinx Remarques R´ef´erences 17/39 Introduction Doxygen Sphinx Remarques R´ef´erences 18/39 Introduction Doxygen Sphinx Remarques R´ef´erences 19/39 Introduction Doxygen Sphinx Remarques R´ef´erences 20/39 Introduction Doxygen Sphinx Remarques R´ef´erences 21/39 Introduction Doxygen Sphinx Remarques R´ef´erences 22/39 Introduction Doxygen Sphinx Remarques R´ef´erences 23/39 Introduction Doxygen Sphinx Remarques R´ef´erences Documentation project/ docs/ html/ latex/ xml/ src/ Lecture des pages firefox html/index.html G´en´erationd'un pdf LATEX cd latex make 24/39 Introduction Doxygen Sphinx Remarques R´ef´erences 25/39 Introduction Doxygen Sphinx Remarques R´ef´erences 26/39 Introduction Doxygen Sphinx Remarques R´ef´erences 27/39 Introduction Doxygen Sphinx Remarques R´ef´erences 28/39 Introduction Doxygen Sphinx Remarques R´ef´erences Sphinx (2008-) derni`ere version 2018 Langages support´es Python, PHP, JavaScript Expoite les commentaires pr´esentsdans les sources Formats de sortie • documentation `apartir des sources document´ees: HTML, LATEX, PS, PDF, XML, Unix man pages • extensions • fonctionne sous Mac OS X, Linux, Windows • sous licence BSD 29/39 Introduction Doxygen Sphinx Remarques R´ef´erences Exemple project/ src/ data manager.py physical system.py run.py solver.py 30/39 Introduction Doxygen Sphinx Remarques R´ef´erences Utilisation Premi`ere´etape : initialiser la documentation sphinx-quickstart R´epondre aux questions. Recommandation : s´eparer les r´epertoires build et source Welcome to the Sphinx 1.8.5 quickstart utility. Please enter values for the following settings (just press Enter to accept a default value, if one is given in brackets). Selected root path: . You have two options for placing the build directory for Sphinx output. Either, you use a directory "_build" within the root path, or you separate "source" and "build" directories within the root path. > Separate source and build directories (y/n) [n]: y (...) 31/39 Introduction Doxygen Sphinx Remarques R´ef´erences (...) > Create Makefile? (y/n) [y]: y > Create Windows command file? (y/n) [y]: n Creating file ./source/conf.py. Creating file ./source/index.rst. Creating file ./Makefile. Finished: An initial directory structure has been created. You should now populate your master file ./source/index.rst and create other documentation source files. Use the Makefile to build the docs, like so: make builder where "builder" is one of the supported builders, e.g. html, latex or linkcheck. La documentation sera g´en´er´eedans source (ici renomm´een docs) mv source docs dans ce cas, modifier aussi dans le Makefile SOURCEDIR = docs soit dans textttmake.bat pour Windows set SOURCEDIR=docs 32/39 Introduction Doxygen Sphinx Remarques R´ef´erences Configuration cd docs Dans le fichier conf.py expliciter le chemin vers les modules import os import sys sys.path.insert(0, os.path.abspath('../src')) et ajouter l'extension vers l'auto-documentation extensions = [ 'sphinx.ext.autodoc' ] Le style peut ^etremodifi´e,par exemple au lieu de alabaster, pour passer `a rtd, ajouter extensions = [ 'sphinx.ext.autodoc', 'sphinx_rtd_theme', ] html_theme = 'sphinx_rtd_theme' 33/39 Introduction Doxygen Sphinx Remarques R´ef´erences G´en´erationautomatique A` la racine du projet (o`use trouve le Makefile) ex´ecuter sphinx-apidoc -f -o docs src cela g´en`ereles pages de documentation pour chaque fichier python et un document modules Creating file docs/data_manager.rst. Creating file docs/physical_system.rst. Creating file docs/run.rst. Creating file docs/solver.rst. Creating file docs/modules.rst. Ajouter le document modules dans le document principal docs/index.rst g´en´er´epar sphinx .. Lorenz documentation master file, created by sphinx-quickstart
Recommended publications
  • Epydoc: API Documentation Extraction in Python
    Epydoc: API Documentation Extraction in Python Edward Loper Department of Computer and Information Science University of Pennsylvania, Philadelphia, PA 19104-6389, USA Abstract • All API documentation must be written (and read) in plaintext. Epydoc is a tool for generating API documentation for Python modules, based on their docstrings. It • There is no easy way to navigate through the supports several output formats (including HTML API documentation. and PDF), and understands four different markup • The API documentation is not searchable. languages (Epytext, Javadoc, reStructuredText, and plaintext). A wide variety of fields can be used to • A library's API documentation cannot be viewed supply specific information about individual objects, until that library is installed. such as descriptions of function parameters, type sig- natures, and groupings of related objects. • There is no mechanism for documenting vari- ables. 1 Introduction • There is no mechanism for \inheriting" docu- mentation (e.g. in a method that overrides its Documentation is a critical contributor to a library's base class method). This can lead to dupli- usability. Thorough documentation shows new users cation of documentation, which can often get how to use a library; and details the library's specific out-of-sync. behavior for advanced users. Most libraries can ben- efit from three different types of documentation: tu- Epydoc is a tool that automatically extracts a li- torial documentation, which introduces new users to brary's docstrings, and uses them to create API doc- the library by showing them how to perform typical umentation for the library in a variety of formats. tasks; reference documentation, which explains the li- Epydoc addresses all of these limitations: brary's overall design, and describes how the different • pieces of the library fit together; and API documenta- Docstrings can be written in a variety of markup tion, which describes the individual objects (classes, languages, including reStructuredText and Javadoc.
    [Show full text]
  • Python Guide Documentation Publicación 0.0.1
    Python Guide Documentation Publicación 0.0.1 Kenneth Reitz 17 de May de 2018 Índice general 1. Empezando con Python 3 1.1. Eligiendo un Interprete Python (3 vs. 2).................................3 1.2. Instalando Python Correctamente....................................5 1.3. Instalando Python 3 en Mac OS X....................................6 1.4. Instalando Python 3 en Windows....................................8 1.5. Instalando Python 3 en Linux......................................9 1.6. Installing Python 2 on Mac OS X.................................... 10 1.7. Instalando Python 2 en Windows.................................... 12 1.8. Installing Python 2 on Linux....................................... 13 1.9. Pipenv & Ambientes Virtuales...................................... 14 1.10. Un nivel más bajo: virtualenv...................................... 17 2. Ambientes de Desarrollo de Python 21 2.1. Your Development Environment..................................... 21 2.2. Further Configuration of Pip and Virtualenv............................... 26 3. Escribiendo Buen Código Python 29 3.1. Estructurando tu Proyecto........................................ 29 3.2. Code Style................................................ 40 3.3. Reading Great Code........................................... 49 3.4. Documentation.............................................. 50 3.5. Testing Your Code............................................ 53 3.6. Logging.................................................. 57 3.7. Common Gotchas...........................................
    [Show full text]
  • Week 7 " Reuse Your Own Code Top 20 Tools of All Time CS 212 – Spring 2008 (
    Programming Language as a Tool Software Tools ! Use the language that best fits your task ! Think small " Write little programs that test various concepts " Test them! " Comment them! " Build collections of these little programs Week 7 " Reuse your own code Top 20 Tools of All Time CS 212 – Spring 2008 (http://uk.gizmodo.com/) Languages for Different Domains Scripting Languages ! General purpose ! Concurrent/distributed ! A script is a sequence of ! Example scripting languages: " Examples: Lisp, Algol, PL/1, processes common commands made into a Unix shell, Python, Perl, Scheme, Java, Python " Control of multiple threads single program Tcl (Tool command language) ! Systems programming " Examples: Ada, Oz, Smalltalk, " Unix uses shell scripts " Emphasis on efficiency and Java " The shell is the interactive ! Some Python code: tight control of data ! Educational interface to Unix structures " " Examples: Basic, Haskell, You can combine commands class Stack (object): " Examples: C, C++, Forth, Pascal, Python, Scheme, from the Unix shell to create def __init__ (self): Modula-2 Smalltalk programs self.stack = [ ] ! Scripting ! Various other domains def put (self, item): self.stack.append(item) " ! Examples: Unix shell, Perl, " Discrete event simulation: A scripting language is usually def get (self): Python, Ruby, Tcl Simula " Easy to learn return self.stack.pop() " Web scripting: Javascript " Interpreted instead of def isEmpty (self): " Realtime applications: Ada compiled return len(self.stack) == 0 " Text processing: Snobol, Perl " Printing:
    [Show full text]
  • Application Programming Interface (API) Is a Specification Intended to Be Used As an Interface by Software Components to Communicate with Each Other
    Application programming interface 1 Application programming interface An application programming interface (API) is a specification intended to be used as an interface by software components to communicate with each other. An API may include specifications for routines, data structures, object classes, and variables. An API specification can take many forms, including an International Standard such as POSIX or vendor documentation such as the Microsoft Windows API, or the libraries of a programming language, e.g. Standard Template Library in C++ or Java API. An API differs from an application binary interface (ABI) in that the former is source code based while the latter is a binary interface. For instance POSIX is an API, while the Linux Standard Base is an ABI.[1] Language used An API can be: • language-dependent, meaning it is only available by using the syntax and elements of a particular language, which makes the API more convenient to use. • language-independent, written so that it can be called from several programming languages. This is a desirable feature for a service-oriented API that is not bound to a specific process or system and may be provided as remote procedure calls or web services. For example, a website that allows users to review local restaurants is able to layer their reviews over maps taken from Google Maps, because Google Maps has an API that facilitates this functionality. Google Maps' API controls what information a third-party site can use and how they can use it. The term API may be used to refer to a complete interface, a single function, or even a set of APIs provided by an organization.
    [Show full text]
  • Red Hat Enterprise Linux Developer's Getting Started Guide
    WHITE PAPER RED HAT ENTERPRISE LINUX DEVELOPER'S GETTING STARTED GUIDE EXECUTIVE SUMMARY Red Hat Enterprise Linux is an enterprise-class open-source operating system that is widely adopted world wide, scaling seamlessly from individual desktops to large servers in the datacenter. Certified by leading hardware and software vendors, Red Hat Enterprise Linux delivers high performance, reliability, and security along with flexibility, efficiency, and control. For developers, Red Hat provides an extensive set of resources, technologies, and tools that can be used to efficiently develop powerful applications for the Red Hat Enterprise Linux platform. These applications can be deployed with great flexibility, as Red Hat Enterprise Linux supports major hardware architectures, comprehensive virtualization solutions, and a range of cloud-computing options. This document is intended for software developers who are new to Red Hat Enterprise Linux and want to understand the key touch points for any phase of application development – from planning and building, through testing and deploying. The following sections describe the resources and tools that are available on Red Hat Enterprise Linux and provide links to additional information. www.redhat.com WHITE PAPER RED HAT ENTERPRISE LINUX DEVELOPER'S GETTING STARTED GUIDE TABLE OF CONTENTS Developing Software On Red Hat Enterprise Linux................................................................................................4 Overview..................................................................................................................................................................................
    [Show full text]
  • A Language-Independent Static Checking System for Coding Conventions
    A Language-Independent Static Checking System for Coding Conventions Sarah Mount A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy 2013 This work or any part thereof has not previously been presented in any form to the University or to any other body whether for the purposes of as- sessment, publication or for any other purpose (unless otherwise indicated). Save for any express acknowledgements, references and/or bibliographies cited in the work, I confirm that the intellectual content of the work is the result of my own efforts and of no other person. The right of Sarah Mount to be identified as author of this work is asserted in accordance with ss.77 and 78 of the Copyright, Designs and Patents Act 1988. At this date copyright is owned by the author. Signature: . Date: . Abstract Despite decades of research aiming to ameliorate the difficulties of creat- ing software, programming still remains an error-prone task. Much work in Computer Science deals with the problem of specification, or writing the right program, rather than the complementary problem of implementation, or writing the program right. However, many desirable software properties (such as portability) are obtained via adherence to coding standards, and there- fore fall outside the remit of formal specification and automatic verification. Moreover, code inspections and manual detection of standards violations are time consuming. To address these issues, this thesis describes Exstatic, a novel framework for the static detection of coding standards violations. Unlike many other static checkers Exstatic can be used to examine code in a variety of lan- guages, including program code, in-line documentation, markup languages and so on.
    [Show full text]
  • Python Guide Documentation Publicación 0.0.1
    Python Guide Documentation Publicación 0.0.1 Kenneth Reitz 15 de November de 2016 Índice general 1. Getting Started with Python 3 1.1. Eligiendo un Interprete..........................................3 1.2. Properly Installing Python........................................5 1.3. Installing Python on Mac OS X.....................................6 1.4. Installing Python on Windows......................................7 1.5. Installing Python on Linux........................................8 2. Writing Great Python Code 11 2.1. Structuring Your Project......................................... 11 2.2. Code Style................................................ 21 2.3. Reading Great Code........................................... 31 2.4. Documentation.............................................. 31 2.5. Testing Your Code............................................ 33 2.6. Logging.................................................. 37 2.7. Common Gotchas............................................ 40 2.8. Choosing a License............................................ 43 3. Scenario Guide for Python Applications 45 3.1. Network Applications.......................................... 45 3.2. Web Applications............................................ 46 3.3. HTML Scraping............................................. 52 3.4. Command-line Applications....................................... 54 3.5. GUI Applications............................................. 55 3.6. Databases................................................. 57 3.7. Networking...............................................
    [Show full text]
  • How to Think Like a Computer Scientist: Learning with Python Documentation Release 2Nd Edition
    How to Think Like a Computer Scientist: Learning with Python Documentation Release 2nd Edition Jeffrey Elkner, Allen B. Downey and Chris Meyers February 22, 2017 CONTENTS 1 Learning with Python3 Index 295 i ii How to Think Like a Computer Scientist: Learning with Python Documentation, Release 2nd Edition CONTENTS 1 How to Think Like a Computer Scientist: Learning with Python Documentation, Release 2nd Edition 2 CONTENTS CHAPTER ONE LEARNING WITH PYTHON 2nd Edition (Using Python 2.x) by Jeffrey Elkner, Allen B. Downey, and Chris Meyers Last Updated: 21 April 2012 • Copyright Notice • Foreword • Preface • Contributor List • Chapter 1 The way of the program • Chapter 2 Variables, expressions, and statements • Chapter 2b A light look at lists and looping • Chapter 3 Functions • Chapter 4 Conditionals • Chapter 5 Fruitful functions • Chapter 6 Iteration • Chapter 7 Strings • Chapter 8 Case Study: Catch • Chapter 9 Lists • Chapter 10 Modules and files • Chapter 11 Recursion and exceptions • Chapter 12 Dictionaries • Chapter 13 Classes and objects • Chapter 14 Classes and functions 3 How to Think Like a Computer Scientist: Learning with Python Documentation, Release 2nd Edition • Chapter 15 Classes and methods • Chapter 16 Sets of Objects • Chapter 17 Inheritance • Chapter 18 Linked Lists • Chapter 19 Stacks • Chapter 20 Queues • Chapter 21 Trees • Appendix A Debugging • Appendix B GASP • Appendix c Configuring Ubuntu for Python Development • Appendix D Customizing and Contributing to the Book • GNU Free Document License Copyright Notice Copyright (C) Jeffrey Elkner, Allen B. Downey and Chris Meyers. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with Invariant Sections being Foreward, Preface, and Contributor List, no Front-Cover Texts, and no Back-Cover Texts.
    [Show full text]
  • Ipython Documentation Release 0.10.1
    IPython Documentation Release 0.10.1 The IPython Development Team October 11, 2010 CONTENTS 1 Introduction 1 1.1 Overview............................................1 1.2 Enhanced interactive Python shell...............................1 1.3 Interactive parallel computing.................................3 2 Installation 5 2.1 Overview............................................5 2.2 Quickstart...........................................5 2.3 Installing IPython itself....................................6 2.4 Basic optional dependencies..................................7 2.5 Dependencies for IPython.kernel (parallel computing)....................8 2.6 Dependencies for IPython.frontend (the IPython GUI).................... 10 3 Using IPython for interactive work 11 3.1 Quick IPython tutorial..................................... 11 3.2 IPython reference........................................ 17 3.3 IPython as a system shell.................................... 42 3.4 IPython extension API..................................... 47 4 Using IPython for parallel computing 53 4.1 Overview and getting started.................................. 53 4.2 Starting the IPython controller and engines.......................... 57 4.3 IPython’s multiengine interface................................ 64 4.4 The IPython task interface................................... 78 4.5 Using MPI with IPython.................................... 80 4.6 Security details of IPython................................... 83 4.7 IPython/Vision Beam Pattern Demo.............................
    [Show full text]
  • Python Guide Documentation Release 0.0.1
    Python Guide Documentation Release 0.0.1 Kenneth Reitz July 06, 2018 Contents 1 Getting Started with Python 3 1.1 Picking an Python Interpreter (3 vs. 2).................................3 1.2 Properly Installing Python........................................5 1.3 Installing Python 3 on Mac OS X....................................6 1.4 Installing Python 3 on Windows.....................................8 1.5 Installing Python 3 on Linux.......................................8 1.6 Installing Python 2 on Mac OS X.................................... 10 1.7 Installing Python 2 on Windows..................................... 12 1.8 Installing Python 2 on Linux....................................... 13 1.9 Pipenv & Virtual Environments..................................... 14 1.10 Lower level: virtualenv.......................................... 16 2 Python Development Environments 21 2.1 Your Development Environment..................................... 21 2.2 Further Configuration of Pip and Virtualenv............................... 26 3 Writing Great Python Code 29 3.1 Structuring Your Project......................................... 29 3.2 Code Style................................................ 40 3.3 Reading Great Code........................................... 49 3.4 Documentation.............................................. 50 3.5 Testing Your Code............................................ 53 3.6 Logging.................................................. 58 3.7 Common Gotchas............................................ 60 3.8 Choosing
    [Show full text]
  • Exposing Hidden Exploitable Behaviors in Programming Languages Using Differential Fuzzing
    WHITE PAPER Exposing Hidden Exploitable Behaviors in Programming Languages Using Differential Fuzzing Fernando Arnaboldi IOActive Senior Security Consultant Abstract Securely developed applications may have unidentified vulnerabilities in the underlying programming languages. Attackers can target these programming language flaws to alter applications' behavior. This means applications are only as secure as the programming languages parsing the code. A differential fuzzing framework was created to detect dangerous and unusual behaviors in similar software implementations. Multiple implementations of the top five interpreted programming languages were tested: JavaScript, Perl, PHP, Python, and Ruby. After fuzzing the default libraries and built-in functions, several dangerous behaviors were automatically identified. This paper reveals the most serious vulnerabilities found in each language. It includes practical examples identifying which undocumented functions could allow OS command execution, when sensitive file contents may be partially exposed in error messages, how native code is being unexpectedly interpreted – locally and remotely – and when constant's names could be used as regular strings for OS command execution. The vulnerabilities, methodology, and fuzzer will be made open source, and the accompanying talk will include live demonstrations. © 2017 IOActive, Inc. All Rights Reserved WHITE PAPER Contents Introduction ..................................................................................................................
    [Show full text]
  • Python Security Documentation Release 0.0
    Python Security Documentation Release 0.0 Victor Stinner Sep 15, 2021 Contents 1 Pages 3 1.1 Python Security Vulnerabilities.....................................3 1.2 Packages and PyPI............................................ 110 1.3 Python SSL and TLS security...................................... 122 1.4 Python Security............................................. 125 i ii Python Security Documentation, Release 0.0 This page is an attempt to document security vulnerabilities in Python and the versions including the fix. Contents 1 Python Security Documentation, Release 0.0 2 Contents CHAPTER 1 Pages 1.1 Python Security Vulnerabilities Status of Python branches lists Python branches which get security fixes. Total: 84 vulnerabilities. Vulnerability Disclosure Fixed In Vulnerable CVE CVE-2013-0340 Billion Laughs fixed in Expat 2.4.0 2021-06-11 3.6.15 3.7.12 3.8.12 3.9.7 – CVE-2013-0340 CVE-2021-3737: urllib HTTP client possible infinite loop on a 100 Continue response 2021-05-03 3.6.14 3.7.11 3.8.11 3.9.6 – – ipaddress leading zeros in IPv4 address 2021-03-30 3.8.12 3.9.5 – CVE-2021-29921 ftplib should not use the host from the PASV response 2021-02-21 3.6.14 3.7.11 3.8.9 3.9.3 – – CVE-2021-3733: ReDoS in urllib.request 2021-01-30 3.6.14 3.7.11 3.8.10 3.9.5 – – Information disclosure via pydoc getfile 2021-01-21 3.6.14 3.7.11 3.8.9 3.9.3 – CVE-2021-3426 urllib parse_qsl(): Web cache poisoning - semicolon as a query args separator 2021-01-19 3.6.13 3.7.10 3.8.8 3.9.2 – CVE-2021-23336 ctypes: Buffer overflow in PyCArg_repr 2021-01-16 3.6.13
    [Show full text]