The Python Distribution
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Hsafew2019 S2b.Pdf
How to work with SSM products From Download to Visualization Apostolos Giannakos Zentralanstalt für Meteorologie und Geodynamik (ZAMG) https://www.zamg.ac.at 1 Topics • Overview • ASCAT SSM NRT Products • ASCAT SSM CDR Products • Read and plot ASCAT SSM NRT Products • Read and plot ASCAT SSM CDR Products • Summary 2 H SAF ASCAT Surface Soil Moisture Products ASCAT SSM Near Real-Time (NRT) products – NRT products for ASCAT on-board Metop-A, Metop-B, Metop-C – Swath orbit geometry – Available 130 minutes after sensing – Various spatial resolutions • 25 km spatial sampling (50 km spatial resolution) • 12.5 km spatial sampling (25-34 km spatial resolution) • 0.5 km spatial sampling (1 km spatial resolution) ASCAT SSM Climate Data Record (CDR) products – ASCAT data merged for all Metop (A, B, C) satellites – Time series format located on an Earth fixed DGG (WARP5 Grid) – 12.5 km spatial sampling (25-34 km spatial resolution) – Re-processed every year (in January) – Extensions computed throughout the year until new release 3 Outlook: Near real-time surface soil moisture products CDOP3 H08 - SSM ASCAT NRT DIS Disaggregated Metop ASCAT NRT SSM at 1 km (pre-operational)* H101 - SSM ASCAT-A NRT O12.5 Metop-A ASCAT NRT SSM orbit 12.5 km sampling (operational) H102 - SSM ASCAT-A NRT O25 Metop-A ASCAT NRT SSM orbit 25 km sampling (operational) H16 - SSM ASCAT-B NT O12.5 Metop-B ASCAT NRT SSM orbit 12.5 km sampling (operational) H103 - SSM ASCAT-B NRT O25 Metop-B ASCAT NRT SSM orbit 25 km sampling (operational) 4 Architecture of ASCAT SSM Data Services -
Getting Started with Astropy
Astropy Documentation Release 0.2.5 The Astropy Developers October 28, 2013 CONTENTS i ii Astropy Documentation, Release 0.2.5 Astropy is a community-driven package intended to contain much of the core functionality and some common tools needed for performing astronomy and astrophysics with Python. CONTENTS 1 Astropy Documentation, Release 0.2.5 2 CONTENTS Part I User Documentation 3 Astropy Documentation, Release 0.2.5 Astropy at a glance 5 Astropy Documentation, Release 0.2.5 6 CHAPTER ONE OVERVIEW Here we describe a broad overview of the Astropy project and its parts. 1.1 Astropy Project Concept The “Astropy Project” is distinct from the astropy package. The Astropy Project is a process intended to facilitate communication and interoperability of python packages/codes in astronomy and astrophysics. The project thus en- compasses the astropy core package (which provides a common framework), all “affiliated packages” (described below in Affiliated Packages), and a general community aimed at bringing resources together and not duplicating efforts. 1.2 astropy Core Package The astropy package (alternatively known as the “core” package) contains various classes, utilities, and a packaging framework intended to provide commonly-used astronomy tools. It is divided into a variety of sub-packages, which are documented in the remainder of this documentation (see User Documentation for documentation of these components). The core also provides this documentation, and a variety of utilities that simplify starting other python astron- omy/astrophysics packages. As described in the following section, these simplify the process of creating affiliated packages. 1.3 Affiliated Packages The Astropy project includes the concept of “affiliated packages.” An affiliated package is an astronomy-related python package that is not part of the astropy core source code, but has requested to be included in the Astropy project. -
Conda Build Meta.Yaml
Porting legacy software packages to the Conda Package Manager Joe Asercion, Fermi Science Support Center, NASA/GSFC Previous Process FSSC Previous Process FSSC Release Tag Previous Process FSSC Release Tag Code Ingestion Previous Process FSSC Release Tag Code Ingestion Builds on supported systems Previous Process FSSC Release Tag Code Ingestion Push back Builds on changes supported systems Testing Previous Process FSSC Release Tag Code Ingestion Push back Builds on changes supported systems Testing Packaging Previous Process FSSC Release Release Tag Code Ingestion All binaries and Push back Builds on source made changes supported systems available on the FSSC’s website Testing Packaging Previous Process Issues • Very long development cycle • Difficult dependency • Many bottlenecks management • Increase in build instability • Frequent library collision errors with user machines • Duplication of effort • Large number of individual • Large download size binaries to support Goals of Process Overhaul • Continuous Integration/Release Model • Faster report/patch release cycle • Increased stability in the long term • Increased Automation • Increase process efficiency • Increased Process Transparency • Improved user experience • Better dependency management Conda Package Manager • Languages: Python, Ruby, R, C/C++, Lua, Scala, Java, JavaScript • Combinable with industry standard CI systems • Developed and maintained by Anaconda (a.k.a. Continuum Analytics) • Variety of channels hosting downloadable packages Packaging with Conda Conda Build Meta.yaml Build.sh/bld.bat • Contains metadata of the package to • Executed during the build stage be build • Written like standard build script • Contains metadata needed for build • Dependencies, system requirements, etc • Ideally minimalist • Allows for staged environment • Any customization not handled by specificity meta.yaml can be implemented here. -
The Astropy Project: Building an Open-Science Project and Status of the V2.0 Core Package*
The Astronomical Journal, 156:123 (19pp), 2018 September https://doi.org/10.3847/1538-3881/aabc4f © 2018. The American Astronomical Society. The Astropy Project: Building an Open-science Project and Status of the v2.0 Core Package* The Astropy Collaboration, A. M. Price-Whelan1 , B. M. Sipőcz44, H. M. Günther2,P.L.Lim3, S. M. Crawford4 , S. Conseil5, D. L. Shupe6, M. W. Craig7, N. Dencheva3, A. Ginsburg8 , J. T. VanderPlas9 , L. D. Bradley3 , D. Pérez-Suárez10, M. de Val-Borro11 (Primary Paper Contributors), T. L. Aldcroft12, K. L. Cruz13,14,15,16 , T. P. Robitaille17 , E. J. Tollerud3 (Astropy Coordination Committee), and C. Ardelean18, T. Babej19, Y. P. Bach20, M. Bachetti21 , A. V. Bakanov98, S. P. Bamford22, G. Barentsen23, P. Barmby18 , A. Baumbach24, K. L. Berry98, F. Biscani25, M. Boquien26, K. A. Bostroem27, L. G. Bouma1, G. B. Brammer3 , E. M. Bray98, H. Breytenbach4,28, H. Buddelmeijer29, D. J. Burke12, G. Calderone30, J. L. Cano Rodríguez98, M. Cara3, J. V. M. Cardoso23,31,32, S. Cheedella33, Y. Copin34,35, L. Corrales36,99, D. Crichton37,D.D’Avella3, C. Deil38, É. Depagne4, J. P. Dietrich39,40, A. Donath38, M. Droettboom3, N. Earl3, T. Erben41, S. Fabbro42, L. A. Ferreira43, T. Finethy98, R. T. Fox98, L. H. Garrison12 , S. L. J. Gibbons44, D. A. Goldstein45,46, R. Gommers47, J. P. Greco1, P. Greenfield3, A. M. Groener48, F. Grollier98, A. Hagen49,50, P. Hirst51, D. Homeier52 , A. J. Horton53, G. Hosseinzadeh54,55 ,L.Hu56, J. S. Hunkeler3, Ž. Ivezić57, A. Jain58, T. Jenness59 , G. Kanarek3, S. Kendrew60 , N. S. Kern45 , W. E. -
Data Analysis Tools for the HST Community
EXPANDING THE FRONTIERS OF SPACE ASTRONOMY Data Analysis Tools for the HST Community STUC 2019 Erik Tollerud Data Analysis Tools Branch Project Scientist What Are Data Analysis Tools? The Things After the Pipeline • DATs: Post-Pipeline Tools Spectral visualization tools Specutils Image visualization tools io models training & documentation modeling fitting - Analysis Tools astropy-helpers gwcs: generalized wcs wcs asdf: advanced data format JWST Tools tutorials ‣ Astropy software distribution astroquery MAST JWST data structures nddata ‣ Photutils models & fitting Astronomy Python Tool coordination photutils + others Development at STScI ‣ Specutils STAK: Jupyter notebooks - Visualization Tools IRAF Add to existing python lib cosmology External replacement Build replacement pkg ‣ Python Imexam (+ds9) constants astropy dev IRAF switchers guide affiliated packages no replacement ‣ Ginga ‣ SpecViz ‣ (MOSViz) ‣ (CubeViz) Data Analysis Software at STScI is for the Community Software that is meant to be used by the astronomy community to do their science. This talk is about some newer developments of interest to the HST user community. 2. 3. 1. IRAF • IRAF was amazing for its time, and the DAT for generations of astronomers What is the long-term replacement strategy? Python’s Scientific Ecosystem Has what Astro Needs Python is now the single most- used programming language in astronomy. Momcheva & Tollerud 2015 And 3rd-most in industry… Largely because of the vibrant scientific and numerical ecosystem (=Data Science): (Which HST’s pipelines helped start!) IRAF<->Python is not 1-to-1. Hence STAK Lead: Sara Ogaz Supporting both STScI’s internal scientists and the astronomy community at large (viewers like you) requires that the community be able to transition from IRAF to the Python-world. -
Installation Instructions
Installation instructions The following instructions should be quite detailed and easy to follow. If you nevertheless encounter a problem which you cannot solve for yourself, please write an email to Thomas Foesel. Note: the monospaced text in this section are commands which have to be executed in a terminal. • for Linux/Mac: The terminal is simply the system shell. The "#" at the start of the line indicates that root privileges are required (so log in as root via su, or use sudo if this is configured suitably), whereas the commands starting with "$" can be executed as a normal user. • for Windows: Type the commands into the Conda terminal which is part of the Miniconda installation (see below). Installing Python, Theano, Keras, Matplotlib and Jupyter In the following, we show how to install these packages on the three common operating systems. There might be alternative ways to do so; if you prefer another one that works for you, this is also fine, of course. • Linux ◦ Debian/Mint/Ubuntu/... 1. # apt-get install python3 python3-dev python3- matplotlib python3-nose python3-numpy python3-pip 2. # pip3 install jupyter keras Theano ◦ openSUSE 1. # zypper in python3 python3-devel python3- jupyter_notebook python3-matplotlib python3-nose python3-numpy-devel 2. # pip3 install Theano keras • Mac 2. Download the installation script for the Miniconda collection (make sure to select Python 3.x, the upper row). In the terminal, go into the directory of this file ($ cd ...) and run # bash Miniconda3-latest-MacOSX-x86_64.sh. 3. Because there are more recent Conda versions than on the website, update it via conda update conda. -
Python for Astronomers an Introduction to Scientific Computing
Python for Astronomers An Introduction to Scientific Computing Imad Pasha Christopher Agostino Copyright c 2014 Imad Pasha & Christopher Agostino DEPARTMENT OF ASTRONOMY,UNIVERSITY OF CALIFORNIA,BERKELEY Aknowledgements: We would like to aknowledge Pauline Arriage and Baylee Bordwell for their work in assembling much of the base topics covered in this class, and for creating many of the resources which influenced the production of this textbook. First Edition, January, 2015 Contents 1 Essential Unix Skills ..............................................7 1.1 What is UNIX, and why is it Important?7 1.2 The Interface7 1.3 Using a Terminal8 1.4 SSH 8 1.5 UNIX Commands9 1.5.1 Changing Directories.............................................. 10 1.5.2 Viewing Files and Directories........................................ 11 1.5.3 Making Directories................................................ 11 1.5.4 Deleting Files and Directories....................................... 11 1.5.5 Moving/Copying Files and Directories................................. 12 1.5.6 The Wildcard.................................................... 12 2 Basic Python .................................................. 15 2.1 Data-types 15 2.2 Basic Math 16 2.3 Variables 17 2.4 String Concatenation 18 2.5 Array, String, and List Indexing 19 2.5.1 Two Dimensional Slicing............................................ 20 2.6 Modifying Lists and Arrays 21 3 Libraries and Basic Script Writing ............................... 23 3.1 Importing Libraries 23 3.2 Writing Basic Programs 24 3.2.1 Writing Functions................................................. 25 3.3 Working with Arrays 26 3.3.1 Creating a Numpy Array........................................... 26 3.3.2 Basic Array Manipulation........................................... 26 4 Conditionals and Loops ........................................ 29 4.1 Conditional Statements 29 4.1.1 Combining Conditionals........................................... 30 4.2 Loops 31 4.2.1 While-Loops.................................................... -
Introduction to Anaconda and Python: Installation and Setup
¦ 2020 Vol. 16 no. 5 Python FOR Research IN Psychology Introduction TO Anaconda AND Python: Installation AND SETUP Damien Rolon-M´ERETTE A B , Matt Ross A , Thadd´E Rolon-M´ERETTE A & Kinsey Church A A University OF Ottawa AbstrACT ACTING Editor Python HAS BECOME ONE OF THE MOST POPULAR PROGRAMMING LANGUAGES FOR RESEARCH IN THE PAST decade. Its free, open-source NATURE AND VAST ONLINE COMMUNITY ARE SOME OF THE REASONS be- Nareg Berberian (University OF Ot- HIND ITS success. Countless EXAMPLES OF INCREASED RESEARCH PRODUCTIVITY DUE TO Python CAN BE FOUND tawa) ACROSS A PLETHORA OF DOMAINS online, INCLUDING DATA science, ARTIfiCIAL INTELLIGENCE AND SCIENTIfiC re- Reviewers search. This TUTORIAL’S GOAL IS TO HELP USERS GET STARTED WITH Python THROUGH THE INSTALLATION AND SETUP TARIQUE SirAGY (Uni- OF THE Anaconda software. The GOAL IS TO SET USERS ON THE PATH TOWARD USING THE Python LANGUAGE BY VERSITY OF Ottawa) PREPARING THEM TO WRITE THEIR fiRST script. This TUTORIAL IS DIVIDED IN THE FOLLOWING fashion: A SMALL INTRODUCTION TO Python, HOW TO DOWNLOAD THE Anaconda software, THE DIFFERENT CONTENT THAT COMES WITH THE installation, AND A SIMPLE EXAMPLE RELATED TO IMPLEMENTING A Python script. KEYWORDS TOOLS Python, Psychology, Installation guide, Anaconda. Python, Anaconda. B [email protected] 10.20982/tqmp.16.5.S003 Introduction MON problems, VIDEO tutorials, AND MUCH more. These re- SOURCES ARE OFTEN FREE TO ACCESS AND COVER THE MOST com- WhY LEARN Python? Python IS AN object-oriented, inter- MON PROBLEMS DEVELOPERS RUN INTO AT ALL LEVELS OF DIffiCULTY. preted, mid-level PROGRAMMING LANGUAGE THAT IS EASY TO In ADDITION TO THE BENEfiTIAL FEATURES MENTIONED above, LEARN AND USE WHILE BEING VERSATILE ENOUGH TO TACKLE A vari- Python HAS MANY DIFFERENT QUALITIES SPECIfiCALLY TAILORED ETY OF TASKS (Helmus & Collis, 2016). -
Tensorflow and Keras Installation Steps for Deep Learning Applications in Rstudio Ricardo Dalagnol1 and Fabien Wagner 1. Install
Version 1.0 – 03 July 2020 Tensorflow and Keras installation steps for Deep Learning applications in Rstudio Ricardo Dalagnol1 and Fabien Wagner 1. Install OSGEO (https://trac.osgeo.org/osgeo4w/) to have GDAL utilities and QGIS. Do not modify the installation path. 2. Install ImageMagick (https://imagemagick.org/index.php) for image visualization. 3. Install the latest Miniconda (https://docs.conda.io/en/latest/miniconda.html) 4. Install Visual Studio a. Install the 2019 or 2017 free version. The free version is called ‘Community’. b. During installation, select the box to include C++ development 5. Install the latest Nvidia driver for your GPU card from the official Nvidia site 6. Install CUDA NVIDIA a. I installed CUDA Toolkit 10.1 update2, to check if the installed or available NVIDIA driver is compatible with the CUDA version (check this in the cuDNN page https://docs.nvidia.com/deeplearning/sdk/cudnn-support- matrix/index.html) b. Downloaded from NVIDIA website, searched for CUDA NVIDIA download in google: https://developer.nvidia.com/cuda-10.1-download-archive-update2 (list of all archives here https://developer.nvidia.com/cuda-toolkit-archive) 7. Install cuDNN (deep neural network library) – I have installed cudnn-10.1- windows10-x64-v7.6.5.32 for CUDA 10.1, the last version https://docs.nvidia.com/deeplearning/sdk/cudnn-install/#install-windows a. To download you have to create/login an account with NVIDIA Developer b. Before installing, check the NVIDIA driver version if it is compatible c. To install on windows, follow the instructions at section 3.3 https://docs.nvidia.com/deeplearning/sdk/cudnn-install/#install-windows d. -
Conda-Build Documentation Release 3.21.5+15.G174ed200.Dirty
conda-build Documentation Release 3.21.5+15.g174ed200.dirty Anaconda, Inc. Sep 27, 2021 CONTENTS 1 Installing and updating conda-build3 2 Concepts 5 3 User guide 17 4 Resources 49 5 Release notes 115 Index 127 i ii conda-build Documentation, Release 3.21.5+15.g174ed200.dirty Conda-build contains commands and tools to use conda to build your own packages. It also provides helpful tools to constrain or pin versions in recipes. Building a conda package requires installing conda-build and creating a conda recipe. You then use the conda build command to build the conda package from the conda recipe. You can build conda packages from a variety of source code projects, most notably Python. For help packing a Python project, see the Setuptools documentation. OPTIONAL: If you are planning to upload your packages to Anaconda Cloud, you will need an Anaconda Cloud account and client. CONTENTS 1 conda-build Documentation, Release 3.21.5+15.g174ed200.dirty 2 CONTENTS CHAPTER ONE INSTALLING AND UPDATING CONDA-BUILD To enable building conda packages: • install conda • install conda-build • update conda and conda-build 1.1 Installing conda-build To install conda-build, in your terminal window or an Anaconda Prompt, run: conda install conda-build 1.2 Updating conda and conda-build Keep your versions of conda and conda-build up to date to take advantage of bug fixes and new features. To update conda and conda-build, in your terminal window or an Anaconda Prompt, run: conda update conda conda update conda-build For release notes, see the conda-build GitHub page. -
Sustainable Software Packaging for End Users with Conda
Sustainable software packaging for end users with conda Chris Burr, Henry Schreiner, Enrico Guiraud, Javier Cervantes CHEP 2019 ○ 5th November 2019 Image: CERN-EX-66954B © 1998-2018 CERN LCG from EP-SFT LCG from LHCb [email protected] ○ CHEP 2019 ○ Sustainable software packaging for end users with conda https://xkcd.com/1987/2 LCG from EP-SFT LCG from LHCb Homebrew ROOT builtin GCC 7.3 GCC 8.2 GCC9.2 Homebrew + Python 3.7 Self compiled against Python 3.6 (but broken since updating to 3.7 homebrew) Clang 5 Clang 6 Clang 8 Clang 9 [email protected] ○ CHEP 2019 ○ Sustainable software packaging for end users with conda https://xkcd.com/1987/3 LCG from EP-SFT LCG from LHCb Homebrew ROOT builtin GCC 7.3 GCC 8.2 GCC9.2 Homebrew +Python 3.7 Self compiled against Python 3.6 (broken since updating to 3.7 homebrew) Clang 5 Clang 6 Clang 8 Clang 9 [email protected] ○ CHEP 2019 ○ Sustainable software packaging for end users with conda https://xkcd.com/1987/4 LCG from EP-SFT LCG from LHCb Homebrew ROOT builtin GCC 7.3 GCC 8.2 GCC9.2 Homebrew +Python 3.7 Self compiled against Python 3.6 (broken since updating to 3.7 homebrew) Clang 5 Clang 6 Clang 8 Clang 9 [email protected] ○ CHEP 2019 ○ Sustainable software packaging for end users with conda https://xkcd.com/1987/5 LCG from EP-SFT LCG from LHCb Homebrew ROOT builtin GCC 7.3 conda create !-name my-analysis \ GCC 8.2 python=3.7 ipython pandas matplotlib \ GCC9.2 root boost \ Homebrew +Python 3.7 tensorflow xgboost \ Self compiled against Python 3.6 (broken since updating -
Investigating Interoperability of the LSST Data Management Software Stack with Astropy
Investigating interoperability of the LSST Data Management software stack with Astropy Tim Jennessa, James Boschb, Russell Owenc, John Parejkoc, Jonathan Sicka, John Swinbankb, Miguel de Val-Borrob, Gregory Dubois-Felsmannd, K-T Lime, Robert H. Luptonb, Pim Schellartb, K. Simon Krughoffc, and Erik J. Tollerudf aLSST Project Management Office, Tucson, AZ, U.S.A. bPrinceton University, Princeton, NJ, U.S.A. cUniversity of Washington, Seattle, WA, U.S.A dInfrared Processing and Analysis Center, California Institute of Technology, Pasadena, CA, U.S.A. eSLAC National Laboratory, Menlo Park, CA, U.S.A. fSpace Telescope Science Institute, 3700 San Martin Dr, Baltimore, MD, 21218, USA ABSTRACT The Large Synoptic Survey Telescope (LSST) will be an 8.4 m optical survey telescope sited in Chile and capable of imaging the entire sky twice a week. The data rate of approximately 15 TB per night and the requirements to both issue alerts on transient sources within 60 seconds of observing and create annual data releases means that automated data management systems and data processing pipelines are a key deliverable of the LSST construction project. The LSST data management software has been in development since 2004 and is based on a C++ core with a Python control layer. The software consists of nearly a quarter of a million lines of code covering the system from fundamental WCS and table libraries to pipeline environments and distributed process execution. The Astropy project began in 2011 as an attempt to bring together disparate open source Python projects and build a core standard infrastructure that can be used and built upon by the astronomy community.