Package Management with Anaconda TTS skills clinic
Rebecca Batorsky
March 2019
1 / 16 Python versions and Packages
You probably start using python with whatever version of python came installed on your laptop.
python --version
## Python 2.7.10
Next, you install a bunch of packages and start making beautiful plots. Eventually, a new package version comes along that requires an updated version of python in order to install:
pip install matplotlib==3.0.3
Collecting matplotlib==3.0.3 Could not find a version that satisfies the requirement matplotlib==3.0.3 (from version No matching distribution found for matplotlib==3.0.3
2 / 16 Python versions and Packages
You may try a number of options to upgrade your python installation...
Image credit: biostars
3 / 16 Conda Environments
Conda enviornments allow you to maintain package versions together
Image credit: https://medium.freecodecamp.org
4 / 16 Pip, Conda and Miniconda
Pip : package manager for python packages only Conda : package and environment manager for any language
Image credit: https://medium.freecodecamp.org 5 / 16 Choosing a Miniconda Installation
6 / 16 Image credit: https://medium.freecodecamp.org Installing Miniconda on your system https://docs.conda.io/en/latest/miniconda.html
Follow installation instructions for your OS
7 / 16 Root environments
Conda creates a root environment that contains a version Python and some basic packages
conda info
active environment : base active env location : /anaconda3 shell level : 1 user config file : /Users/rbator01/.condarc populated config files : conda version : 4.5.11 conda-build version : 3.17.1 python version : 3.6.6.final.0 base environment : /anaconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/osx-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/pro/osx-64 https://repo.anaconda.com/pkgs/pro/noarch package cache : /anaconda3/pkgs /Users/rbator01/.conda/pkgs envs directories : /anaconda3/envs /Users/rbator01/.conda/envs platform : osx-64 8 / 16 Create and Activate an enviornment
conda create --name testenv python=3.4
Activate your new environment:
source activate testenv
Leave your environment:
source deactivate testenv
You can see your environment
conda env list
# conda environments: # base * /Users/rbator01/miniconda3 /anaconda3 /anaconda3/envs/testenv
9 / 16 Installing packages
source activate testenv conda install pandas
View installed packages
conda list -n testenv python=3.4
# packages in environment at /anaconda3/envs/testenv: # # Name Version Build Channel blas 1.0 mkl ca-certificates 2019.1.23 0 intel-openmp 2019.1 144 mkl 2017.0.4 h1fae6ae_0 numpy 1.11.3 py34_0 openssl 1.0.2r h1de35cc_0 pandas 0.19.2 np111py34_1 pip 9.0.1 py34_1 python 3.4.5 0 python-dateutil 2.6.1 py34_0 pytz 2017.2 py34_0 readline 6.2 2 setuptools 27.2.0 py34_0 six 1.10.0 py34_0 sqlite 3.13.0 0 10 / 16 Channels
Image credit: https://medium.freecodecamp.org 11 / 16 Finding packages
If you're not sure if your package is available from conda, just google it! E.g. "conda IGV"
12 / 16 Adding channel
You could either install it by specifying the channel
conda install -c bioconda igv
Or by adding the channel to the default list of channels to search:
conda config --add channels bioconda cat ~/.condarc
channels: - bioconda - defaults
You can now install packages from bioconda without specifying:
conda install igv
13 / 16 Using conda on the Tufts HPC
... is easy! Anaconda is isntalled system wide, and loadable as a module
module load anaconda/3 conda create -n testenv python=3.4
Note that this will both store package downloads and environments in your home directory This is Not Good on an HPC, where home directories are small.
You can change this by specifying the path of the environment in a space where you have more storage
conda create -p /cluster/tufts/bio/tools/conda_envs/testenv python=3.4
And by explicitly setting the package cache location in your ~/.condarc
channels: - conda-forge - bioconda - defaults envs_dirs: - /cluster/tufts/bio/tools/conda_envs/ pkgs_dirs: - /cluster/tufts/bio/tools/conda_envs/pkgs/ 14 / 16 Sharing environments with others
source activate testenv conda env export > testenv.yml
In order to create it on a new system:
conda env create -f environment.yml
The YAML file will look like this:
name: testenv channels: - bioconda - defaults dependencies: - igv=2.4.17=0 - blas=1.0=mkl - ca-certificates=2019.1.23=0 - intel-openmp=2019.1=144 - mkl=2017.0.4=h1fae6ae_0 - numpy=1.11.3=py34_0 - openjdk=8.0.152=h393ad39_1 - openssl=1.0.2r=h1de35cc_0 - pandas=0.19.2=np111py34_1 - pip=9.0.1=py34_1 - python=3.4.5=0 15 / 16 Removing an enviornment
conda remove -n testenv --all
For more information see this blog post: https://medium.freecodecamp.org/whyyouneedpythonenvironmentsandhowtomanage themwithconda85f155f4353c
Documentation on Anaconda: https://conda.readthedocs.io/en/latest/ Thanks!
16 / 16