Free and Open Source Scientific Software

Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Free and Open Source Scientiﬁc Software

Horea Christian

[project21] TheAlternative | ETHZ and UZH

October 20, 2015

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Got everything?

I User name: userNN

I Password: userNN

I Shell Client (Windows): GitBASH I Reliable Text Editor (Windows):

I Notepad++ (https://notepad-plus-plus.org/) I Atom (https://atom.io/)

I Social Coding Account: GitHub

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS and Science FOSS

I Free: as in freedom, creativity, collaboration

I Open Source: as in transparent, reproducible

I Software: as in powerful

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS and Science Scientiﬁc Software

= software used for research, experiments.

I Is mostly FOSS

I Oft relies on non-FOSS compilers/environments (e.g. MATLAB, LabVIEW)

I Evolves “organically”

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS and Science Issues with Scientiﬁc Software

I Only indicentally FOSS

I Seldom refactored

I Often bundled

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS Management and Package Examples NeuroDebian

I http://neuro.debian.net/

I Based on the Debian package manager, “dpkg”

I Mature, independent project

I Fast binary distribution

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS Management and Package Examples NeuroGentoo

I https://github.com/TheChymera/neurogentoo

I Based on the Gentoo package manager, “Portage”

I Part of “Gentoo Science”

I Flexible, reproducible source-based distribution

I Seamless live software integration

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS Management and Package Examples Major Neuroscience Players

Great software, not without issues:

I SPM: (still) reliant on MATLAB backend

I AFNI: lacks a build system

I FSL: restrictive licensing

I FreeSurfer: very outdated dependencies

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS Management and Package Examples New Neuroscience Players

Maturing software, in a Python environment.

I NiPype: pipeline interface for SPM, AFNI, FSL, etc.

I NiPy: native Python neuroimaging

I NiBabel: read/write for common neuroimaging formats

I Nilearn: machine learning and plotting

I etc.

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS Management and Package Examples Collaboration Platform

"Git-logo" by Jason Long - CC BY 3.0 via Commons

I Versioning system

I Decentralized fork and merge

I Wide choice of social coding platforms

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

FOSS Management and Package Examples Collaboration Backend

I Widespread use

I Newbie-friendly GUI

I Integrated issue tracking, documentation, and archiving

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Working Remotely on Linux: easy Got everything?

I User name: userNN

I Password: userNN

I Shell Client (Windows): GitBASH I Reliable Text Editor (Windows):

I Notepad++ (https://notepad-plus-plus.org/) I Atom (https://atom.io/)

I Social Coding Account: GitHub Log in via: ssh [email protected]

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Write your own program in Python! Prepare Directory structure

Create directory for your programs: mkdir ~/src

Create directory for this program: mkdir ~/src/actibrainNN

Create directory for the Python module: mkdir ~/src/actibrainNN/plottingNN

Mark directory as python module: touch ~/src/actibrainNN/plottingNN/__init__.py

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Write your own program in Python! Write Program

Choose one modality:

I Remotely: nano ~/src/actibrainNN/plottingNN/brain_activation.py

I Local (editor of your choice): create and open ﬁle named brain_activation.py

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Write your own program in Python! Example Program

Imports and activation by ID: from nilearn import datasets import matplotlib.pyplot as plt from nilearn import plotting

def plot_activation_by_ID(identifier): localizer_dataset= datasets.fetch_localizer_contrasts( [identifier], n_subjects=2, get_tmaps=True) localizer_tmap_filename= localizer_dataset.tmaps[1]

plotting.plot_glass_brain(localizer_tmap_filename, threshold=3)

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Write your own program in Python! Example Program - Continuation

Activation from data ﬁle: def plot_activation_by_data(localizer_data): plotting.plot_glass_brain(localizer_data, threshold=3)

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Write your own program in Python! Finalize Program

I Remotely: press ctrl-o and then ctrl-x

I Local (editor of your choice): scp /your/local/brain_activation.py\ [email protected]:src/actibrainNN/plottingNN

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Write your own program in Python! Add setup.py

Open ﬁle: nano ~/src/actibrainNN/setup.py

Write: from distutils.core import setup

setup(name=’actibrainNN’, version=’’, packages=[’plottingNN’] )

Press: ctrl-o and ctrl-x

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Collaborative coding Identify Yourself

Tell Git your (nick)name: git config --global user.name "Your Name"

Tell Git the email you used on GitHub: git config --global user.email "[email protected]"

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Collaborative coding Commit your Program to Git

Navigate to package directory: cd ~/src/actibrainNN/

Initialize empty git repository: git init

Add all ﬁles to git: git add .

Commit ﬁles to git: git commit -a -m "new program"

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Collaborative coding Commit your Program to Git

GitHub: Create empty repository named actibrainNN

Navigate to package directory: git remote add origin https://github.com/username/actibrainNN.git

Push to GitHub: git push origin master

GitHub: Check out your own open source program!

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Package your live software for Gentoo Get the Overlay Source

GitHub: Fork Chymeric Overlay https://github.com/TheChymera/overlay

Navigate to sources directory: cd ~/src

Clone your GitHub repository: git clone https://github.com/username/overlay.git

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Package your live software for Gentoo Create new Ebuild

GitHub: Fork Chymeric Overlay https://github.com/TheChymera/overlay

Create appropriate directory: mkdir ~/src/overlay/dev-python/actibrainNN

Navigate to appropriate directory: cd ~/src/overlay/dev-python/actibrainNN

Edit appropriate ﬁle: nano actibrainNN-9999.ebuild

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Package your live software for Gentoo Create new Ebuild - Continued

Write: EAPI=5

PYTHON_COMPAT=( python{2_7,3_4})

inherit distutils-r1 git-r3

DESCRIPTION="A tool to plot brain activation" EGIT_REPO_URI="git://github.com/username/actibrainNN"

LICENSE="GPL-3" SLOT="0" KEYWORDS="" IUSE=""

DEPEND="dev-python/setuptools" RDEPEND="dev-python/nilearn"

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Package your live software for Gentoo Commit to Git and upload to GitHub

Add ﬁle to git: git add .

Commit ﬁle to git: git commit -a -m "new ebuild"

Push to GitHub: git push origin master

GitHub: File pull request!

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Package your live software for Gentoo Sync and install

Just one person, please: sudo emerge --sync

Install Package: sudo emerge actibrainNN

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Live Python Figures in your Documents Conﬁgure Example Document

Navigate to sources directory: cd ~/src

Clone from BitBucket: git clone https://[email protected]/TheChymera/ld_expert-doc.git

Navigate to document root: cd ld_expert-doc

Replace plottingNN with your favourite function: nano pythontex/functions.tex

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Live Python Figures in your Documents Compile Exmple Document

Run pdﬂatex: pdflatex doc.tex

Generate ﬁgures: pythontex.py doc.tex

Place ﬁgures on page: pdflatex doc.tex

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Live Python Figures in your Documents Download and View your Document

On your machine: Open a new Shell.

Download via scp: scp [email protected]:src/ld_expert-doc/doc.pdf

On your machine: Open document

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Live Python Figures in your Documents Play around

Replace data: mv data_new.nii.gz data.nii.gz

Change contrast ID: nano pythontex/pycode.tex

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software Intro FOSSS SSH Python GitHub Portage PythonTeX Meta

Live Python Figures in your Documents These Slides

I Latest Slides: http://chymera.eu/pres/ld_expert/pres.pdf

I Source: https://bitbucket.org/TheChymera/ld_expert/src

I License: CC BY-SA 3.0

I Contact: [email protected]

Horea Christian [project21] TheAlternative | ETHZ and UZH Free and Open Source Scientiﬁc Software