Getting Stuff Done

Software Synergy with / ImageJ2 and KNIME

Max-Planck Institute, Dresden Florian Jug - [email protected] Outline

A. What do we want to get done? (And who is we?)

B. Software synergy with Fiji, IJ2, and KNIME

C. Examples

D. Remarks and opinions A What to get done… A Who wants stuff done?

• You

• Your boss

• Your institute / university

• Your scientific community A Viewpoint: single person A Viewpoint: single person

• do awesome stuff of interest • do it right! • get it done • avoid unnecessary overhead A Research group level A Research group level

• do lots of awesome stuff of interest • synergy between group members • keep knowledge alive! • make starting easy • documentation • teaching • open exchange of ideas A Research group level

The Myers Lab A Research group level

Genome Sequencing

Development of a new assembler (The Dresden Azzembler „DAZZLER“). First assembler that fully leverages on the properties of the new BacBio reads.

We run two new PacBio machines at CRTD/TUD.

Two ongoing projects: • Axolotl (with Tanaka Lab) • Planaria (with Rink Lab) A Research group level

Microscope Development

So far we developed two light sheet microscopes (SPIM) at MPI-CBG.

• The “Betzig Scope” is since month in part-time production mode for our “costumers” and creates stunning datasets.

• Our second scope, the „X-Wing“ is almost completed and will be turned on soon. It was developed to resolve entire organisms at single cell resolution. A Research group level

Image and

Idea: intensive collaboration of biologists, physicists, mathema- ticians, and computer scientists.

Three example projects: • Development of specialised segmentation methods. • Building an adaptive C. Elegans atlas for fully automatic cell annotation. • Lineage tracing (tracking) and analysis of developing organisms. A Institute level

Mission statement of the MPI-CBG

We maintain our fundamental scientific question: how do cells form tissues? More generally we are interested in how the morphology of cells and tissues emerge from the interactions between individual molecules and cells. Key questions we are interested in are the molecular principles underlying morphogenesis, such as regulation of size and shape. Importantly, we want our research programs to span multiple scales of magnitude. We will continue working on size and shape at any of the scales or organizational levels from molecular assemblies, to organelles, cells, tissues, organs and organisms. New RGLs are selected on the basis of their fit to the overall multi-scale research of the Institute, to maintain some proportion between the different scales covered. Ultimately, our goal is to take a multi-scale approach to link tissue level organizational principles to molecular activity. Animal model systems: We are prepared to consider any animal system that amplifies a particular piece of biology and offers particular opportunities and/or is easy to study. Technology: The institute will develop new technologies necessary to realize these goals. A Institute level

• foster collaborations among groups • share knowledge and tools • provide facilities - to ‘facilitate’ productivity! A Institute level

(Dresden) Software Synergy — Vision

• A new lab member comes in, easily understands the framework, has lots of useful software components available, and starts rockin' on their research project right away. Image and Data Processing What is Software Synergy?

If you benefit from work other did in the past and future projects benefit from your current work. How can it be enhanced?

Social Aspect: “One has to know what was done in the past, where to find it, and how to use it.”

Technological Aspects: “Ideally, modules should be interoperable (one can work on output of other one).” Status Quo

Main technologies used at MPI-CBG:

• (C++) • Fiji / ImageJ () And all this on Windows, • KNIME (Java) Mac, and • PLUK . • ITK / VTK / OpenCV / Vigra + Don’t forget • Python the cluster!!! • Matlab Typical Image Processing Workflow

Image Data Storage Visualization Processing Analysis

Some HDD FIJI Excel Fileserver CellProfiler MatLab OMERO ImageJ2 MatLab Matplotlib … MatLab Python GnuPlot OpenCV Excel JFreeChart SciKit-Image Prism … … … A Image Processing Workflow

Automation of Analysis

Own Weka & Fileserver Algorithm & R Excel CellProfiler

run somewhere on many images A Image Processing Workflow

Later in time… ? B Software Synergy What’s FIJI? What’s ImageJ2? http://imagej.github.io/presentations/2015-09-03-imagej2-and-fiji/ What’s KNIME? What’s KNIME? www..org What’s KNIME Image Processing? Example Workflow in KNIME KNIME includes e.g. Python, Matlab, R ImageJ Integration

32 ImageJ2 Integration

ImageJ2 in KNIME • Automatic node generation • Add own ImageJ2 Plugins via installer • Every ImageJ2 Plugin IS a KNIME node!

33 ImageJ2 Integration

Ultimate Goal: full bi-directionality ImageJ2 Integration

Ultimate Goal: full bi-directionality C Example projects C Example projects

“Provide Existing” C Epithelia Cell Segment. C Epithelia Cell Segment. C Epithelia Cell Segment. C Epithelia Cell Segment. C Epithelia Cell Segment. C Epithelia Cell Segment. C Segmentation Pipeline

Christian Dietz Univ. Konstanz

Andreas Graumann Univ. Konstanz

Tobias Pietzsch MPI-CBG, Dresden C Example projects

“Avoid Reinventing” C The Mother Machine The Mother Machine

MotherMachine data by Eric van Nimwegen’s Lab, Basel The Mother Machine Finding The Solution

Optimal Joint Segmentation and Tracking of Escherichia Coli in the Mother Machine BAMBI@MICCAI 2014

Tracking by Assignment Facilitates Data Curation (Best Paper Award) IMIC@MICCAI 2014

MINIMIZE C V FG · FG SUCH THAT (t) hi

t T, P (H(t)): a 1, (1) 8 2 8 2 P i  i P az }|(h(t)) { X2 i2Xr i t T, h H(t) : a a =0, (2) 8 2 8 2 i i a (h) a (h) i2Xr i2Xl t T, h H(t) : H(t) a(t) + a H(t) . (3) 8 2 8 2 | |· ?i  | | a H (h) 2X" Fully Automatic Result Fully Automatic Tracking Results Data Curation …someone has to fix remaining mistakes… Example Example Example

The Interactive MotherMachine C Example projects

“Give Back” C Clear Volume Loic Martin Ulrik Nicola Royer Weigert Günther Maghelli

Open-source live visualization and processing for light sheet microscopy

ClearVolume – Open-source live 3D visualization for light sheet microscopy. Loic A. Royer, Martin Weigert, Ulrik Günther, Nicola Maghelli, Florian Jug, Ivo F. Sbalzarini, Eugene W. Myers Nature Methods 12, 480–481 (2015) The SPIM/DSLM playground The SPIM/DSLM playground Live visualization | GPUs to the rescue

Volumetric Data

GPU OpenCL, CUDA

Drosophila Brain (Tzumin Lee) Volume Rendering Isosurfaces e.g. ~ 50ms for 400MB Dataset on a modest card Fibonacci multi pass for big volumes

naive successive

Fibonacci Fibonacci multi pass for big volumes

naive successive Fibonacci Fibonacci multi pass for big volumes

Drosophila Brain (Tzumin Lee), 3 Channel, 2GB Open / flexible framework

Framework / language bindings Open / flexible framework D Remarks and Opinions D Remarks and Opinions

• BUT • This whole SW landscape can be confusing • We need to make it easy be part of it • informative web resources • teaching how to contribute • It should be in the interest of research groups, institutes, and communities to foster contribution! D Remarks and Opinions

• How can I contribute? • Use stuff that exists + give credit. • Make your own contributions available. • And if you can: hire developers, get funding. Thanks!

Myers Lab @ MPI-CBG (Dresden)

Tobias Pietzsch Stephan Saalfeld Christian Dietz Jean-Yves Tinevez Michael Berthold Carsten Rother MPI-CBG, Dresden Janelia Univ. Konstanz Institute Pasteur Univ. Konstanz CVLD, TU Dresden