Getting Stuff Done Software Synergy with Fiji / ImageJ2 and KNIME Max-Planck Institute, Dresden Florian Jug - [email protected] Outline A. What do we want to get done? (And who is we?) B. Software synergy with Fiji, IJ2, and KNIME C. Examples D. Remarks and opinions A What to get done… A Who wants stuff done? • You • Your boss • Your institute / university • Your scientific community A Viewpoint: single person A Viewpoint: single person • do awesome stuff of interest • do it right! • get it done • avoid unnecessary overhead A Research group level A Research group level • do lots of awesome stuff of interest • synergy between group members • keep knowledge alive! • make starting easy • documentation • teaching • open exchange of ideas A Research group level The Myers Lab A Research group level Genome Sequencing Development of a new assembler (The Dresden Azzembler „DAZZLER“). First assembler that fully leverages on the properties of the new BacBio reads. We run two new PacBio machines at CRTD/TUD. Two ongoing projects: • Axolotl (with Tanaka Lab) • Planaria (with Rink Lab) A Research group level Microscope Development So far we developed two light sheet microscopes (SPIM) at MPI-CBG. • The “Betzig Scope” is since month in part-time production mode for our “costumers” and creates stunning datasets. • Our second scope, the „X-Wing“ is almost completed and will be turned on soon. It was developed to resolve entire organisms at single cell resolution. A Research group level Image and Data Analysis Idea: intensive collaboration of biologists, physicists, mathema- ticians, and computer scientists. Three example projects: • Development of specialised segmentation methods. • Building an adaptive C. Elegans atlas for fully automatic cell annotation. • Lineage tracing (tracking) and analysis of developing organisms. A Institute level Mission statement of the MPI-CBG We maintain our fundamental scientific question: how do cells form tissues? More generally we are interested in how the morphology of cells and tissues emerge from the interactions between individual molecules and cells. Key questions we are interested in are the molecular principles underlying morphogenesis, such as regulation of size and shape. Importantly, we want our research programs to span multiple scales of magnitude. We will continue working on size and shape at any of the scales or organizational levels from molecular assemblies, to organelles, cells, tissues, organs and organisms. New RGLs are selected on the basis of their fit to the overall multi-scale research of the Institute, to maintain some proportion between the different scales covered. Ultimately, our goal is to take a multi-scale approach to link tissue level organizational principles to molecular activity. Animal model systems: We are prepared to consider any animal system that amplifies a particular piece of biology and offers particular opportunities and/or is easy to study. Technology: The institute will develop new technologies necessary to realize these goals. A Institute level • foster collaborations among groups • share knowledge and tools • provide facilities - to ‘facilitate’ productivity! A Institute level (Dresden) Software Synergy — Vision • A new lab member comes in, easily understands the framework, has lots of useful software components available, and starts rockin' on their research project right away. Image and Data Processing What is Software Synergy? If you benefit from work other did in the past and future projects benefit from your current work. How can it be enhanced? Social Aspect: “One has to know what was done in the past, where to find it, and how to use it.” Technological Aspects: “Ideally, modules should be interoperable (one can work on output of other one).” Status Quo Main technologies used at MPI-CBG: • Amira (C++) • Fiji / ImageJ (Java) And all this on Windows, • KNIME (Java) Mac, and • PLUK Linux. • ITK / VTK / OpenCV / Vigra + Don’t forget • Python the cluster!!! • Matlab Typical Image Processing Workflow Image Data Storage Visualization Processing Analysis Some HDD FIJI R Excel Fileserver CellProfiler Weka MatLab OMERO ImageJ2 MatLab Matplotlib … MatLab Python GnuPlot OpenCV Excel JFreeChart SciKit-Image Prism … … … A Image Processing Workflow Automation of Analysis Own Weka & Fileserver Algorithm & R Excel CellProfiler run somewhere on many images A Image Processing Workflow Later in time… ? B Software Synergy What’s FIJI? What’s ImageJ2? http://imagej.github.io/presentations/2015-09-03-imagej2-and-fiji/ What’s KNIME? What’s KNIME? www.knime.org What’s KNIME Image Processing? Example Workflow in KNIME KNIME incluDes e.g. Python, Matlab, R ImageJ Integration 32 ImageJ2 Integration ImageJ2 in KNIME • Automatic noDe generation • ADD own ImageJ2 Plugins via installer • Every ImageJ2 Plugin IS a KNIME noDe! 33 ImageJ2 Integration Ultimate Goal: full bi-directionality ImageJ2 Integration Ultimate Goal: full bi-directionality C Example projects C Example projects “Provide Existing” C Epithelia Cell Segment. C Epithelia Cell Segment. C Epithelia Cell Segment. C Epithelia Cell Segment. C Epithelia Cell Segment. C Epithelia Cell Segment. C Segmentation Pipeline Christian Dietz Univ. Konstanz Andreas Graumann Univ. Konstanz Tobias Pietzsch MPI-CBG, Dresden C Example projects “Avoid Reinventing” C The Mother Machine The Mother Machine MotherMachine data by Eric van Nimwegen’s Lab, Basel The Mother Machine Finding The Solution Optimal Joint Segmentation and Tracking of Escherichia Coli in the Mother Machine BAMBI@MICCAI 2014 Tracking by Assignment Facilitates Data Curation (Best Paper Award) IMIC@MICCAI 2014 MINIMIZE C V FG · FG SUCH THAT (t) hi t T, P (H(t)): a 1, (1) 8 2 8 2 P i i P az Γ }|(h(t)) { X2 i2Xr i t T, h H(t) : a a =0, (2) 8 2 8 2 i − i a Γ (h) a Γ (h) i2Xr i2Xl t T, h H(t) : H(t) a(t) + a H(t) . (3) 8 2 8 2 | |· ?i | | a H (h) 2X" Fully Automatic Result Fully Automatic Tracking Results Data Curation …someone has to fix remaining mistakes… Example Example Example The Interactive MotherMachine C Example projects “Give Back” C Clear Volume Loic Martin Ulrik Nicola Royer Weigert Günther Maghelli Open-source live visualization and processing for light sheet microscopy ClearVolume – Open-source live 3D visualization for light sheet microscopy. Loic A. Royer, Martin Weigert, Ulrik Günther, Nicola Maghelli, Florian Jug, Ivo F. Sbalzarini, Eugene W. Myers Nature Methods 12, 480–481 (2015) The SPIM/DSLM playground The SPIM/DSLM playground Live visualization | GPUs to the rescue Volumetric Data GPU OpenCL, CUDA Drosophila Brain (Tzumin Lee) Volume Rendering Isosurfaces e.g. ~ 50ms for 400MB Dataset on a modest card Fibonacci multi pass for big volumes naive successive Fibonacci Fibonacci multi pass for big volumes naive successive Fibonacci Fibonacci multi pass for big volumes Drosophila Brain (Tzumin Lee), 3 Channel, 2GB Open / flexible framework Framework / language bindings Open / flexible framework D Remarks and Opinions D Remarks and Opinions • BUT • This whole SW landscape can be confusing • We need to make it easy be part of it • informative web resources • teaching how to contribute • It should be in the interest of research groups, institutes, and communities to foster contribution! D Remarks and Opinions • How can I contribute? • Use stuff that exists + give credit. • Make your own contributions available. • And if you can: hire developers, get funding. Thanks! Myers Lab @ MPI-CBG (Dresden) Tobias Pietzsch Stephan Saalfeld Christian Dietz Jean-Yves Tinevez Michael Berthold Carsten Rother MPI-CBG, Dresden Janelia Univ. Konstanz Institute Pasteur Univ. Konstanz CVLD, TU Dresden.