Image Processing with KNIME Who we are?!
Martin Horn Christian Dietz [email protected] [email protected] (+49) 07531 88-5017 (+49) 07531 88-3641 Z815 Z815
Active Segmentation Active Classification
1. Big Picture – Scientific Image Processing
2. KNIME Image Processing (KNIP) – The Plugin
KNIME Developer Training 3 The “Zoo” of Image Processing Tools
Development Processing UI Handling
ImgLib ImageJ OMERO OpenCV KNIP BioFormats MatLab Fiji … NumPy CellProfiler VTK Ilastik VIGRA CellCognition … Icy Photoshop … = Single, individual, case specific, incompatible solutions
KNIME Developer Training 4 The “Zoo” of Image Processing Tools • Which combination should I use? • Reusability? • Interoperability?
Many wheels out there…
KNIME Developer Training 5 The “Zoo” of Image Processing Tools
Development Processing UI Handling
ImgLib ImageJ2/Fiji2 OMERO OpenCV KNIP BioFormats MatLab CellProfiler … NumPy Icy VTK Ilastik VIGRA CellCognition … Photoshop …
= Standardized Java Framework for Scientific Image Processing
KNIME Developer Training 6 SciJava – Overview
Fiji/ImageJ KNIP (Konstanz) (Madison) OMERO (Dundee) BioFormats ImgLib (MPI Dresden) (Madison)
KNIME Developer Training 7 SciJava
ImgLib2 • Developed at MPI-CBG Dresden • Generic Framework for data (image) processing algorithms and data-structures
KNIME Developer Training 8 SciJava – ImageJ
ImageJ • Wayne Rasband (National Institutes of Health) • Popular, highly interactive image processing program • Many plugins available
KNIME Developer Training 9 SciJava – ImageJ/FIJI
FIJI • Extension of ImageJ1 with plugin-update mechanism
ImageJ2 • New Version of ImageJ based on ImgLib2 • IJ1 plugins still work
KNIME Developer Training 10 SciJava - OME
Open Microscopy Environment • Joint project Dundee, Baltimore, Harvard Medical School and LOCI • Open tools to support data management for biological light microscopy • Standardized file-format (OME-XML)
KNIME Developer Training 11 SciJava - OME
OMERO • Tools for storing (database), visualizing, managing and annotating images and metadata
BioFormats • Library for reading and writing > 120 microscopy file formats
KNIME Developer Training 12 SciJava - KNIP
Basic information • KNIME Image Processing • Basic data structures: ImgLib2 • High-throughput screening • Fast prototyping • Understandable workflows
KNIME Developer Training 13 KNIME Image Processing (KNIP)
Who? • Martin Horn (University of Konstanz) • Christian Dietz (University of Konstanz) • Thorsten Rieß (University of Konstanz) • Slawek Mazur (BioQuant Heidelberg) • …
KNIME User Meeting 14 KNIME Image Processing (KNIP)
Who? … Students • Felix Schoenenberger (University of Konstanz) • Clemens Muething (University of Konstanz) • Jan-Dirk Verbeek (University of Konstanz) • Jens Metzner (University of Konstanz) • Maximilian Ortwein (University of Konstanz) • … several others
KNIME User Meeting 15 Why Image Processing with KNIP?
• Analysis of huge image data sets (HCS) • Further analysis of the data Machine learning Visualization Statistics … everything which comes with KNIME
KNIME User Meeting 16 Image IO in KNIP?
Image Reader using Bio-Formats • 122 supported formats
OMERO Image Reader (experimental) • Images from the OMERO image data base
Image Writer using Bio-Formats • 11 supported formats (e.g. tif, jpeg, png, ome- tiff,…) KNIME User Meeting 17
KNIP - How images are handled?
KNIME User Meeting 18 KNIME User Meeting 19 KNIP - How images are handled?
KNIME User Meeting 20 KNIP - How images are handled?
KNIME User Meeting 21 What’s an image in KNIP?
KNIME User Meeting 22 What’s an image in KNIP?
KNIME User Meeting 23 What’s an image in KNIP?
…
KNIME User Meeting 24 What’s an image in KNIP?
…
time
KNIME User Meeting 25 KNIP - How to inspect images?
KNIME User Meeting 26 KNIP How to inspect images?
KNIME User Meeting 27 KNIP How to inspect images?
KNIME User Meeting 28 KNIP - How to process images?
KNIME User Meeting 29 KNIP - How to process images?
KNIME User Meeting 30 KNIP - How to process images?
KNIME User Meeting 31 KNIP - How to process images?
KNIME User Meeting 32 KNIP - How to process images?
KNIME User Meeting 33 KNIP - How to process images?
KNIME User Meeting 34 KNIP - How to segment images?
KNIME User Meeting 35 KNIME User Meeting 36 How to represent region of interests?
KNIME User Meeting 37 KNIP - Feature calculation
KNIME User Meeting 38 KNIME User Meeting 39 KNIP - Feature calculation
First order statistics • Mean, Standard Deviation, Skewness, …
Texture features • Haralick, Tamura, …
Geometric features • Circularity, Convexity, Centroid, Size, …
KNIME User Meeting 40 KNIP - Feature calculation
Shape Features • Fourier Descriptors, Radial Distances, …
Many under development • Bag of gradients, Histogram of Gradients, Zernike, …
KNIME User Meeting 41 KNIP – Feature calculation
KNIME User Meeting 42 KNIP Feature calculation
KNIME User Meeting 43 KNIP - Feature calculation
KNIME User Meeting 44 KNIME User Meeting 45 KNIP – Segment Overlay
KNIME User Meeting 46 KNIP
KNIME User Meeting 47 KNIP – So what?
Some projects solved with KNIP • Mitotic Index • Chromosome Counting • Cell-Lifecycle Classification • Membrane Breakdown (ETH - Zürich) • Single Molecule Tracking • DNA-Repair Measurement • …. many more
KNIME User Meeting 48 KNIP – Chromosome Counting
KNIME User Meeting 49 Chromosome Counting
KNIME User Meeting 50 Chromosome Counting
KNIME User Meeting 51 Chromosome Counting
KNIME User Meeting 52 Chromosome Counting
KNIME User Meeting 53 Chromosome Counting
KNIME User Meeting 54 Chromosome Counting
KNIME User Meeting 55 Chromosome Counting
KNIME User Meeting 56 Chromosome Counting
KNIME User Meeting 57 Chromosome Counting
KNIME User Meeting 58 Chromosome Counting
KNIME User Meeting 59 Chromosome Counting
KNIME User Meeting 60 Chromosome Counting
KNIME User Meeting 61 Chromosome Counting
KNIME User Meeting 62 KNIME User Meeting 63 KNIP – Example High-Content Screening
positive negative
KNIME User Meeting 64 High-Content Screening
KNIME User Meeting 65 High-Content Screening
KNIME User Meeting 66 High-Content Screening
KNIME User Meeting 67 High-Content Screening
KNIME User Meeting 68 High-Content Screening
KNIME User Meeting 69 KNIME User Meeting 70 Some more examples …
KNIME User Meeting 71 Mitosis Classification
KNIME User Meeting 72 Mitosis Classification
KNIME User Meeting 73 Mitotic Index
KNIME User Meeting 74 Mitotic Index
KNIME User Meeting 75 Mitotic Index
KNIME User Meeting 76 KNIP – Nice to know
Currently ~52 official nodes • IO nodes: ~ 4 • Image processing nodes ~ 25 • Filter nodes ~ 6 • Segmentation-related nodes ~ 10 • Feature calculation nodes ~ 3 • Viewer nodes ~ 4
KNIME User Meeting 77 KNIP – Nice to know
Sub-projects about to be released • VTK based 3D-Viewer
Sub-projects under development • Tracking framework • IJ2 integration • Pixel-Classification • OMERO integration
KNIME User Meeting 78 KNIP – What’s next?
Further OMERO integration • Visit from OME-Team members early March
Data handling • Efficient data management
KNIME User Meeting 79 KNIP – What’s next?
ImageJ2 integration • High priority! Waiting for some changes in IJ2
Tracking framework • Builds up-on the network analysis framework • Work in progress, some first results
KNIME User Meeting 80 KNIP – What’s next?
ImgLib2 synchronization • Synchronizing code for de/serialization • Integrating KNIPLib algorithms to ImgLib2 • Refactoring KNIPLib algorithms
KNIME User Meeting 81 KNIP – Community Contributions
• Update site • News, mailing list, SVN-Access • Use the FORUM!
http://tech.knime.org/community/image-processing
KNIME User Meeting 82 KNIP – How to take part?
• We need feedback! • We are open for any suggestions, help, feature requests, bug reports…
KNIME User Meeting 83
Thank you…
KNIME User Meeting 84