Biological Imaging Software Tools Pu to K.W.E

Home , 3D Slicer, Bioimage informatics, CellProfiler

© 2012 Nature America, Inc. All rights reserved. FOCUS ONBIOIMAGE INFORMATICS field is still a fairly nascent community compared compared community nascent fairly a still is field loop. iterative tight a in techniques and imaging of experimental their aspects to refine flexibility the with manner in a reproducible data imaging of amounts vast visualize and interpret analyze, manipulate, store, collect, to tools with ture infrastruc informatics an demands which richness, and complexity increasing of data digital processing of challenge the with confronted often are biologists for breakthroughs, quest In the data. the interpreting properly and visualizing for also cases many in but not just for the capturedigital that all systems now use optics, the as important as just is technology tational tational approaches. In fact, in many cases the compu on rely compu heavily are multiparametric, increasingly which modalities, imaging These relevance. cal and biologi viability complexity maintaining while all and scale, dimensionality, specificity, resolution, unprecedented with to phenomena ability biological the monitor of introduction the with opti imaging in cal advances great seen have years 20 last The software for bioimage informatics, focusing on open-source options. dealing with digital images, the inherent challenges and the overall status of available imaging data. We review each computational step that biologists encounter when of software to enable the acquisition, management, analysis and visualization of the unimagined capabilities. Almost all these advances have required the development Recent advances in optical technologies and instrumentation are providing hitherto Few technologies are more widespread in modern biological laboratories than imaging. Nico Stuurman Maryann E Martone Kevin W Eliceiri Biological imaging software tools pu to K.W.E. or ([email protected]) A.E.C. ([email protected]). Institute of Harvard and the Massachusetts Institute of Technology, Cambridge, Massachusetts, USA. Correspondence should be addressed of Dundee, Dundee, UK. of San California, Francisco, San Francisco, USA. California, and Computer UniversityEngineering, of Houston, Houston, Texas, USA. 9 University, Pittsburgh, Pennsylvania, USA. Imaging Research, University of SanCalifornia Diego, La Jolla, USA. California, Bio-image Informatics, University of SantaCalifornia Barbara, Santa Barbara, USA. California, Baltimore, Maryland, USA. of Computer and Information UniversitätScience, Konstanz, Konstanz, Germany. 1 Biochemical Science Division, Science Biochemical National Institute of Standards and Technology, Gaithersburg, Maryland, USA. Laboratory for andOptical Computational Instrumentation, University of Wisconsin at Madison, Madison, Wisconsin, USA. b Despite this great need, the bioimage informatics informatics bioimage the need, great this Despite lished online 28 june 2012; C 11 1 13 , , Jason R Swedlow , , Michael R Berthold Max Planck Institute of Molecular andBiology Cell Dresden, Genetics, Germany. 4 Kitware Inc., New York, New York, USA. 6 , Robert F Murphy O RR E CT

ED 8 Janelia Farm Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, USA. AFT E R P 12 R IN , , Pavel Tomancak T 20 2 7 , , Ilya G Goldberg , Hanchuan Peng 12 JUL Wellcome Trust Centre for Gene Regulation and Expression, University Y 2012; - - - - - 5 Department of Electrical andDepartment of ComputerElectrical Center Engineering, for by measuring photon flux in parallel (using a (using photon by in camera) flux parallel measuring acquired are usually images laboratories In biological ACQUISITION IMAGE analysis data mining. and to storage and acquisition data solution from laboratory end-to-end an provide be to can used software imaging open-source how of view over an provide to is goal Our options. open-source on focusing informatics, bioimage for software able avail of status overall the and domain that in lenges ( images digital with encounter dealing when biologists that step computational each to review collaborated community informatics image informatics bioimage of number the in increase great a been has there ics, informat bioimage prioritize to agencies funding by imaging approaches by biologists and the commitment optical- advanced of adoption mainstream increased the with But community. microscopy optical the of side hardware-development established more the to c 11 o For this Review, of members representative the bio rr Department Department of Cellular and Molecular Pharmacology, University e ct 7 Lane Lane Center for Computational Biology, MellonCarnegie nature 13 ed 3 National Institute on Aging, National Institutes of Health, & Anne E Carpenter 8 3 aft , Anne L Plant , , Luis Ibáñez e r

p met r in 6 h National Center for Microscopy and t o 29 d 1 s a tools over the last 5 years. 5 last the over tools

ugus | O. N. | JULY | NO.7 VOL.9 4 , , B S Manjunath 9 t , Badrinath Roysam 14 2012; Imaging Platform, Broad 10 Department of Electrical Department of Electrical doi:10.1038/nme 14 Fig. Fig. re 1 2 ), the chal the ),

Department Department 2012 v 5 ie , t h.208 |

697 10 4 - - - - - ,

© 2012 Nature America, Inc. All rights reserved. SlideBook (3i), Image-Pro (MediaCybernetics) and Volocity Volocity and (MediaCybernetics) Image-Pro (3i), SlideBook tion and analysis. These include Metamorph (Molecular Devices), system. microscope new a planning when software compatibility with hardware is an essential consideration hardware component can rarely be added by Hence,third parties. port is driven by typically interests, commercial and support for a sup will package software given a hardware which of choice the is hardware high, to support code to cost the write ware. Because hard microscopy-related all support packages software all not But automation. and control microscope for packages software independent different several between choose to researcher the are systems not, requiring microscope wide-field typical whereas acquire an image are almost always bundled with control software, ( events acquisition of sequence the researcher to permitting and design easily execute the desired the hardware as functions quickly and as flawlessly while possible that such actions their and coordinate components with various these communicate to needed therefore is software acquisition Image- experiment. large-scale a in samples multiple or sample series, in time-lapse images of numbers large of acquisition unattended the allow to or a from sample information multiparametric desired to gather necessary is automation This hardware). or software in mented x-y wheels, filter as shutters, such equipment other cases computer-controllable with most synchronized tightly in be to needs straightforward, acquisition image is computer the into that camera equipment and a from image an capturing detector Although interest). of area point the scans a (using sequentially or of workflow. the stages most for tools of software use 1 Figure 698 initially were packages software these of Most (Perkin-Elmer). re Several commercial software packages combine packages image software acquisi commercial Several that systems require computer control microscope to Confocal stages, stages, Workflow

system | v O. N. | JULY | NO.7 VOL.9 ie | s Overview of imaging workflow. Modern bioimaging requires the requires bioimaging Modern workflow. of imaging Overview w z -axis focus drives and autofocus mechanisms (imple and drives focus autofocus mechanisms -axis

Image analysisand Image acquisition Image storage visualization z -stacks, multiple spatial locations in a large locations spatial multiple -stacks,

2012 | nature Fig. Fig. 2 met ). h o d s Public image repositories and toolkits Bioimaging Ontologies annotation Machine learning libraries Image

Katie Vicari - - - -

series, collecting three-dimensional (3D) stacks at multiple multiple at stacks (3D) three-dimensional collecting series, analysis strategies (acquiring individual images, taking time-lapse ages is that they provide a turnkey solution to all ‘standard’ image- (Olympus). cellSens and (Nikon) NIS-Elements nies have their own software packages, such as AxioVision (Zeiss), compa microscope major the of each later. addition, In cialized and were commer laboratories research in individual developed can be used to create a graphical user interface. Developing new new Developing interface. user a graphical to create used be can and equipment available of subset a to interfaces provide which (Mathworks), Matlab and Instruments) (National LabView code. as own their write often by such environments is development toolkit facilitated Software must equipment and needs ing inventor. the of that outside to laboratories can be transferred techniques new imaging which the rate atreduces This system. to hardware changes the imaging extend any of these software packages or make substantial custom substantially to researchers individual for however, impossible, virtually Itis system’s the configuration. with compatible be not pur are as of is there will part chased they less danger an system, imaging packages such when Moreover, on). so and positions tion, and supports continuous image acquisition synchronized synchronized acquisition image continuous supports automa and basic tion, and acquisition image of modes standard most used is extensively for and two-photon excitation microscopy microscopes laser-scanning control to framework overnight plate multiwell a in cells live in events fusion cell tagged fluorescently tracking as such time, and space over data to multichannel used collect be can program The wheels. filter and stages cameras, as such microscope, light the of components the of control full components. hardware for adapters device own its contribute and write to commercial, or academic entity, any enables framework software The approaches. imaging new for of software and transfer development the facilitating Python, as Icy, such and Matlab, LabView environments in interface user ImageJ its without used be also can layer abstraction hardware plug-ins. or scripts as distributed easily be can tions func The routines. solu microscopy tions as image-acquisition well as customized common execute and design to researchers enables and plug-in ImageJ an as runs that interface easy-to-use systems scanning with used also is it although micro of terms in control. scope environments toolkit the than use of ease vide tools with more than flexibility commercial tools and greater is development whose by driven are researchers, to intended pro developed. sufficiently not are the are channels and costly back-ends the distribution because infrastructure primarily others to tools these distributing for ate appropri less are they them, wrote that group the for intended environments provide and high flexibility are well suited for tools microscopy microscopy laboratories structured-illumination include research in written software by enabled opments not of devel Examples could packages. software simply existing in anticipated be techniques new these for needs the as code, instrument-control writing necessitates technologies imaging The The obvious advantage pack of image-acquisition commercial eerhr ta hv nntnad r rqety chang frequently or nonstandard have that Researchers Another open-source package, ScanImage, provides a ScanImage, package, software open-source Another Two software projects, open-source FOCUS ONBIOIMAGE INFORMATICS 3– 5 and Bessel-beam microscopy Bessel-beam and µ 8 . Manager mainly targets camera-based imaging, imaging, camera-based targets mainly Manager µ Manager Manager and ScanImage, µ 6 2 Manager provides provides Manager . Although toolkit toolkit Although . , super-resolution super-resolution , 7 9 . It includes an an includes It . . . It implements µ Manager’s x-y ------

© 2012 Nature America, Inc. All rights reserved. ments, including both images and image metadata. Such systems and images metadata. image both including ments, experi biological with associated data visualize and share lyze, ana process, search, organize, to platforms integrated provide databases databases’.image These ‘image to as referred are often These images. scientific of collections large managing for tions and analyze datathese ( software applications that can properly manage, view, share, process‘Enterprise-level’ data generation must therefore be matched with techniques tion automa New sophisticated generated. day are routinely per data of image of gigabytes hundreds to tens In many ble. laboratories, infeasi computer ing is the software typically data with everyday analyz or viewing finding, fact, In experiments. imaging about an notebook unsuitable solution for keeping track of information experiments and acquisition protocols has rapidly made a paper lab The vast increase in data volume and the complexity of bioimaging IMAGE STORAGE vendors. commercial to and community scientific to the benefits providing cameras, scientific-grade ling a become essentially has interface its that cameras manufacturers’ different many so supports now requiring without Furthermore, efforts. to major run them software also development, labs instrumentation from on techniques focused optical new of dissemination rapid the by facilitating community scientific the aids domain in ware this soft Open-source software. bioimaging open-source from cally cultures. neuron in signaling axon such as is ratio tracking needed, signal-to-noise high where in- as recordings complex for allow and such systems, confocal house–built microscopes, laser-scanning control can it that in complements ScanImage development. plug-in and oriented and event-driven to promote extensibility, online analysis is object- framework software The animals. in intact for imaging useful particularly is which data, physiological or behavioral to 2 Figure for image storage and retrieval are quite dependent on suitable suitable on dependent quite are retrieval and storage image for FOCUS ONBIOIMAGE INFORMATICS In the last 10 years several groups have developed applica developed have groups several years 10 last the In that can benefits organi arise certain exemplify projects These | basedonanalysisofimagedata • Acquisitionprotocolispredefinedordeterminedonthefly • Singlechannel,multiplechannelsorhyperspectral • Singlefocalplaneorthree-dimensionalstack • Singletimepointorseries • Manualorautomatedacquisition Image acquisition spans a range of complexity and variation. and a range of complexity spans acquisition Image 1 0 promise to increase and accelerate this trend. trend. this accelerate and increase to promise Fig. 3 ). de facto de standard for control for standard µ µ

Manager Manager Manager Manager

Katie Vicari ------

tational resources in clusters and in the cloud. the in and clusters in resources tational compu of power increasing the of advantage taking collections, data image large analyze and process to server a on hosted use tools researchers where framework a provides architecture This metadata, annotations and output analytic in a images, client such as a web browser.the of view a delivers and data the manages and holds application server a where architecture’ ‘client-server a as to referred is technology of type This connection. internet a standard using or data her his with work can a scientist access’: applications. also invokeBoth their individual the idea of ‘remote that allow users to extend both metadata and workflow models for project (BISQUE) platform Environment’s (OMERO) Microscopy the Open Objects Remote texts for bioimaging experiments, it to is for texts create nearly impossible experiments, bioimaging plant biologists to for platform cyberinfrastructure analysis and management image iPlant scalable a provide the with integrated is platform BISQUE The available. also are (Quartz), Database Image PCI the and OMERO, on based (PerkinElmer) system Management Data Image Columbus the as such solutions Commercial tions. collec image manage scientists help to systems software of logy (PSLID). Database Image Location Subcellular Protein algorithm (FALCON) Retrieval Loop Content-Based for Adaptive Feedback the on building capability, this vides below). pro OMERO.searcher these software open-source the As an example, discuss (we parameters model or features using for images that are similar to a byquery some measure of distance searching content’.involves image This by ‘query as known also image retrieval, be can evant found images also by content-based gies that formalize the names and relationships among image image among ( metadata relationships and names the formalize that ontolo gies by enhanced greatly is annotation image of usefulness of or the human content interpretation of the image ( automatic the as well as image the experi create to the used details mental describing of terms in both images, of annotation Figure 3 Figure Image databases can be searched using annotations, and rel and annotations, using searched be can databases Image Two examples of open-source bioimaging database projects are BISQUE and OMERO are just two examples of a growing eco of a growing examples are two just and OMERO BISQUE notebookorstoredinanimagefilemanagement system • Imagemetadatahand-writteninlabnotebook, storedinanelectronic • Singlecopyofdataorperiodicbackups orageneralimagefilemanagementsyste • Filesdepositedintofolders,aninstrument-specific database • Imagesstoredonalocalharddriveorremote server • Imagetransferandstorage:manualorautomatic | Options for image storage vary in complexity and size. and complexity in vary storage image for Options 1 1 and the Bio-Image Semantic UserQuery Environment Box Box 1 2 4 ). . Given the vast variety of applications and of con . applications variety Given vast the nature 1 2 . Both are web-based open-source projects projects open-source web-based are Both .

met h File folder o d s

| O. N. | JULY | NO.7 VOL.9 m Server 1 re 3 Box Box used in the the in used

2012 v ie 1 ). ). The |

w 699 Katie Vicari ------

© 2012 Nature America, Inc. All rights reserved. Bio-Formats to import proprietary microscopy image data. Given OME the uses and model data XML OME the supports BISQUE (MathWorks). Further, BISQUE and OMERO Matlab share data models. with interfacing for exist implementations and ImageJ, and CellProfiler with work can BISQUE and OMERO both ple, facilitates connections to also other open-source and bioimaging software. project For exam development the in participate and to that the community allow review, opment contribute practices ( repositories available can be deployed in an lab, individual a network of labs or publicly Together community. they represent a class of solutions imaging database that imaging the of requirements unique the with that couple the and technology server leverage file and database OMERO enterprise latest and BISQUE applications. monolithic complete, not and frameworks adaptable provide OMERO and BISQUE why is This developers. database the by unanticipated if even application scientific the by demanded functionality and scientific Rather, needs. image databases must be flexible and scientists’ able to integrate all applications for solution universal a 700 for tools visualization and analysis of development rapid the re This This flexibility stems in part from the use of open-source devel metadata. and to tary metadata header automatically ess tools The record imaging annotation impractical. a A experiments annotation initial amount like experiment use machine-computable means itself). ated information with images To Box 1

bioimaging

major |

Some Despite help v

O. N. | JULY | NO.7 VOL.9 be

for

of challenge allow

some

form.

ie as metadata

analyzed,

them, analysis.

just

of researchers. of

Whether

require address

well w tools entice of challenge data

im annotation, and the

for other

its The The

information

the

required has

or 5 This

about as

either easy experiment recognized are is

resulting ,

even store

also

Bio-Formats Open

the biologists

(information though information annotation, the information

not

overwhelming, readouts, retrieved,

using is in A

and

reuse lies how For

the

challenge in information

development yet GE

owing

Microscopy for Box Box

that

manner 2012

part describing

robust formal image

in image pixel efforts

become that the all

of value,

to is 3

the capturing

biological

to visualized ).

because

project conceivable

image NNO currently

necessary

the regularly

images

about data

nature about

acquisition, of harvesting files. the

lack is ontologies

are

often

causing

image about Environment widespread proprietary process

often

to images

are scale

underway

5 data of

the

TAT Unfortunately,

7 the

the and ease

all

met

is images

user-friendly not often

time-consuming record and/or

image for metadata

critical

content

a of types

for

images, inherent most downstream

h the of

ION of the

library in

( o fully microscopes their the Box

d associating

a acquisition stored

image for a

samples

can

in metadata. annotation acquisition

new researchers has

shared, of

systematic bioimaging

for

3 extracted

bioimaging own certain of

including

) metadata capture used

contain amount

two

purpose. or making in these

the formats annotation

purposes.

uses informal a

were

solutions by biological

usually

image

and proprie

images areas.

proc

and

to and a in of of

full

for

cre

the vast

the

- -

hundreds or thousands of images are collected by automated automated by collected are images of thousands or hundreds which in experiments for step necessary a is analysis image lar, convert microscopy images into quantitative data to analysis image using in interested increasingly are Biologists analysis Image information. forand papers resources cited detailed online explore to readers encourage and available tools the of overview from myriad types of microscopy images. We provide a high-level and techniques to extract biologically meaningful quantitative data experimental designs require a rich diversity of ever-evolving tools and applications modalities, imaging different from derived data image and linked, are inextricably processes two These alization. visu and analysis their for methods and themselves images the bioimaging pipeline, the heart of bioimage informatics is, of course, the in step of every importance the denying no is there Although IMAGE ANALYSIS AN future. near the in and now biologists for tools powerful very provide to integration promises and code of use collaborative of type this imaging, visual plex details shtm#wi volume organism tool the annotation annotation system record a the a metadata image that many and annotation workflow, Annotation the analysis adoption. acquisition, lection produced. sheer tent curation,

practical container Although

need database. main

implemented. is experiments, are allows

number open-source

formats map

metadata in

leading of of is

b in and

for

FOCUS ONBIOIMAGE INFORMATICS

/ export development approaches type

to metadata found the As but

These

choice

development

(

of for of with

format is

are http://vizbi.org/ images

the full robust To the

The Open and

systems the needed regions

and and to the

in encourage

OME-XML ( systems, format in current ontologies field

the

for

Cell-Centered full ProtocolNavigator

volume

the

and

the

full anatomical for

an tp://openccdb.org/so must curation Microscopy at

D for software

capture

increases of

workflow

covering open early

acquisition the to accurate commercial parsing V

and

every ISUALIZATION automated support interest

often

enable whether

of data biologists Bio-Formats. OME-XML

and

image stages

and are images

Posters/2011/D06

tools stage

region

be of

Environment model.

both

its

still conversion Database

annotations

especially

images, manual sharing for

for

annotated or

providing format. of demand imaging

annotation for

that need of data captures text images to

manual manual

implementation where

The

the

than which record OME-TIFF

especially annotation

of are

ftware/index. and model Web to

The

needed

OME-TIFF biological for of

software

the

stored

the automatic be will 120 also and annotation

now through

protocols graphical

the

Image experimental quantitative OMERO

developed of image and

image

increase. proprietary 15 automated

offers

routinely is

proprietary aim image

given , in for

therefore

6 of serves

tools project Browser

. . In particu

imaging

manual

and

data, data

com

col the as

image

after the to con

- a

- is

- -

© 2012 Nature America, Inc. All rights reserved. ages have been developed for biological applications, especially especially applications, biological for developed been have ages and ( reproducible that areresults objective accurate, such as fluorescence-lifetime data data at nonspatial or looking cells cancer invading tracking time, zation of a signaling protein, measuring changes in structures over many for important biological studies: for is example, quantifying the amount processing and locali image context, throughput high- a in analysis image to addition In data. image of volumes or time collecting samples, multiple screening for whether microscopy, FOCUS ONBIOIMAGE INFORMATICS A A vast number of image-analysis algorithms and software pack Biology data which ogy accompany which and The repositories several generation similar gies cal computable form each ontologies the using same plurals ularies vide with between of consistent used of readable expressions enhanced analysis resources, Public Annotation Box 3 Box 2

There classes image

languages

nucleus

Cell: (JCB)

The

can

associated ontologies. class and of

item

the

provides merged

bioimage

to and to years, -series data, or other technologies that generate vast vast generate that technologies other or data, -series

a effective

are

Still

across Cell perform

form them

metadata, An

the

pu on uniform

same annotation. (such that

through DataViewer via often

and are description

published lexical

of of of ‘handle’. a

are

Image

Centered

others

5 cell

new such and number

in computable. human images, the (an

of disambiguates

experiments

single full

with geared unique

as data

focused

First, Ontologies

and categorization 2012.

a or

B T

resource the

hierarchies variants)

brain

organelle cell the

as Library human.

access such

OLOGIES

are LI

tissue

Third,

articles

repositories

flexible

OWL papers,

( term

of knowledge Database number This

at of

to http://jcb-datavi use or to

Others identifier, C

and

on valuable

the

visual be

organelle) education

(web

of type

( allows

identifier,

when

(for For the enable

http://www.c particular and to

useful is

the

published

most

original

means focused well-constructed ontologies. such

multiple based

gather of 1

be A part

example,

type example, ( annotations through

imaged. ontology laboratories, . Image analysis can help ensure

http://ccdb.ucsd.ed

nucleus

them constructed about have benefits resolved multiple ideally

basic

the for

on to and and of

R which

multidimensional sets

on the

domains. a

items

image

been is consistent

apply in

microscopy rules a

ewer.rupress.org cell). level,

They

a EPOSI outreach, ontologies the of

providing

structured the Ontologies growing

domain language), ellimagelibrary.org Journal

the of to reasoning as terms

serves

available by

images being relationships

image and nucleus controlled

retrieval describing is

using

can Journal of Cell ontologies ont atom).

defining greatly These

(synonyms, 6

axioms

in of description o 0 also

such as described .

research logy Fig. Fig. or

annotation

formal

enable

O that are machine- Some Cell

u ontolo process

an for Second, / stains

include and R

the enable

image

4 vocab a names formal

as the

IES easily

Biol (for ).

pro

cell,

set logi the the

)

- - examples that have been proven useful for light microscopy across is tool which choose to task. given a for appropriate difficult often is it solutions, mercial com and open-source of number the Given readers. format file images generated by or either most directly instruments, through for most used now can be programs software third-party scopes, the with micro bundled the use software of only the commercial necessitated often formats file proprietary past the in Whereas areas, application usability, of openness and the code source cost. in in differ intended The the their packages past software decade. ess elements protocols data (Cell cal Ontology), cover the US efforts ing databases. especially to form ter). it example, nist.gov/bioroot/bioroot.pl concordance the terms have pace scientifically and help standard ing uq.edu.au tories Atlas lular Project Brain the

Below we highlight some of the more prominent and widely used is Ontologies

bear

National entities

of experiments

images Neuroscience experimentalist, Human

which

both

acquisition

identify with Ontology). By

Location also as

( major Atlas can formal (

htt

such http://flybase.

(

on a expressing http://ww

a methods (Information

(Ontology p://www.wormatlas.or

been formal a /

rapidly

across

Tools

effective

applications Purkinje

chemicals ). from

(Foundational ( type Protein biological

Center as http://www.brai

with ontology useful. developed and Methods which

Image

the proposed

Other

and (OME-XML), exist these

ontology, multiple

Information ontology-based

evolving existing nature of

Open w.emouseatlas.org for

neuron image

Atlas

cell of

data search

with are tracking Database

for

org entities, ontologies for construction databases Biomedical Biomedical Artifact

and

and biological

is in

Biological

are

( certain

guidance

searching, met annotation disciplines,

Model http://proteinatlas.

ontologies

) imaging human / development

and and

which

functionality evolved and ).

type experimental most n-map.org

h Framework. access

( o

Ontology). such uses h interpretation

d g

LOCATE of

ttp://pslid.org metadata are cover s /

services

interest Investigation) Ontologies

highly valuable

knowledge ),

Ontologies

Anatomy) technologies. provided |

GABAergic O. N. | JULY | NO.7 VOL.9 locally as GABA

still and Drosophila analyzing

(ref. in scales slow ),

cell entities /

for

machine-computable ( ),

the

downloads developing.

http://locate.imb.

granular

can are Ontologies

However,

techniques

(Chebi), 59

as relevant the and components

into for

others,

and

that Protein

Bioportal

and a most and

can be project,

org of

e-Mouse the neuron

and describing neurotransmit may

/ Image

modalities.

and built ), image

Systems

encourages /

http:// be cell

vocabulary

anatomi comm

ontology the through to

distribut the

re not

will

Subcel

brought the now data

Using

and

types

bioimag the

into

Reposi because

and 2012

Worm

data, proc Atlas (Gene

v keep

u Allen

xpdb.

nity ie

- |

- - 701

- - © 2012 Nature America, Inc. All rights reserved. and sold together with imaging instrumentation. There are and manywith instrumentation. imaging sold together companies microscopy by offered often are these (Zeiss); and ZEN Cybernetics) (Media Plus ImagePro SlideBook, Elements, NIS- Software), Scientific (Bitplane (Visage Imaris Volocity, Amira Imaging), MetaMorph, include category this in tools cial commer Some applications. multiple to flexibility greater offer problems. These software packages are typically modular and thus is that packages can those more address software analysis general tools. image-analysis Generalist edu/resources/external-links Systems for and ( Manipulation Microscopy Integrated Computer Health–funded of Institutes National ( US neuroscience focuses for which analysis image (NITRC), on Clearinghouse Resource and Tools Neuroimaging the as such specialties, biological certain for ings list online through or engines search web via software this find to appropriate sometimes is it tools, such of variety and number the to Owing devices. neuroprosthetic surrounding tissue brain of substrate glio-vascular map the to need out of the grew which toolkit, (FARSIGHT) insight multidimensional for rules ciation continued.dis Oneand more prominentactive example both is the fluorescencetools, asso such of hundreds modalities. literally are imaging There and readouts assay organisms, neurons), (especially types cell particular for solely designed is that exists Software hand. at problem research a to relevant tasks specific very accomplish to written are academia in developed packages tools. image-analysis Niche storage). and acquisition data as (such flow work bioimaging the of steps other for software with each and other with interoperability support that tools image-analysis visualization on and focus We areas. research biomedical many variation. and 4 Figure 702 open-source image-analysis solutions originally developed to solve re

| v O. N. | JULY | NO.7 VOL.9 andscoring • Model-basedimageanalysisormachinelearning–basedclassification • Analysisofwholeimagesorindividualbiologicalobjectsin • Multipleapproachestoprocessandrendermultidimensionalimages • Visualizationofrawimagedataorextensivepreprocessing • Manual,assistedorautomatedimagesegmentationandanalysis • Quantitativeanalysisorqualitativevisualization ie | Image analysis and visualization span a range of complexity a range of complexity span visualization and analysis Image w

2012 | /

). nature Most image-analysis software software image-analysis Most The second category of image- category The second met http://nitrc.or h o http://cismm.cs.u d s g / ), or the the or ),

nc. Katie Vicari - - - - -

ware. Similarly, the active mailing lists and chat-room channels channels chat-room and lists mailing active the Similarly, ware. soft the of strengths the explore to place excellent an are forms plat most for available resources online extensive The (see below). visualization 3D with for facilities together best the and, offers BioimageXD, applications neurobiology behavioral toward for biased features unique offers analysis, cell segmentation and cell tracking, and Vaa3D is Icy heavily data, microscopy For do. to example, designed Fiji is currently the tool of been choice in analysis of had electron- they work the for relevant most are that analysis image of aspects emphasize they tasks, most for usable are they although therefore, and projects, biology ticular par in involved researchers by developed are platforms source ImageSurfer Symposium Visualization Pacific IEEE the of Proceedings in Hansen C. and Chien C.-B. (Y.Otsuna, Wan,H. FluoRender Macro to Nano From Imaging: Biomedical on Symposium International IEEE 3rd the of (S. Pieper, W. B. Lorensen, and Kikinis; R. Schroeder analysis) visualization–assisted (3D BioImageXD as such purposes, other to expanded or used later but community particular a of needs the munity and to offer new functionality for microscopy analysis for microscopy functionality new and to offer munity plug-ins and tailored to features the specifically microscopy com was distribution to developed offer ImageJ bundled together with ImageJ Fiji the challenge, this To address solutions. many offers that tool a within begin to where but use, to tool which picking in only not is scientist the for challenge major A overwhelming. be can them among selecting because few too having as difficult many too is just having options as In contexts, and some macros. projects software is the proliferation of features, options, plug-ins resources. developer improved and large multidimensional image as support, a such more flexible data features model many for support include to ImageJ of eration ( project ImageJ2 com community-driven the by evidenced scientific munity,as the of needs the meet to evolving constantly is ImageJ large. very grown has community user the and sciences, image-processing and image-analysis tasks, particularly in the life specialized) many (and common of variety wide a perform can and Icy ImageJ status, as pioneering and such history rich its platforms to Vaa3D.Owing by adopted later a was it architecture made general the has and users, end and extensibility developers both among This favorite hand. at algorithm application- specific the just developing on dis focusing and while semination infrastructure ImageJ’s leverage can scientists that tool is this of success the for reasons main the of One areas. tion applica different in tasks image-analysis and image-processing various accomplish to software this within macros and plug-ins tool image-analysis multipurpose widespread and always popular most has the notably, is It and, free. been time of period longest the for been use has in it because tools open-source of landscape the in tion tools. the of impact the evaluating of means tions are and published cited, offers which traditional extensively solu innovative Many developers. software’s dedicated the interaction with direct in engage to used be can projects the by run http://developer.imagej.ne As with niche image-analysis software, even generalist open- generalist even software, image-analysis niche with As One challenge in extensible, interoperable, community-driven community-driven interoperable, in extensible, One challenge posi unique a occupies Image) NIH called (originally ImageJ FOCUS ONBIOIMAGE INFORMATICS 2 4 , OsiriX , 27– , vol. 1; 2006), Image Slicer, Reconstruct Slicer, Image 2006), 1; vol. , 2 2 5 9 and IMOD and . Researchers have written hundreds of of hundreds written have Researchers . t ), which is developing the next gen next the developing is which ), 2 2 6 1 . , CellProfiler , 1 8 , Icy , 1 9 , Fiji , 2 2 Proceedings Proceedings , 3D Slicer Slicer 3D , 2 0 , Vaa3D , , 2012), 2012), , 2 2 3 0 ------. ,

© 2012 Nature America, Inc. All rights reserved. to provide a next-generation ImageJ with improved low- and and low- of The coupling are this functionality. benefits numer improved high-level with ImageJ next-generation a provide to workflow automated more a enabling them, between link a of because ing to use ImageJ for image for and processing cell CellProfiler track Itsoftware packages. image-analysis is now for possible, example, from has benefited many between interfaces recently constructed community biological The analysis. machine-learning advanced using measurement and tracking, and identification cell lution, run to need deconvo and may denoising image including routines many analysis researcher a analysis, cell time-lapse of case the in example, For needs. researcher a function every offer tool can one no as interoperable, and extensible ideally is software Extensibility, interoperability and code sharing. further for atlases anatomical analysis to integrative mapping and brains) (embryos and systems biological complex of imagery dimensional multi algorithms large-scale of analysis state-of-art and visualization provides registration, for Vaa3D system. update less inte tools, build ing pipeline integrated the by documented as tools the of existing features best very the combine to aims such as and field bioimage-analysis open-source in the project youngest is Icy the sets data image multidimensional of renderings 3D paths through flight recording by visualizations immersive generate to can be used BioImageXD and and macros. plug-ins via are extensible analysis 3D and 2D for options many offer (ITK), Toolkit Insight the and (VTK) Toolkit Visualization the on based both high- or BioImageXD data have image emerged. dimensional 3D on focusing tools analysis and learning tracking. object machine by scoring type pheno intensity measurements, morphology including and several address areas, to high-throughput application used is it analysis, accommodate to and colonies cells, bio including of systems variety logical a for image- pipelines customized analysis create to matched and mixed be can that modules curated of success record and track utility a has that sciences life the for tool image-analysis open-source communities. research science puter com and biology the between laboration col interdisciplinary productive facilitate and algorithms new prototype rapidly to written in Java such as ImgLib (see below), libraries algorithm with combination in used be can that languages scripting sev eral bundles Fiji Additionally, authors. and users plug-in between feedback rapid facilitates that system updater integrated an through distributed are plug-ins The ous, particularly in the areas of performance, multidimensional multidimensional performance, of areas the in particularly ous, FOCUS ONBIOIMAGE INFORMATICS More recently, comprehensive image- comprehensive recently, More multipurpose flexible a is CellProfiler arohbii elegans. Caernorhabditis 3 0 . Fiji is closely collaborating with the ImageJ2 project project ImageJ2 the with collaborating . closely Fiji is r actions with microscopy hardware and a seam a and hardware microscopy with actions 2 2 2 1 . It contains highly highly contains It . . Designed Designed - - - -

Software name Table 1 KNIME CellOrganizer PatternUnmixer CellProfiler Analyst Ilastik PSLID WND-CHARM OpenCV ITK VTK FarSight Vaa3D CellProfiler Icy BioImageXD Fiji ImageJ Bio-Formats OMERO.searcher Bisque OMERO ScanImage µ Manager Image-analysis 1 | 8 and Icy Summary of open-source software discussed in this Review

1 1 8 9 - - - - - Machine learning Machine learning and data Machine learning Machine learning Machine learning Bioimaging library Bioimaging library Bioimaging library Visualization Visualization and image Image analysis Image analysis Image analysis Image analysis Image analysis Image format conversion Image content search Image database Image database Image acquisition Image acquisition Workflow system Machine learning, modeling . ,

analysis analysis and visualization Primary function tions that make the choice easier or unnecessary. which is All why there the is a big challenge, open-sourcemovement to this develop software applica realizes fully operating community an developer The chooses system. user computer a way same the in criteria, much intangible other and use of ease interface, familiar for to down preference comes often tools between image choosing analysis in However storage. user or a acquisition features data the for have want would will tool one only often as alone sets on feature and can be based is much easier decision this analysis, image than other workflows informatics image of a aspects some is there categories With overlap. feature substantial with these options of array daunting across and software open-source and commercial among user: Both the use. to for tool which challenge choose to key how a up bring listed) not more many ( general in informatics image and analysis image platform. image-analysis an Choosing to challenges. platform image-analysis daunting each the solve of features best the match and mix to logists bio enable will interoperability only as code and data share to ways developing are explicitly in section this discussed platforms forms using Java. It is encouraging to note that all the open-source to and repackage share with in other packages Fiji and other plat easier centralized, more is that code for allow will core ImageJ2 micro scopy, in illumination available Fiji uniquely plane selective or ImageJ2 in microscopy lifetime fluorescence as such modalities, new including analysis image multidimensional support better will Fiji and ImageJ2 of multi The making difficult. dimensions, analysis three dimensional ImageJ or of two to model limited data largely is current The modularity. and support nature http://murphylab.cbi.cmu.edu/software/ http://www.cellprofiler.org/ http://www.ilastik.org/ http://pslid.org/ http://code.google.com/p/wnd-charm/ http://opencv.willowgarage.com/wiki/ http://www.itk.org/ http://www.vtk.org/ http://www.farsight-toolkit.org/ http://www.vaa3d.org/ http://www.cellprofiler.org/ http://icy.bioimageanalysis.org/ http://www.bioimagexd.net/ http://www.fiji.sc/ http://rsbweb.nih.gov/ij/ http://www.openmicroscopy.org/ http://murphylab.web.cmu.edu/software/searcher/ http://www.bioimage.ucsb.edu/bisque/ http://www.openmicroscopy.org/ http://www.scanimage.org/ http://www.micro-manager.org/ http://www.knime.org http://cellorganizer.org/ PatternUnmixer2.0/

met h o 3 d 1 . . Similarly, improvements in the s

| O. N. | JULY | NO.7 VOL.9 n -dimensional ( -dimensional W / ebsite All these options for for options these All re Table n

2012 D) model model D) v ie 1 | and and

w 703 - -

© 2012 Nature America, Inc. All rights reserved. live developing embryo. Below we review some of the visualiza the of some review we Below embryo. developing live a in localization and expression protein of dynamics observing for time over recorded are sets data multiview multichannel, as multiplied becomes expansion dimensionality to the microscopy, montaging this enable dimension axial the to microscopy extend section serial as such resolution sub-micrometer with montage a as to copy,cent large of regions (millimeters imaging tissue micros step-and-repeat perform to researcher the allows stage computer-controlled a of addition the as large very be can alone across three spatial dimensions and time). The spatial dimensions enable direct capture of using a variety of software packages, modern microscopy is images microscopy 2D most of visualization Although visualization Image applications. many from sets feature select and analysis pipeline, allowing users to build their own virtual systems ibility. as each application components in enable calling They the flex more even offer community analysis image biological the in emerge to beginning just are that below) Workflow (discussed systems both. use can they packages; software alternate two between decide to have not do they if easier is user a for choice the means approach monolithic typical a beyond below. Moving in architecture a way that mutual makes integration as we software discuss their develop projects the that important is It ImageJ. inside Icy to run It however, is, not possible plug-ins. ImageJ call directly CellProfiler, and Icy as many, such and data, their save) cases some in (and to open Bio-Formats use discussed programs the of all example For another. inside run be even can platform one and in below) cases some (see in cations the form of libraries above from use code other appli described applications software of R. courtesy (YaleCoifman and (Duke L. University). University) Carin measurements. cell-track ( the data cell into on groups the based data. cytometric ( ( measurements morphological (size). same movement ( on paths a overlaid spatiotemporal in view,on a are strings’ ‘beads displayed showing the 3D movement paths of for cells detecting anomalies ( are and delineated identified to correspond that rows numbers with of a of table measurements cell (data not ( shown). F5 in thymocytes cyan, green, and dendritic in The violet. cells is shown step segmentation, cell as first an orthogonal ( 3D movements of in thymocytes an profiling of data, the using the ( extracted spatiotemporal FARSIGHT toolkit. Figure 5 704 in microscopy. used We currently approaches tion the direct also re b a

| v O. N. | JULY | NO.7 VOL.9 ie | Screenshots illustrating the image-analysis steps starting from a multichannel, multiphoton time-lapse movie from in the starting a time-lapse a Screenshots culminating steps multiphoton illustrating multichannel, image-analysis bioinformatics w

) Coifman bi-cluster plot organizing the organizing plot tracks modules into cell on measurements. are groups the based ) bi-cluster Coifman track-based Bi-cluster n 2012 -dimensional data (several data channels c 0.0257 |

ex vivo 3 nature 4 e . In selective plane illumination illumination plane selective In . ) Scatter plots provideplots ) Scatter a ofsummary visual cytometric of pairs measurements. ( preparation of a live developing mouse thymus at 2-min intervals, with wild-type thymocytes displayed in preparation of displayed thymocytes a mouse developing live thymus wild-type with intervals, at 2-min met Brightness Opacity h o d x, x, y s 32 and g 0.8 , ) Histogram of) Histogram ( tracking cell measurements (track tortuosity). 3 3 . Techniques Techniques . t ) projection for convenience of assessing cell-tracking accuracy ( ) accuracy for projection of convenience cell-tracking assessing

methods i meters) meters)

trivial trivial g d - - - - a ) This movie (courtesy ) of movieThis E. (courtesy Robey, of recorded University Berkeley) California software optimization such as that used in the Vaa3D the in used software that as such optimization software careful and engines exploitationof hardware processing graphics angle and chosen zoom factor. results from usually the The speed at screen the onto interactive speeds, allowing the user to examine projected the data from any be can cube image multichannel and contemporary desktop graphics hardware, a three age, visualization and analysis needs. With modern software tools However, the benefits come at the cost of observations. more complex data separate stor together piece to need the avoiding text, con and temporal spatial full in entities biological of labeled son data image of visualization the to devoted specifically Review a included which data, biological prior the to reader into the RGB space of computer monitors for 3D rendering. 3D for monitors computer of space RGB the into channels data map and blend efficiently to manager color based systems software cases, 3D– of For thousands such channels. as different organized stacks image registered or probes antibody of dozens example, for samples, independent in reporters of numbers large of imaging image stack to terabytes of image data sets). (for example, per for gigabytes large > extremely image 8–10 files dering windows in memory and an additional navigation window ren 3D local and global both combines that method a alization, visu hierarchical is approach obvious One challenge. a remains volumes image multidimensional massive through browsing Maps–style Google scale. This strategy can also be easily realized on the web, allowing views that display only the data currently viewed at the appropriate strategy used, for example, by ImageJ’s View5D uses cross-sectional Vaa3D can be used to produce real-time 5D rendering. An alternate When the entire image can series be loaded in computer memory, utdmninl mgn tcnqe alw iet compari direct allow techniques imaging Multidimensional Another source of increased dimensionality is systematic systematic is dimensionality increased of source Another FOCUS ONBIOIMAGE INFORMATICS 3 6 . Interactive 3D visualization of large data sets sets data large of visualization 3D Interactive . h e Nature Methods Nature 2 b 1 ), ), and a ‘3D kymograph view’, showing the now allow users to use a spreadsheet- a use to users allow now 3 5 f . ) Coifman bi-cluster plots organize organize plots ) bi-cluster Coifman b x, x, y, z h , c ) Scatter plot view plot ) of Scatter ofpairs ) The cell-tracking ) results The cell-tracking supplement on visualizing visualizing on supplement and time ( i f c ). ). ( d ) Histogram of) Histogram cell- t )) )) view. Cells - dimensional 2 1 - - - - - .

© 2012 Nature America, Inc. All rights reserved. tive mechanism for getting feedback on the capabilities of the the of capabilities the on feedback getting for mechanism tive effec an are and active quite be to tend They resource. valuable are and lists forums mailing another Online library’s capabilities. the of overview an provide and showcase library the of uses typically common the They host. projects software these of most that tutorials introductory the visiting by start to convenient is workflow, it image-processing for a given needed functionalities ing or in library a whether practice toolkit provides particular the ( required computer background science of level and speed focus, algorithmic supported, images of requirements, types memory compatibility, language programming community. scientific the from input with rapidly evolve and iterate to potential their of because libraries for helpful ticularly aging libraries are free and open-source. Open-source code is par as the Image Processing toolbox in Matlab, the majority of bioim applications. end-user to tionality func of addition enable and modular be to tend They software. into end-user incorporation before tested and flexibly rapidly be to approaches new role by a allowing critical serve toolkits These built. as be can or applications software directly end-user which either upon pieces experience, programming with those by used, be can that algorithms of low-level a collection is A library processing software but also imaging libraries, or toolkits ( not only robust image- needs community informatics The image BIOIMAGING LIBRARIES AN below. discussed toolkits to image-processing major extensions from growth in data volume to terabytes and beyond has motivated stemming The and challenges software. analysis the visualization both from the perspective of the imaging hardware technology and easily. together glued be can others and output, and input image segmentation, and registration image as such modules many which with face, Vaa3D handles various image-analysis tasks using a plug-in inter segmentation. watershed-type 3D used widely the to compared error, segmentation of rate at lower a identities cell single of tion identifica robust more allows image, 3D acquired newly a to fit built approach Vaa3D upon recognition and segmentation simultaneous a instance, For spaces. feature the of analysis further for platform Vaa3Da as features. act can the among or groups of outliers tion quantitative analysis techniques, such as the automatic identifica and results tracking and segmentation of editing enables data. also It numerical derived their among and objects among ships mult the with and other each with displays these links actively toolkit FARSIGHT The displays. progression and forms query database panels, bi-clustering dendrograms, histograms, plots, scatter ing includ tools, adapted specially requires data multidimensional from features image extracted of the interpretation and alization ‘features’. termed commonly data, visu analysis, subsequent The and tracking operations produce massive amounts of multivariate ( ( segmentation example, for analysis, image software, practical advice on problems how to advice apply it to specific practical software, Fig. FOCUS ONBIOIMAGE INFORMATICS A variety of image-analysis libraries exist, differing in their their in differing exist, libraries image-analysis of variety A Although some commercial image-oriented libraries exist, such development, of phase rapid a in is imaging Multidimensional emerge after automated A needs new whole set of visualization

i 5 dimensional image data, enabling exploration of relation of exploration enabling data, image dimensional b ), feature extraction ( 3 7 , which deforms the atlas of of atlas the deforms which , Fig. 5 D TOOLKITS c ) and modeling. Segmentation Table C. elegans C. i. 5 Fig. 2 ). ). When evaluat a ), tracking tracking ), 3

8 to best best to Fig. Fig. 6 ). ). ------

tive cellular descriptors. cellular tive ( EBImage and structures, data and algorithms customizable on emphasis (VIGRA; iwr.uni-heidelberg.d Algorithms Generic with Vision libraries include used actively of examples some below, detail more in to code In new support functionality. to described addition those and in some cases even ideas or assistance in modifying the source of functionalities. 6 Figure algorithms that can identify notable structures from images, images, from structures notable identify can that algorithms feature-extraction example, for offers, OpenCV (C++, Python). and C languages native for vision, computer of domain the in source library that provides a rich set of image-analysis algorithms architecture. client-server a provides which VTK, of top on built application open-source ( ParaView use to well as common is it allel computation platforms, such as clusters and supercomputers, par distributed of use the demand that sets data large very with ( Go-Figure BioImageXD, Icy, are VTK or ITK on based are that larger than 4 gigabytes. Among the many bioimaging applications HDF5 JPEG 2000, and formats, including image TIFF images file of variety for a large support provides also ITK dimensions. four n supports The toolkit and feature image stitching tion, extraction. for image registra image in segmentation, particular algorithms, of image-analysis collections largest one of the provides ITK ers. to find ITK and VTK being used together by application develop it is common visualization; than rather data processing on actual focused library is a complementary ITK clusters. and trees plots, such as information, charts, of for methods visualizing collection a provides also It visualized. being objects the with interactions rendering techniques, together with 3D widgets that facilitate user 2D and 3D meshes with a images and large of geometrical variety are then processed one by one. VTK’s main focus is on visualizing memory by the partitioning input data into segments smaller that computer main the in fit to large too are that sets data managing for support provide toolkits two These TclJava. and Python, lar particu in languages, other into wrapped are then and C++ in written are VTK and ITK Both needs. processing by unexpected ing that pipelines provide and the required adaptability flexibility processed output data. These filters can be combined into process produce and data input take that filters units, of data-processing a prominent and role. VTK as Both ITK are a designed collection http://w http://gofigure2.org -dimensional -dimensional images, with emphasis particular in two, three and OpenCV ( OpenCV have ITK and VTK libraries, bioimaging open-source Among | Bioimaging libraries and toolkits are available to cover a range cover to available are toolkits and libraries Bioimaging ww.r-project.org 3 9 , a modular package that leverages the R the environment that leverages package , a modular http://opencv.wil nature e/vigra / ), Vaa3D and FARSIGHT. When dealing dealing When FARSIGHT. and Vaa3D ),

/ met ) ) to segment cells and quantita extract / ), a computer vision library with an an with library vision computer a ), h lowgarage.com/wiki o d s

| O. N. | JULY | NO.7 VOL.9 • Reportgeneration • Statisticalanalysis • Machine-learningalgorithms analysisalgorithm • Imageprocessingand • Databaseconnectors • Fileformatreadersandwriters • Instrumentcontrol http://paraview.or s / re ) is an open- an is )

2012 http://hci. v ie g / | ), an an ),

w 705 Katie Vicari ------

© 2012 Nature America, Inc. All rights reserved. such as KNIME will be indispensable for ecosys diverse creating such be as indispensable will KNIME data management such as OMERO or BISQUE or tools workflow imagefor platforms Integrative analysis. downstream for them capable of capturing efficiently the analysis results and presenting a need for additional development of agreed-upon data structures exchange. As bioimage data become increasingly complex, there is format and to rely on the standardized OME-TIFF format for data by packages each one enabling of to them anyimport proprietary software diverse of usage enables library Bio-Formats The age. output can be used as seamlessly input for another software pack data and the results of computational analysis so that one software (Java C) versus is to on agree to formats common store the image that rely on incompatible fundamentally programming languages libraries. by both provided functionalities the upon build and VTK ITK. to between This empowers application developers and ITK, and OpenCV between example, for bridges, by linked been recently have libraries bioimaging many addition, In as Fiji). such microscopy, multidimensional from images large very processing emphasizes that software versus CellProfiler, as such screens, high-throughput in of images of thousands of hundreds processing automated for designed software example, (for cases integration among diverse platforms geared toward use particular maximum achieve to mechanism one are libraries software eral and gen informatics, in bioimage progress for long-term crucial is interoperability of issue The data. same the on software ferent dif using pipeline processing the of parts different running or platforms between solutions algorithmic moving easily enables interoperability. their improve substantially future the in will which representation, data image of means alternate least at or primary the as ImgLib using toward migrating of process the and are OMERO Icy) in (KNIME), Miner Information Konstanz major Several server). Java-based software projects (Fiji, ImageJ2, internet or disc (memory, strategy storage or types) complex or and 3D 2D, (1D, dimensionality mulation into a single piece of code that will run on images of any for mathematical a transforming algorithm, image-analysis the of essence the on concentrate to developer software the enables called library Java a under the Fiji primarily ImgLib ImgLib was project. developed, Java, in written are or use analysis image video. of hours of hundreds of analysis requires often which models, of animal behavior of phenotype monitoring automated the in especially used Itbeen sensor. has imaging an of plane 2D into the project that they features with from 3D space ing objects objects in video sequences and calibration algorithms for correlat feature matching and that algorithms tracking can follow moving EBImage VIGRA ImgLib OpenCV ITK VTK Library Table 2 706 tools. bioimage-analysis cooperating of tems re Another way to achieve interoperability even among way Another platforms to interoperability achieve data image representing for library common a using Indeed, bio for platforms generalist used widely the of many Because

| v O. N. | JULY | NO.7 VOL.9 | ie Summary of image-analysis libraries w Language

R C++ Java C++ C++ C++

2012 R Python Java Python and Java Python and Java Tcl, Python and Java Compatible with | nature n D), type (8-bit, 12-bit, 16-bit 16-bit 12-bit, (8-bit, type D), met h o d s Image dimensions 2D 2D and 3D n n 2D plus time n 2D and 3D D D D 2 0 ------

based image processing algorithms are often used to directly directly to used often are algorithms processing model- image Whereas based images. whole or cells) in example, interest (for of images regions pixels, individual either of classification ( algorithms ured config manually use that tools than expertise less require often tools these reality in learning, machine of concepts the by dated performance whose may be intimi non-experts Although programs improves experience. with creating with concerned field images microscopy involving experiments to applied powerfully been has learning Machine MAC will be sensitive. be will program machine-learning the to which changes of image types the determines directly represent of features content image these variety The image. the approximate that equations polynomial in factors and statistics pixel textures, the as such of whole a as image properties on report also can they but and boundaries), nuclei cell example, (for identi interest first biological by of structures derived fying be can images to algorithms learning achieved system the accuracy classification the about statistics of set a as well as can, human a than better often classes, predefined the into images new sort to used be can that classifier trained a is output the case, either In use. to set ‘learn’, optimal or an determine, methods tational compu the let and features of set a large extract to is alternative an cases, these In goal. desired the achieve practice in not do or measurements robust an get to for sufficiently define difficult to are experimenter priori, a unclear are measure to quantities However, the often classes. the between distinguish that features the of values the decide to used are methods machine-learning and features, the calculate to Image- used then are nuclei). methods cell processing of shape or size number, the as (such lem prob to the relevant to be or expected are thought that features) (called two quantities or of and more images specifies collections or differentiation undergone has a cell whether belongs, such as whether a protein is in image one organelleparticular a class or which another, to decide to or need often biologists cell), a as such entity, biological a showing thereof portion a or of view field a whole At (whether of regions each. level image the which class of interest, based on the researcher providing example pixels of which an to belong image to identify algorithm learning one tool that open-source enables to researchers train a machine- ( Ilastik cases. challenging in par ticularly class, each to belonging as pixels classify automatically to approach alternative an be can learning machine background), and cytoplasm nucleus, example, (for interest of regions identify level for users Computer science Machine learning in bioimaging is mainly used for classification— h faue cluae a te ai fr plig machine- applying for basis the as calculated features The provides experimenter an problems, such many address To H Basic Medium Advanced Medium Advanced Medium INE LEARNING FOCUS ONBIOIMAGE INFORMATICS Fig. Fig.

7 Analysis and segmentation Filtering Segmentation and registration Feature extraction, tracking and visualization Segmentation and registration Filtering and visualization ). 42– 40 , 4 4 1 4 Algorithmic focus and is typically defined as a as defined typically is and . htt p://www.ilastik.org

transformation. / ) is is ) ------

© 2012 Nature America, Inc. All rights reserved. by their effects by their groups find within data. Forto example, clustering has used been used to group be drugs can learning, machine unsupervised of a form analysis, cluster case, In this (or uncertain). are unknown a an of absence the model. of priori because learning machine in crucial and is experimenter the of hands the in is set training the in systematic bias and bias selection of Elimination day). to day from the images training are acquired on one day and there is variation of all if as (such samples experimental the in present variability the represent not adequately does set training if the encountered are sometimes Problems overtraining. as known a phenomenon the that so images, training the to used customized are too become not does images system training the images. of new subsets to Different generalize can system trained the that ensure concentra tions) increasing at drug a to (responses curves response dose- include Examples a curve’). ‘standard (a function regression construct to outputs known their train by The grouped are used. images are ing classification, with common in concepts many have which methods, regression variable, continuous a is output desired the instead initial When the classes. into images than of sorting other input no or little with images biological segmentation such as cells or nuclei, classification has also been achieved without ing to first identify and isolate (that is, segment) regions of interest learning programs for microscopy rely machine- on somemany image preprocessAlthough response. of extent the estimating or which of the features are informative for distinguishing the classes automatically determines program the cases, both In problem). regression (a received sample each that drug a of concentration pr classification (a classes different for images example together grouping by as such images, between func PSLID. in chooser tionality image typical the using done be can as lication, pub for image an choose to example, for set, a of most representative is image which or distinguishable statistically are images of sets two whether determine to as such directly, distributions feature on the operate comparisons Statistical learning. pervised unsu and learning supervised comparisons, statistical features: bioimaging. in used is 7 Figure for a particular bioimaging problem is often a difficult choice difficult a often is problem bioimaging particular a for FOCUS ONBIOIMAGE INFORMATICS Whether to use supervised or unsupervised machine learning learning machine unsupervised or supervised use to Whether grouped be should images which into classes the Sometimes, to important is it learning, machine supervised using When relationships defines biologist the learning, supervised In the using out carried be can tasks of categories general Three 4 7 | and time points in a time series time a in points time and Machine learning. Application areas in which machine learning learning machine which in areas Application learning. Machine 45 4 , 8 4 and proteins by their subcellular patterns subcellular by their proteins and 6 . Thus, machine learning can be applied to to applied be can learning machine Thus, . • Predictivemodeling • Classification • Featureextraction • Objectorcellidentification o 4 blem) or specifying the the specifying or blem) 4 .

4 9 . 4 1 ------.

age score age individual assign to to lifespan classifier a the train throughout points time and using morphology (WND-CHARM) distance representing algorithms neighbor of hierarchy weighted compound used this of be example An then can (unsupervised). classes samples defined these morphologically into and clustered (supervised), identity treatment the a on of based grouped be might images example, For some information about and classes then extend it using the data. approach referred to as semi-supervised learning. The idea is to use time. over class a within variation or to learning classes known within subclasses classes, new unsupervised attempt to discover use then and classes major the at least distinguish can features the well how determine to learning supervised use first to is approach frequent a obtained), be can (or available are images labeled If analyzed. be will that images other for all of class that are representative class to each assigned images training the that believes user the which with confidence the on also but known already are (patterns) classes important all that believes user the which to extent the on mainly Ithinges damental’ compartments (for example, 10% is in lysosomes and and lysosomes in is 10% example, (for compartments damental’ ‘fun of multiple each in is protein how of much a given estimate directly to is problem this to solution A pieces). into continuum the divide artificially to (or ‘chained’ form to clusters algorithms clustering lead can and patterns when mixed of images with results presented arbitrary produce to classifiers leads This ments. in one compart or more to other fully in one compartment fully compartment. A protein can be anywhere on a continuum between in each on localization showing machine example images training after identify localizes protein to particular a learning organelle which machine to of application the example, Take, for classes. of mixtures handle to used typically not are they available. are organelles some and shape cell for reasonably accurate methods for models building nuclear and however, in applied approach bioimaging; has been only recently are drawn from the same as distribution the training images. This sense statistical a in that images new that synthesize can in and pattern variation the and essence the capture to trained be can erative Given models. multiple images of a given pattern, a model gen image-derived using or pattern morphology cell subcellular patterns they are on. trained An alternative is to to try represent a ‘represent’ the only can systems and clustering classification that systems; retraining must often be performed. A related problem is imaging different or between types cell between well not transfer bioimaging, but there are some limitations. Classifiers typically do accuracy. its increase to examples additional acquiring for optimally system to an control acquisition used was classifier be tially made fully automatic, as demonstrated in ref. examples in selected a process oflabel active to learning user the ask can learner machine a sifications, clas the in uncertainty of degree high a is there If classifier. the retrain to used is information new This errors. correct to tunity of on input previous based new their cells and is given the oppor Analyst CellProfiler in example, For process. learning the during duced There are also several approaches in which input can be intro be input can which in approaches several are also There an combined, be can methods unsupervised and Supervised A final limitation of machine-learning approaches is that that is approaches machine-learning of limitation final A Machine learning has been proven useful for several purposes in 5 0 4 , the biologist is presented with machine classifications classifications machine with is presented , biologist the 4 . nature met h o d s

| O. N. | JULY | NO.7 VOL.9 C. elegans C. 4 1 . . This process can poten a physiological physiological a re 51, 51, where a

2012 v

distinct distinct ie |

w 707 ------

© 2012 Nature America, Inc. All rights reserved. tionality (for example, image processing and data mining) while while mining) data and processing image example, (for tionality func of variety a to access enabling domain, user- this in to friendliness approach interface. common user most the intuitive is an programming’ ‘Visual needing base user broader the to expert both algorithm, an of parameter well: every tweak to how know who community users bioimaging the serve have to tools Workflow begun tool. seamless a integrated single create a to using workflow non-experts enabling to tools, software single-purpose many together string manually experts that ing ago. years 20 ing process word like much very skill, a necessary become has data to able Being reproducibly and intuitively process and read, transform, analyze biologists. modern of majority from the to spreading is specialists way sophisticated a in data analyze and proc to ess need the because attractive increasingly become have tools workflow These applications. of array broad a to adapt to need or solutions testing and exploring are instead but day after day analysis same the are not repeating researchers where niche, tools ( monolithic application-focused, to constrained being or scripts complex writing without of array tools a diverse integrate and protocols and analysis processing data model and intuitively flexibly to researchers allow tools Workflow-based bioimaging. including biology, in fields many across role important an serve Workflow systems for data and processing have analysis to begun W fractions mixing the and patterns fundamental the both estimating by unmixing’ ‘unsupervised do to possible even It is software. PatternUnmixer open-source the using done approach unmixing’ has ‘supervised successful been This methods. and unmixing standard types using various object distinct in fluorescence of amount the by image each resenting rep classifiers), training in used proteins ‘marker’ the as (such training of that are to proteins images known be in just one compartment given done be can This complex). Golgi the in is 90% 8 Figure 708 also giving access to multiple data sources (for example, chemical, Fig. Fig. re ORKFLO Workflow systems can enable a field to transition from requir from to transition a Workflowfield can enable systems

| v O. N. | JULY | NO.7 VOL.9 8 ). As such, these tools often address a complementary complementary a address often tools these such, As ). ie | • Integrationwithotherinformationresources • Workflowsdevelopedbyexpertsserveastemplatesfornoviceusers • Automatictranslationbetweenrequiredformats • Linkingdisparatesoftwaretools • Documentationoftheworkflow,enablingreproducibleresearch • Visualmodelingofentireimage-processingandanalysisworkflo Workflow systems. Benefits of using a workflow system. a workflow of using Benefits systems. Workflow W w SYSTEMS

2012 | nature met h o d s 5 3 . 5 2 w and can be - - - - -

like like is the case for Linux—companies are more willing to invest in and—much backbone tool-integration and data- a as role stage back a taken has itself system workflow The popularity. gained have systems workflow open-source years in recent that surprise steps of for workflow,the bioimaging individual it is therefore no tools open-source state-of-the-art of availability the Given ogy. technol relevant the of all integrate and domain this in needs of single commercial entity to offer solutions that cover the spectrum for a has it made challenging software of tion bioimaging-related evolu continual and spread rapid the Still, research. academic steep a at offered and teaching for used been often thus are and academics have for discount but proprietary are capabilities. tools analysis These data some have also and analysis, which image Pilot, and Pipeline cheminformatics address Accelrys’ that tools of is combination a offers market science life the on focused More analysis. statistical and mining data general more for tools offering Clementine, IBM/SPSS and Miner Enterprise SAS’ as such tools, workflow offer companies Several so. or ade approaches. new of sharing rapid and research reproducible supporting analysis, an disseminate and document and advantages Other are biological textual). the ability to readily what one has done is also important. In thinking about the steps steps the about document In thinking important. also is visually done has one what to ability The tasks. different for library or tool different a use and easily out tools swap to able being user any of importance the emphasize to important is it pipeline the way.in a of analysis sustainable such image using For in the steps steps the and itself pipeline image-analysis the in steps the steps: important two are there workflow, a in involved steps the about thinking When able KNIME. of be version that still using 2018 will in it you run to 2008 in 2.4.3 KNIME in workflow a you ran if contrast, In time. over vary can services web as ‘archive’ to harder lot a are services web that is approach API open the of advantage but the effective, are quite Both format. data-type and by adopting a standard this KNIME (web has services), its own does open API Taverna one. another for tool one swap easily can you that way a such in tools, and libraries other various of tion flexible integra that the way). easy The allow latter are very frameworks open a in necessarily not but tools other incorporating (sometimes by themselves to do are the former everything trying applications similar and tools such as Tavernaother and and KNIME is that Icy between difference main The CellProfiler. or Icy in tasks image-processing of series a as such run, and created be to workflows robust for allow above discussed tools Many the of first to approach a or is monolithic a needed. whether framework define important is it necessary, even is approach workflow a problems. analysis multistep scale, large- to applied be to potential great have and research ceutical still new,very but these approaches are routinely used in pharma is imaging biological academic in software workflow of use The workflows. analysis and image-processing complex of creation the allowing others, among CellProfiler, and OMERO ImageJ, as such tools, bioimaging other to connections has also KNIME domains, including business intelligence and predictive analytics. of set broader even an serves that option an is KNIME whereas ( Galaxy and ( Taverna as such tools workflow open standards for such critical infrastructure pieces. Open-source eea wrfo sses ae mre oe te at dec past the over emerged have systems workflow Several When considering what workflow system to use, or whether whether or use, to system workflow what considering When FOCUS ONBIOIMAGE INFORMATICS http://ga l a x y . p s u . e d u http://www.taver / ) focus on bioinformatics, bioinformatics, on focus ) na.org.uk / ------)

© 2012 Nature America, Inc. All rights reserved. was was solely to address a single step in the bioimaging workflow, the research to biomedical of its value recognition increasing is there but behind, lagged often has Usability interest. mercial com of scope the outside problems emerging new to response in often and needs biology current to answer direct in usually is community developer this by produced software the Thus, labs. typically embedded in, or have strong ties to, experimental biology ity stems from the fact that of developers bioimaging aresoftware terms of both functionality and usability. Emphasis on functional fortunately rule; bioimaging software has recently this rapidly developed in to exception no are microscopy use who Researchers tools. software and techniques computational with familiar ingly As technology progresses, modern biologists must become increas CONCLUSIONS future. the in reproducibility workflow ensuring or state, a to alternately freeze certain to webservices offering call by Tools respect a in time. this choice users offer as such KNIME over results different or may produce to function may cease calls WebService it tools the although via archived, be to itself a workflow case, allow calls) extreme the (in toolkits external on rely of tionality ‘superbuilds’ and ‘external projects’. Tools that heavily CMake makes this process straightforward by providing the func tool of versions configuration ITK. The to multiplatform specific pointing by built also are Icy Python and Vaa3D tools. VTK, other many ITK, and of version specific a to pointing by built is 4.0 Slicer application end-user the example, For bases. code the of repository official the in version tagged specific a to referring simply by or repository combined a in bases code those storing and depends application an end-user on which bases code of the commonly is addressed by This taking snapshots offeasible. the particular versionleast at are reproducibility and control version tools, in For perpetuity. available being open-source the software nature source of the limits of ability the software to others rely on closed- the and analysis, researcher’s another reproduce cannot software the for licenses without those Further, case. the always not is this but software, the to updates progressive with results in theory, to ensure continue that to workflows produce the same most domains. scientific tools Proprietary have the ability, at least version challenge: control. The a ability to reproduce an analysis precisely is in yields critical also system workflow a within ules process. processing image arbitrary an do to way preferred their for toolbox in-house own even their launch can user A analysis). text and modeling chemical as such image the loading, for processing or analysis one wants libraries to use (or other routines software whatever integrate to user the on concentrates of the modeling the KNIME ‘pipeline’analysis and So allows work. the do actually that pieces software the from platform such as KNIME is that it is a separate piece of technology a of strengths the of One applications. image-analysis current in approach monolithic more the by addressed be can needs flow work many and tasks, analysis and image-processing many for workflows. share to ability the and archivability reproducibility, are system workflow a by addressed issues key way, the a sustainable in workflows image-analysis such of using bioimaging bioimaging software community has recently begun to address an FOCUS ONBIOIMAGE INFORMATICS Although the initial focus of each software project in this domain The as ability different to mod technology access cutting-edge needed not is system workflow a that note to important is It

5 4 . ------without community is a static resource of Software limited lifespan,itself. whereas software the than important more much usually is projects software bioimaging open-source most surrounding researchers of community the fact, In thinking. and interdisciplinary innovation rapid fostering by software the of progress the hackathons—promote and conferences tutorials, example, for where the and developers users interact and develop their skills— maintainability and sharing of modular code. The many activities and standards, software quality control, training, documentation, also on software coding itself, including consensus on best practices need to not only collaborate on bioimaging ideas and approaches but packages. software multiple between data transfer tediously to need the reducing by work researchers’ ease greatly connections These here. highlighted as completed, have been already some and prioritized, be to begun have projects software independent among the connections momentum, As gains community meeting. Phenotyping High-Throughput and Imaging Informatics and the Cold Spring Harbor Laboratory’s Automated collaborate, assisted and reflected by conferences such as BioImage and to communicate has begun community software bioimaging open-source The interoperability. usability: of aspect important online version of the pape The authors declare competing financial interests: details are available in the COMPETING FINANCIAL INTERESTS imply that such products are necessarily fit for their intended purpose.or endorsement by the National Institute of Standards and Technology,identified nor in doesthis itreview. In no case does such identification Certainimply recommendationtrade names, company products and open-source software productsD are Health grants R01 GM089652 (to A.E.C.) and RC2 GM092519 (to K.W.E.). the manuscript. A.E.C. and K.W.E. were supported by US National Institutes of and L. Kamentsky and M. Bray of the Carpenter lab for useful input and edits on A. Narayanswamy of the Roysam lab for their assistance in preparing figures, laboratories for feedback and useful comments, in particular A. Merouane and We acknowledge our respective funding sources and members of our A to the thousands of biologists relying on bioimaging. well as connections among them promise to yield great dividends and maintenance of important bioimaging software applications as possible. be Resourcesnot invested otherwise in the development methods but as the catalyst for new biological discovery that would approachesnot only foundation as the serve will for imaging new matics solutions. Continued advances in bioimaging computational innovation depends even more squarely on developing image infor relied on computational methods. Thedirectly future which of of biologicalmany imaging tools, imaging biological new by driven of development the platform. a robust informatics bioimaging validation using crowd sourcing. All of these challenges necessitate for resources, pooling such as in software coding or in testing and opportunities also but sets, data heterogeneous complex sharing and annotating large multidimensional data sets and retaining and community, imaging the manyfacing challenges such as tracking or interdisciplinary to be addressed in individual labs large too problems scientific of attack the enabling and resource source software projects, multiplying the value of a simple software highly networked collaborations are a common property of open- an active community adapts continuously to new problems. These ckno ISCLAIMER As described in Commentaries in in this Asissue Commentaries described The last 20 years have seen extraordinary biological advances advances biological extraordinary seen have years 20 last The w le d gments nature r . met h o d s

| O. N. | JULY | NO.7 VOL.9 54 , 5 5 , , there is a great re 5 6

2012 . . There are

v ie |

709

© 2012 Nature America, Inc. All rights reserved. 30. 29. 28. 27. 26. 25. 24. 23. 22. 21. 20. 19. 18. 17. 16. 15. 14. 13. 12. 11. 10. 9. 8. 7. 6. 5. 4. 3. 2. 1. com/reprints/index.htm Reprints and permissions information is available online at Published online at 710 re

Bioinformatics software. analysis image high-throughput modular CellProfiler: analysis. image of years 25 International Biophotonics three-dimensionalimagedata using IMOD. (2004). 205–216 images.multidimensional DICOM in navigating (2007). 12757–12760 Microsc. J. phenotypes. cell quantifying and sets. data image biological large-scale of analysis quantitative and visualization 3D real-time analysis. research.reproducible extended platform.image-processing throughput 1999). York, New Biol. Comput. PLoS screening. cell-based for images microscopy fluorescence two-dimensional microscopy.high-throughput biology. BasesDataLargeVery on content-based retrieval.in (2010). 544–552 management.and analysis bioimage for platform a biology.experimental biology.systems for imaging operatingscanningformicroscopes.laser products. fusion eukaryotic of analysis complementation fluorescence Bimolecular (2010). 14.20.11–14.20.17 using microscopes of control usingcellsplaneBesselbeamillumination. cells. live of imaging super-resolution (2006). 4258–4272 microscopy. localization photoactivation fluorescence 319 microscopy.reconstructionoptical stochastic by imaging resolution USA Sci. Acad. resolution. unlimited theoretically with imaging fluorescence Bioinformatics Kamentsky,L. ImageJ: to Image K.W.NIH Eliceiri, W.S.& Rasband, Schneider,C.A., ImageJ. with processing Image S. Ram, P.& Magalhaes, M., Abramoff, microscopy.for ImageJT.J. Collins, Kremer,J.R.,Mastronarde, McIntosh,D.N.& J.R.Computer visualization of for software open-source An X. Osiri, & O.Ratib, L., Spadola, A., Rosset, D. Feng, microscopy.section serial for editor free a Reconstruct: J.C. Fiala, Carpenter,A.E. enables E.W.V3D Myers, & J.H. Simpson, F.,Long, Z., Ruan, H., Peng, J. Schindelin, F.Chaumont, de P.Kankaanpää, J.R. Lakowicz, of analysis quantitative the Introductionto Carpenter,A.E. V.& Ljosa, and determination location subcellular Murphy,AutomatedR.F. & E. Glory, S.A. Goff, Payne,Sycara,K.P.Faloutsos,C.,L.,&T.R. for FeedbackadaptiveWu, loop Bisque: B.S. Manjunath, & A. Singh, B., Obara, Fedorov,D., K., Kvilekval, C. Allan, C. Conrad, Pologruto,ScanImage:Svoboda,T.A.,flexibleK. software Sabatini,&B.L. B.M. Ogle, & T.K.K.W.,Kerppola, Eliceiri, C., H.P.,Vincenz, Lin, Computer N. Stuurman, & R. Vale, Hoover,K., N., Amodaj, A., Edelstein, Planchon,T.A. three-dimensional Fast, X. Zhuang, & J. He, S.H., Shim, S.A., Jones, by imaging resolution Ultra-high M.D. Mason,T.P. Girirajan,& S.T., Hess, Three-dimensionalsuper- X. Zhuang, & M. W.,Bates, Wang,B., Huang, wide-field microscopy:structured-illumination Nonlinear M.G. Gustafsson, biology.engineering of area new a informatics:Bioimage H. Peng, | v O. N. | JULY | NO.7 VOL.9 , 810–813 (2008). 810–813 , ie w Frontiers in Plant Science Plant in Frontiers et al. et Nat. Methods Nat. et al. et Biol. Cell Biol.

et al. et et al. et 218

Stepping into the third dimension. third the into Stepping et al. et et al. et et al. et 24 27 OMERO: flexible, model-driven data management for data model-driven flexible, OMERO: Principals of Fluorescence Spectroscopy Fluorescence of Principals , 52–61 (2005). 52–61 , 102 et al. et The iPlant Collaborative: cyberinfrastructure for plant for cyberinfrastructure Collaborative: iPlant The et al. et http://www.nature.com/doifinder/10.1038/nmeth.208 Micropilot: automation of fluorescence microscopy-basedfluorescence of automation Micropilot: et al. et , 1827–1836 (2008). 1827–1836 , , 1179–1180 (2011). 1179–1180 ,

, 13081–13086 (2005). 13081–13086 , 102 5 Rapidthree-dimensional isotropic imaginglivingof Improved structure, function and compatibility for compatibility and functionstructure, Improved Fiji: an open-source platform for biological-imagefor platform open-source an Fiji: l CellProfiler: image analysis software for identifying for software analysis image CellProfiler: , e1000603 (2009). e1000603 , BioImageXD: an open, general-purpose and high- and general-purpose open, an BioImageXD: . Nat. Methods Nat.

Icy: an open bioimage informatics platform for platform informaticsbioimage open an Icy: 9 , 525–537 (2010). 525–537 ,

(MorganKaufmannPublishers Inc., 2000). , 676–682 (2012). 676–682 , 2012

11 Proceedings of the 26th International26thConference theProceedings of Nat. Biotechnol. Nat. Nat. Methods Nat. Dev. Cell Dev. , 36–42 (2004). 36–42 , Nat. Methods Nat. µ Manager. |

Nat. Methods Nat. nature

Genome Biol. Genome 2

, 34 (2011). 34 , 9 Biotechniques , 245–253 (2012). 245–253 ,

12 Nat. Methods Nat. Nat. Methods Nat. Curr. Protoc. Mol. Biol. Mol. Protoc.Curr.

, 7–16 (2007). 7–16 , Biomed. Eng. OnlineEng.Biomed. J. Struct.J. Biol.

9 met Nat. MethodsNat. 8 , 671–675 (2012). 671–675 ,

28 , 246–249 (2011). 246–249 ,

9 h , 348–353 (2010). 348–353 , J. Digit. Imaging Digit. J. 7 , 690–696 (2012). 690–696 , o , R100 (2006). R100 , d 43 J. Neurosci. J. s Bioinformatics

. (Academic Press,(Academic . , 25–30 (2007). 25–30 , 9

Biophys. J. Biophys. , 683–689 (2012). 683–689 , 8

http://www.nature. 116 8 , 499–508 (2011). 499–508 , , 417–423(2011)., , 71–76(1996).,

2 Proc. Natl. Proc.

, 13 (2003).13 , 27 92

91 , , 26

Science , ,

4 . .

41. 40. 39. 38. 37. 36. 35. 34. 33. 32. 31. 60. 59. 58. 57. 56. 55. 54. 53. 52. 51. 50. 49. 48. 47. 46. 45. 44. 43. 42.

PLoS Comput. Biol. Comput. PLoS analysis. image biological for techniques and software recognition Bioinformatics phenotypes. cellular to applications with processing image for package R (2009). 667–672 elegans C. in application Bioinformatics data. image of amounts massive for toolkit annotation collaborative Methods Nat. 9 sections.microscopy ultrathin of series fromreconstruction microscopic3D imageacquisitions. Microsc. J. images.multi-channel 3-D for synthesismontage and registrationimage Methods Nat. data. microscopy illumination plane selective of registration bead-based repositories. ontologies.image cell for for? good they are what and world. 2011). Press, University (Princeton informatics. usability.software Bioinformatics unmixing. pattern unsupervised using locations subcellular between probes USA Sci. Acad. Natl. Proc. patterns. subcellular of unmixing automated through locations subcellular classification. location subcellular protein to application with acquisitionintelligent and USA Sci. learning. machine and feedback iterative with screens (2005). patterns. location subcellular microscopy. cells. single from responses drug of analysis. understanding.image in aging during transitions structural distinct reveals analysis image Quantitative (2003). 31–43 micrographs.fluorescence of evaluation machine: microscopeimages. descriptionclassificationand subcellular of location fluorescence patterns in Biol. Chem. Nat. Murphy, R.F. An active role for machine learning in drug development. drug in learning machine for role active Murphy,An R.F. Pattern I.G. Goldberg, & D.M. Eckley, N., Orlov, Delaney,J.D., L., Shamir, Huber,W.EBImage–an & M. Boutros,O., Sklyar,F., Fuchs, G., Pau, of atlas digital 3D A E. Myers, & S.K. Kim, X., Liu, H., Peng,F., Long, L. Qu, Tomancˇák,V.& Hartenstein, A., Cardona,P.CATMAID: S., Saalfeld, Walter,T. volume Tomancak,P.Elastic & R. Cardona, R., Fetter, S., Saalfeld, Tomancˇák,& Saalfeld,Preibisch,S.S., P.tiledGloballyoptimalstitching of C.L. Tsai, for Tomancak,P.Software & J. Schindelin, S., Saalfeld, S., Preibisch, Swedlow, J.R. Finding an image in a haystack: the case for public image public for case the haystack: a in image an Finding J.R. Swedlow, vocabulary building for concepts NewT.N. Bhat, & J.T.Elliott, A.L., Plant, they are what neuroscience:for Ontologies M.E. Martone, & S.D. Larson, M. Linkert, Science.Networked of Era New The Discovery: Reinventing M. Nielsen, bioimage open-source in challenges Tomancak,P.Current & A. Cardona, bioimaging for call K.W.A Eliceiri, & Kamentsky,L. Carpenter,A.E., of distribution the Murphy,QuantifyingR.F. & T. L.P.,Peng, Coelho, T.Peng, building Model J. Kovacevic, Murphy,& R.F.E., Glory-Afshar, C., Jackson, T.R.Jones, on based proteins of clustering Objective R.F.Murphy, & X. Chen, Z.E. Perlman, profiling multivariate Image-based S.J. Altschuler, & L.F. Wu, L.H., Loo, L. Shamir, automated to microscopy quantitative Murphy,FromR.F. & K. Huang, C.A. Wolkow, & I.G. Goldberg, D.K., Chow,W.B., Iser, J., Johnston, vs W.Human Schubert, & Nattkemper,T.W.,H. Ritter,T.,Twellmann, Murphy,R.F.,Porreca,Velliste,numericalRobust &G. featuresM.for , 717–720 (2012). 717–720 , et al. et J. Cell Biol. Cell J. FOCUS ONBIOIMAGE INFORMATICS

et al. et 106 Source Code Biol. Med. Biol. Code Source Caenorhabditis elegans Caenorhabditis and its application to single-cell analyses. single-cell to application its and et al. et et al. et

et al. et 243 et al. et et al. et Simultaneous recognition and segmentation of cells: of segmentation and recognition Simultaneous Science Nat. Methods Nat.

Nat. Cell Biol. Cell Nat. , 1826–1831 (2009). 1826–1831 ,

7 7 et al. et Determining the distribution of probes between different between probes of distribution the Determining Bioinformatics 26 25 26 C. elegans C. , S26–S41 (2010). S26–S41 , (2010). 418–419 , , 154–171 (2011). 154–171 , Robust, globally consistent and fully automatic multi- automatic fully and consistent globally Robust, Visualization of image data from cells to organisms. to cells from data image of Visualization

Wndchrm – an open source utility for biological image biological for utility source open an – Wndchrm Scoring diverse cellular morphologies in image-based in morphologies cellular diverse Scoring Metadata matters: access to image data in the real the in data image to access matters: Metadata 7 , 979–981 (2010). 979–981 , (2009). 1984–1986 , , i7–i12 (2010). i7–i12 ,

, 327–330 (2011). 327–330 ,

Nat. Methods Nat. 189 Multidimensional drug profiling by automated by profiling Multidimensionaldrug

J. VLSI Signal Process.SignalVLSI J. 6 306 , e1000974 (2010). e1000974 , J. Biomed. Opt. Biomed. J. , 777–782 (2010). 777–782 , , 1194–1198 (2004). 1194–1198 , .

Bioinformatics 9 107 BMC Bioinformatics BMC

13 , 661–665 (2012). 661–665 ,

27 Front. Neurosci. Front. , 183 (2011). 183 , J. Biomed. Biotechnol. Biomed. J. , 2944–2949 (2010). 2944–2949 , , 1854–1859 (2011). 1854–1859 ,

tissues. tissues. 3 9 , 13 (2008). 13 , , 666–670 (2012). 666–670 , Bioinformatics

Nat. Methods Nat. 9

, 893–912 (2004). 893–912 , 27

PLoS ONE PLoS 35 , 2895–2902 (2011). 2895–2902 , , 311–321(2003).,

3 12 , 60–67 (2009). 60–67 ,

, 487 (2011). 487 , 25 Comput. Biol. Med. Biol. Comput.

, 1463–1465(2009)., 3 4

, e2821 (2008). e2821 , , 445–453 (2007). 445–453 , 2005 Nat. Methods Nat. Proc. Natl. Acad. Natl. Proc. , 87–95 87–95 , Nat. Methods Nat.

corrigendA

Corrigendum: Biological imaging software tools

Kevin W Eliceiri, Michael R Berthold, Ilya G Goldberg, Luis Ibáñez, B S Manjunath, Maryann E Martone, Robert F Murphy, Hanchuan Peng, Anne L Plant, Badrinath Roysam, Nico Stuurmann, Jason R Swedlow, Pavel Tomancak & Anne E Carpenter Nat. Methods 9, 697–710 (2012); published online 28 June 2012; corrected after print 20 July 2012.

In the version of this article initially published, Nico Stuurman’s last name was incorrect. The error has been corrected in the HTML and PDF versions of the article. © 2012 Nature America, Inc. All rights reserved. America, Inc. © 2012 Nature npg

nature methods corrigenda

Corrigendum: Biological imaging software tools

Kevin W Eliceiri, Michael R Berthold, Ilya G Goldberg, Luis Ibáñez, B S Manjunath, Maryann E Martone, Robert F Murphy, Hanchuan Peng, Anne L Plant, Badrinath Roysam, Nico Stuurman, Jason R Swedlow, Pavel Tomancak & Anne E Carpenter Nat. Methods 9, 697–710 (2012); published online 28 June 2012; corrected after print 20 July 2012; corrected after print 29 August 2012.

In the version of this article initially published, the disclaimer was omitted. The error has been corrected in the HTML and PDF versions of the article. © 2012 Nature America, Inc. All rights reserved. America, Inc. © 2012 Nature npg

nature methods | 1