Education Pattern Recognition Software and Techniques for Biological Image Analysis

Lior Shamir, John D. Delaney, Nikita Orlov, D. Mark Eckley, Ilya G. Goldberg* Laboratory of Genetics, National Institute on Aging/National Institutes of Health, Baltimore, Maryland, United States of America

Abstract other imaging devices. Automated image rules. An example of unsupervised learn- acquisition systems integrated with labo- ing is clustering, where a dataset can be The increasing prevalence of auto- ratory automation have produced image divided into several groups based on pre- mated image acquisition systems is datasets that are too large for manual existing definitions of what constitutes a enabling new types of microscopy processing. This trend led to a new type of cluster, or the number of clusters expected. experiments that generate large biological experiment, in which the image A review of machine learning in the image datasets. However, there is a analysis must be performed by machines. context of bioinformatics (as opposed to perceived lack of robust image analysis systems required to process Clearly, this approach is different than the imaging) can be found in [3]. these diverse datasets. Most auto- bulk of the microscopy performed for the Traditionally, analysis of digital micros- mated image analysis systems are past ,400 years. However, while the copy images requires identifying regions of tailored for specific types of micros- availability of automated microscopy, lab- interest (ROIs) or ‘‘objects’’ within the copy, contrast methods, probes, and oratory automation, computing resources, images. Once a region is isolated from the even cell types. This imposes signif- and digital imaging and storage devices background, the resolution and dynamic icant constraints on experimental has been increasing consistently, in some range afforded by digital microscopy design, limiting their application to cases the bottleneck for high-throughput allows many types of measurements and the narrow set of imaging methods imaging experiments is the efficacy of statistics to be collected about the object in for which they were designed. One computer vision, image analysis, and question, such as intensity, shape, size, and of the approaches to address these pattern recognition methods [1]. Comput- position, as well as the number of objects limitations is pattern recognition, er-based image analysis provides an ob- and their distribution [4]. This region which was originally developed for jective method of scoring visual content selection can be done manually by draw- remote sensing, and is increasingly independently of subjective manual inter- ing boxes or free-hand regions using an being applied to the biology do- main.Thisapproachreliesontrain- pretation, while potentially being more interactive tool [5], or automatically using ing a computer to recognize pat- sensitive, more consistent, and more computer algorithms known as segmenta- ternsinimagesratherthan accurate [2]. These advantages are not tion algorithms [4,6]. While most image developing algorithms or tuning limited to massive image datasets, as they analysis continues to rely on region parameters for specific image pro- allow microscopy to be used as a routine identification, PR can also be used to cessing tasks. The generality of this assay system even on a small scale. process whole images or images tiled on a approach promises to enable data An effective computational approach to grid without a prior region identification mining in extensive image reposito- objectively analyze image datasets is step [7]. ries, and provide objective and pattern recognition (PR, see Box 1). PR Traditional image processing is there- quantitative imaging assays for rou- is a machine-learning approach where the fore predicated on the question, ‘‘can my tine use. Here, we provide a brief machine finds relevant patterns that dis- objects of interest be identified?’’. In overview of the technologies behind tinguish groups of objects after being contrast, PR is predicated on the question, pattern recognition and its use in trained on examples (i.e., supervised ‘‘can these groups of images be distin- computer vision for biological and machine learning). In contrast, the other guished?’’. In this context, the input to a biomedical imaging. We list available software tools that can be used by approach to machine learning and artifi- PR algorithm may be an entire image, a biologists and suggest practical ex- cial intelligence is unsupervised learning, sub-image region identified with segmen- perimental considerations to make where the machine finds new patterns tation algorithms, or simply image samples the best use of pattern recognition without relying on prior training exam- in the form of rectangular tiles. Thus, in techniques for imaging assays. ples, usually by using a set of pre-defined contrast to selecting and tuning a segmen-

Citation: Shamir L, Delaney JD, Orlov N, Eckley DM, Goldberg IG (2010) Pattern Recognition Software and Techniques for Biological Image Analysis. PLoS Comput Biol 6(11): e1000974. doi:10.1371/journal.pcbi.1000974 Introduction Editor: Fran Lewitter, Whitehead Institute, United States of America Computer-aided analysis of microscopy Published November 24, 2010 images has been attracting considerable This is an open-access article distributed under the terms of the Creative Commons Public Domain declaration attention in the past few years, particularly which stipulates that, once placed in the public domain, this work may be freely reproduced, distributed, in the context of high-content screening transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. (HCS). The link between images and Funding: This research was supported entirely by the Intramural Research Program of the NIH, National physiology is well established, and it is Institute on Aging (Z01:AG000685-02). The funders had no role in the decision to publish or preparation of the common knowledge that a significant manuscript. portion of what we know about biology Competing Interests: The authors have declared that no competing interests exist. relies on different types of microscopy and * E-mail: [email protected]

PLoS Computational Biology | www.ploscompbiol.org 1 November 2010 | Volume 6 | Issue 11 | e1000974 Box 1. Pattern Recognition Overview of Bioimage PR Systems N PR is the task of automatically detecting patterns in datasets and using them to Although there are many examples of characterize new data. PR is a form of machine learning, which itself is a field within artificial intelligence. Machine learning can be divided into two major PR systems, the process can be summa- groups. In supervised learning, or PR, a computer system is trained using a set of rized in several steps (Figure 1). pre-defined classes, and then used to classify unknown objects based on the As with traditional image processing patterns detected in training. In unsupervised learning there are no classes approaches based on object identification defined a priori, and the computer system subdivides or clusters the data, alone, PR can also benefit from various usually by using a set of general rules. An example of supervised learning is techniques to subdivide images into ROIs. automatic detection of protein localization, in which the computer system is The three principal reasons for doing so trained using images of probes for known sub-cellular compartments [10]. An are to 1) reduce the number of pixels the example of unsupervised learning is clustering an expression profiling PR algorithm needs to consider all at once microarray experiment into groups of genes with similar expression patterns. to improve response time or increase N Other approaches to PR include semi-supervised learning, which uses pre- statistical power, 2) bias the PR algorithm defined classes to find new similarity relationships and define new groups, and to process objects of interest rather than reinforcement learning, in which decisions are improved iteratively based on a background, and 3) center or align objects feedback mechanism and specified reward criteria. In this educational article we that have inherent orientation. ROI de- focus on the application of supervised learning to automated analysis of tection algorithms and tools are described microscopy image datasets. more thoroughly in the Finding Regions of Interest section. The second step is the extraction of tation algorithm, PR requires training a (and solely) on the availability and image content descriptors (image features), computer to distinguish groups of images. distinguishability of control images. which are values that describe the image These groups correspond to experimental Thus, a PR approach to an imaging content numerically. These values can controls, and the set of images within a experiment is very closely tied to the reflect various texture parameters of the group encompasses the variation within biological experiment itself rather than image, the statistical distribution of pixel each control. Given these groups of intermediate measurements from image intensities, edges, colors, etc. While the images, the machine can learn on its processing, or familiarity with the algo- dimensionality of the raw pixels can own what aspects of the images represent rithms necessary to produce these mea- typically reach ,1,000,000 (assuming a natural experimental variation and are surements. PR can be used in tandem microscopy image of 1000|1000 pixels), therefore irrelevant, and what aspects are with segmentation algorithms when pos- the number of image features ranges important for distinguishing the groups of sible in order to exploit benefits provided between a dozen to a few hundred. While control images from each other [1,4,8]. by both approaches. Except in a limited each pixel value describes the intensity at a This ability to sort image measurements sense, such as analyzing the confusion given X,Y position, each feature value by their relevance to a given imaging matrices of classification experiments as describes a specific image characteristic. A experiment allows the use of a great discussed in the Interpreting Image Classifi- more detailed description of the types of variety of generic image description algo- cation Output section, most PR approaches features commonly used can be found in rithms that are not specifically related or do not yield the types of quantitative the Computing Image Features section. tuned to each imaging problem, potential- results one gets from segmentation algo- In the next step, the image features are ly making the collection of algorithms very rithms. Instead, it can lead directly to a used to draw conclusions about the data. general. The selection of algorithms and qualitative experimental result, such as Generally, PR methods select features and the rules for combining them are done finding the ‘‘hits’’ in a screen. In general, potentially assign weights based on their automatically as part of the machine- PR is useful as an exploratory imaging ability to discriminate the classes. The learning process, eliminating the need for assay that is independent of any precon- refined feature set is then used to infer a microscopist to select the set of imaging ceptions of the nature or existence of rules for combining them in a classifier. algorithms to use, or adjust the parameters morphological differences in the imaging These two steps constitute the training with which to run them. experiment. PR requires little effort or stage in PR, where the goal is to correctly Images taken with phase contrast, differ- expertise to try. It can be used to check classify the training images. The trained ential interference contrast (DIC), or other whether morphological readouts exist classifier is then tested on control images methods for visualizing gross morphology and develop more specific imaging algo- that were excluded from the training stage. are notoriously difficult for computers to rithms if warranted. This cross-validation is important to analyze because the perceptual model may In this review we describe the general establish the classifier’s ability to identify not be visually apparent, or is challenging outline common to all PR systems used for new images, ensuring that it is not to encode in algorithms [9]. The types of biological microscopy, and focus on tech- restricted to recognizing images it was image measurements that can be used for niques and specific software packages that trained with. More information about PR are not limited to what we can perceive, have been used successfully in biological feature selection and classification can be model, and encode in segmentation algo- image analysis. We discuss some of the found in the Feature Selection and Classification rithms, making it possible to use automated requirements of the experimental setup section. methods to analyze gross morphology that are necessary to take full advantage of Finally, the results of image classification rather than being limited to specific probes, PR and point out some of the differences need to be interpreted by the researcher in or a priori perceptual models. between PR experiments and traditional an experimental context to reach a biolog- The applicability of PR in a specific manual evaluation of microscopy images ical conclusion. There are special consid- imaging experiment depends entirely or model-based image analysis. erations in this interpretation specific to

PLoS Computational Biology | www.ploscompbiol.org 2 November 2010 | Volume 6 | Issue 11 | e1000974 to reduce the number of pixels considered user interface. GemIdent [21] (http:// by the PR algorithm. The most labor- www.gemident.com/) is a multi-purpose intensive approach is to manually define tool for detection and segmentation of regions of interest. While in some cases objects of interest in color images, and this can be the only option, this method provides an interactive graphical user introduces bias and inconsistency, and is interfacethatallowstheusertotuneand too labor intensive to be practical for the optimize the detection. Another tool for analysis of large-scale screens. In this case, automatic detection of spot-like objects in the definition of ROIs can be automated microscopy images is FindSpots [22], by simply dividing the image into tiles which is based on the global thresholding using a regular grid pattern. This simple method, and is capable of detecting reduction in pixel number can increase the objects in 2-D as well as 3-D images. throughput of the PR algorithms, as well FindSpotsisavailableaspartoftheOME as provide greater statistical power by software package [23] (http://www.open considering a larger number of individual microscopy.org/). sephaCe [9] (http:// tiles. www.assembla.com/code/sephaCe/sub In cases where objects of interest can be version/nodes/) is a tool that applies easily identified by segmentation algo- advanced edge and cell boundary detec- rithms (e.g., fluorescently labeled cells or tion to address the difficult problem of structures), subsequent image analysis can cell segmentation in brightfield images. A be more effective if only the regions of powerful tool for 3-D segmentation is interest are processed and analyzed, while V3D-Neuron [24], which can visualize, the background areas are left out of trace, and analyze 3-D images of consideration. Similarly to naive tiling, neurons. rejection of biologically irrelevant areas A practical approach to ROI detection is reduces the response time of the system. A the popular ImageJ software (http:// more important role for this type of ROI rsbweb.nih.gov/ij/). ImageJ allows the use detection is that it can potentially elimi- of external plugins, which enhance it with nate the presence of artifacts that can add features that are not supported by ImageJ noise, and degrade the efficacy of the built-in functions. Examples of ROI computer analysis. In cases where the detection and segmentation plugins devel- objects of interest are difficult to detect oped for the ImageJ platform include among all the objects that appear in NeuronJ [25] (http://www.imagescience. the image, PR-based analysis can be org/meijering/software/neuronj/) and Neu- used to ‘‘learn’’ which of these objects riteTracer [26] for working with images of have biological meaning relevant to the neurons, ITCN (http://rsbweb.nih.gov/ experiment. ij/plugins/itcn.html) for finding nuclei in Some implementations of segmentation various cell and image types, a generic algorithms are designed for a specific type watershed segmentation algorithm [12], of object (e.g., cells), and therefore do not and many more listed at http://rsbweb. require intensive tuning of the system, nih.gov/ij/plugins/. A project called Fiji while other tools are more general, and (http://pacific.mpi-cbg.de/) repackages require adoption to the objects of interest. ImageJ along with a selection of plug-ins Widely used methods for ROI detection and other features useful for bioimage include global thresholding [11], water- processing. shed algorithms [12–14], model-based Often, objects of interest have an segmentation [15], and contour methods inherent orientation that is not preserved [16]. In some cases, automatic edge by the imaging system. Examples include detection can be used to segment regions polarized cells, as well as images of tissues of interest [17]. and whole organisms. In these cases, the A useful tool for cell segmentation is efficacy of PR can be improved if the the open-source software CellTracer [18], objects of interest are registered, so that the which is written in Matlab and can orientation variance introduced by imaging Figure 1. High-level architecture of bio- is eliminated. In some cases, objects may image analysis systems. be downloaded at http://www.stat.duke. doi:10.1371/journal.pcbi.1000974.g001 edu/research/software/west/celltracer/. differ not only in orientation, but also in Another powerful tool for ROI detection scale or local geometry, requiring positional PR, which is further discussed in the is ITK (Insight Segmentation and Regis- and rotational registration as well as local Interpreting Image Classification Output section. tration Toolkit) [19] (http://www.itk. morphing. Combined with the required org/), which is an open-source package accuracy of segmentation algorithms to Finding Regions of Interest designed to detect regions of interest in 2- identify landmarks, registration can be a D and 3-D microscopy images, as well as challenging task. Some software tools for As discussed in the Overview of Bioimage other types of biomedical imaging such as ROI detection and segmentation, such as PR Systems section, the first step of MRI and CT. VTK [20] (http://www. ITK, also offer tools for registration. An computer-aided image analysis is usually vtk.org/) enhances ITK with a graphical application-specific example is the registra-

PLoS Computational Biology | www.ploscompbiol.org 3 November 2010 | Volume 6 | Issue 11 | e1000974 tion of various images of Caenorhabditis still not appropriate for direct processing is simply a collection of images from an elegans nematodes. Normalization of these by PR algorithms. Instead, the pixel data is experimental control. Classification is nor- objects for the purpose of image processing further summarized for its ‘‘image con- mally done by first splitting the dataset can be performed by ‘‘straightening’’ the tent’’ using a set of feature extraction into training and test image pools. The worms and rotating them to a fixed algorithms. Each algorithm reads image training data are used to automatically orientation [27]. pixels and outputs one or more numerical define the classification rules, and the test Sophisticated segmentation algorithms values that describe various aspects of the data are used to assess the effectiveness of can greatly increase the signal to noise image. These algorithms are usually very these rules, and their ability to consistently ratio, but the perceptual models they general in that they can operate on any set reflect the data. Typically, several train- implement or the parameters used can of pixels without specifying any parame- ing/testing experiments are done auto- also lead to errors. When segmentation ters. The types of image features vary matically by randomly splitting the dataset errors are random (non-systematic), there depending on the PR software package, and running multiple trials, as described in will be a corresponding reduction in the but generally consist of texture descriptors, the Experimental Considerations for Effective PR signal to noise. Importantly though, these statistical distribution of pixel values, section. errors can be correlated to the experimen- shape and edge features, coefficients of Many of these image features are tal question, leading to systematic bias and polynomial expansions representing the expected to be irrelevant to the specific skewing of the experimental results. image, and others. imaging problem being considered and In some cases, the dataset includes a Since there is no ‘‘typical’’ microscopy contribute only to noise, while others large number of ROIs, and using all of image or experiment, and the types of contribute varying degrees of discrimina- these images might severely slow down the image content descriptors are virtually tive power to the classifier. Selection of response time of the system due to the unlimited, there is no standard set of relevant features is generally performed computing resources required to compute feature extraction algorithms. Additional- automatically using one of the methods image features for high volumes of image ly, different image features can be relevant described below. The automated compu- data, as will be explained in the Computing to different experiments—even when they tation and selection of image features Image Features section. In these cases, to are based on the same types of images. without user intervention or parameter improve system response time, a subset of Finally, image features calculated by tuning is a key factor allowing robust these ROIs can be selected randomly for computers can be almost arbitrarily de- automation of PR and its adaptability to classifier training. Some bioimage analysis tailed, and can describe patterns that virtually any image type. tools also use interactive user interfaces to people cannot readily perceive. The ability There are two major approaches to select the ROIs manually, and refine them to detect differences between images feature selection: filters and wrappers. The based on classifier performance. This can automatically without the requirement filtering approach typically uses statistical be done iteratively, until the researcher is for a pre-conceived perceptual model is methods to process the entire set of satisfied with the classification results. an important advantage of PR for bio- features to select those most informative. CellProfiler-Classifier provides an example image informatics. Therefore, in most The selection is independent of the of this iterative selection and classification bioimage PR systems, a larger set of image classifier ultimately used in the imaging refinement [28]. For 3-D images, objects content descriptors is computed than is experiment, and thus the features selected for classification can be selected and ultimately used after feature selection and are not specific to the downstream classi- annotated using tools such as OMERO training. This is done to cover a variety of fier. Wrapping, in contrast, is based on [5], VANO [29], and V3D [24], which possible morphological aspects of the selecting subsets of features by testing can allow ROI or image-based annota- images and preserve the generality of the them in a classifier. Thus, wrapping can tions of large and complex 3-D microsco- approach [10,28,30]. select features specific to the downstream py images, including combinations of 3-D, As low-level image features are not classifier being employed. multi-channel, and time-lapse. related to any specific imaging problem, A simple example of the filtering It is important to note that when using they have little utility outside of PR. Thus, approach is computing the Fisher score segmentation software developed indepen- modules for computing image features are for each feature, and rejecting a certain dently of PR, a degree of integration with normally a part of broader applications, percentage of the features with the lowest PR-based software tools is required. Of- and are not distributed as independent scores. The Fisher score is a ratio of the ten, the segmentation results can be software products. Higher-level tools such variance in the feature value between exported as separate images that can be as CellProfiler [31], wndchrm [32], and classes to its variance within classes, giving straightforward to transfer to PR software Protein Subcellular Location Image Data- features with high discriminative power for subsequent analysis. In other cases, this base (PSLID) [33] apply image feature higher scores. This approach is imple- integration requires scripting or substantial extraction as an intermediate step, but the mented in the wndchrm image analysis programming. Therefore, experimentalists values can be exported to third party tools. tool (available at http://ome.grc.nia.nih. are encouraged to first consider software gov/wnd-charm/), where the Fisher scores packages developed specifically for PR in Feature Selection and are also used as feature weights. While microscopy images, providing a full start- Classification providing accurate results for a variety of to-finish solution. These comprehensive image types [34], a potential downside of software tools are described in the Software After the image features are computed this method is feature redundancy due to Tools section. for all images in the dataset, the samples correlations in feature values between can be classified or assessed for similarity seemingly unrelated feature extraction Computing Image Features by using PR tools. Image classification is a algorithms. Selecting more than one task in which the computer system auto- image feature from a group of inter- After reduction of the image size by matically assigns images to one of several correlated features will not add to the ROI selection, raw pixel data is usually user-defined image classes. An image class overall effectiveness of the feature set, and

PLoS Computational Biology | www.ploscompbiol.org 4 November 2010 | Volume 6 | Issue 11 | e1000974 may contribute to noise by over-represent- many iterations with different subsets of with large numbers of classes is to ing a certain type of image content. An the feature bank. A collection of several reformulate the classification problem as effort to address this issue directly is the wrapping methods is available through the a system of classifiers, each operating on a minimum redundancy maximum rele- ToolDiag software suite, that can be small set of classes [54]. Some types of vance (mRMR) algorithm [35], available downloaded at http://sites.google.com/ classifiers [30] do not appear to be at http://penglab.janelia.org/proj/ site/tooldiag/. Another useful tool for negatively affected by even a thousand mRMR/. feature selection and classification is Ra- classes (L. Shamir, unpublished data using Another approach is remapping, where pidMiner [41] (http://www.rapidminer. the FERET dataset from NIST, consisting the original feature space is substituted com/), which provides various feature of 994 classes; [55]). In some cases, it is with another of lower dimensionality and selection and classification algorithms with also possible to exploit estimates of class possibly improved separability. A common a graphical user interface environment. similarity (see the Interpreting Image Classifi- transformation used in several areas of WEKA [42] is another open-source utility cation Output section below) to cluster a bioinformatics is principal component that provides a rich set of classification and large set of classes into a smaller subset analysis (PCA) that has been successfully feature selection tools, and can be down- [7,32]. used for the analysis of DNA microarray loaded at http://www.cs.waikato.ac.nz/ data [36], and has also been applied to ml/weka/. Interpreting Image image feature reduction [37]. PCA maps Classifier training automatically deduc- Classification Output the feature space into a smaller number of es rules for combining the most informa- mutually orthogonal principal compo- tive features into a trained classifier that The most basic piece of information nents. While the primary criterion of can be used to associate unknown images obtained when validating a classifier is the PCA is preservation of data variance, with the user-defined classes. One of the classification accuracy. Determining this other techniques have been proposed that simplest types of classifiers is nearest- requires reserving a pool of test images use different principles for the purpose of neighbor, where the class of the unknown that were not used for training, but whose dimensionality reduction. One such family image is determined from the training class is known. Classification accuracy is of methods is manifold learning, where a image with the most similar feature values. measured by the number of test images manifold [38] is assumed to be embedded WND (a part of WND-CHARM [30]), is a that were classified correctly, divided by in the higher dimensional feature space. variation of this approach where the the total number of images that the Determining this manifold effectively im- training images are used to model a classifier attempted to classify. This num- plements a non-linear transformation into probability distribution for each class. ber reflects the ability of the classifier to a smaller sub-space. There are several One of the first applications of PR accurately associate a test image with its additional algorithms and implementa- classifiers to biological imaging [10] used correct class. Clearly, a higher classifica- tions for these transforms, including iso- neural networks for identifying sub-cellular tion accuracy indicates that the image map [39], local linear embedding [40], organelles. The current implementation of classifier is more informative, and can graph embedding [38], and others. Un- this approach is in PSLID [33]. Another discriminate between images that belong fortunately, the public availability of classification approach is GentleBoosting in different classes. However, as explained software that use these techniques is [43], which is used by CellProfiler [28]. in the Experimental Considerations for Effective currently lagging, and where available, More recently, support vector machines PR section, the classification accuracy itself the software requires additional program- (SVMs) [44] have become popular in sometimes does not have any biological ming to be practically useful. Examples of biological image processing as well as PR meaning, and can lead to false conclusions implementations available include Matlab in general [45]. This type of classifier is unless analyzed carefully and tested libraries for ISOMAP (http://isomap. used in Enhanced CellClassifier [46], as against the appropriate controls. stanford.edu/), a demonstration example well as in an analysis of drug response in In binary classification, the accuracy is of several manifold algorithms with a single cells [47]. An implementation of often reported in terms of sensitivity and graphical user interface (http://www. SVM is SVMlight [48] (http://svmlight. specificity, which are commonly used in math.ucla.edu/,wittman/mani/), as well joachims.org/), and SVMperf [49], which disease diagnosis. The sensitivity of a as a Matlab toolbox for dimensionality can be downloaded at http://svmlight. classifier is defined as the proportion of reduction containing a mixture of linear joachims.org/svm_perf.html. These tools true positives that were correctly detected and non-linear algorithms (http://ict. also provide a user interface and can be by the classifier as positives, and the ewi.tudelft.nl/,lvandermaaten/Matlab_ used as independent tools, as opposed to specificity is defined as the proportion of Toolbox_for_Dimensionality_Reduction. some other available SVM libraries such the negatives that were correctly classified html). Several feature selection and trans- as LIBSVM [50], which are meant to be as negatives. Other performance metrics formation techniques for identifying sub- integrated into other programs and thus for binary classifiers include the false cellular organelles are compared in [37], require programming skills. Another use- positive rate (FPR) and the false negative but to date, a systematic analysis of ful software package that offers a wide rate (FNR). A thorough discussion about manifold learning approaches applied to selection of classification methods is the performance metrics for binary classifiers biological imaging problems has not been ToolDiag PR toolbox. can be found in [56]. attempted. Most classifiers reported in the literature A more informative output of a classifier Wrapping selects features based on their are tested using a relatively low number of validation experiment is the confusion actual performance in the classifier, in classes, typically not more than a few matrix (see Table 1). Each cell contains many cases providing better feature selec- dozen. Biological ontologies, however, can the number of test images known to be tion and greater classification accuracy extend to thousands of terms, and if they members of the class specified by the row than filtering. It should be noted that in are used as the basis for classification, the label that were classified as the class most cases filtering is significantly faster number of classes can increase dramati- specified by the column label. The number than wrapping, which relies on running cally [51–53]. One approach to working of correctly classified images for each class

PLoS Computational Biology | www.ploscompbiol.org 5 November 2010 | Volume 6 | Issue 11 | e1000974 Table 1. Confusion Matrix for Classifying H&E-Stained Mouse Liver Sections by Age.

1 Month 6 Months 16 Months 24 Months

1 month 245 22 43 10 6 months 218 719 117 66 16 months 47 18 225 30 24 months 73 99 227 705

doi:10.1371/journal.pcbi.1000974.t001 is found on the diagonal, and the cells off analysis using PR can discriminate be- errors are present, the biological signal of the diagonal report mis-classifications. tween images taken using different micro- predominates. Here, the relative classifi- Thus, the overall classification accuracy is scopes, objectives, cameras, etc., and cation accuracy between the two can be the total in the diagonal cells divided by potentially lead to false conclusions. In compared and the classification result can the total in all of the cells. addition, it can also discriminate images be accepted because the difference in The confusion matrix also allows esti- taken by different experimentalists. For accuracy is sufficiently great. A better mating the degree of similarity between example, if two different treatments are approach is to make unavoidable system- classes. For instance, if the confusion studied, and images for each treatment are atic bias non-systematic. For example, if matrix shows that the image classifier has collected by a different person, the exper- two researchers must collect data, it is a high degree of confusion between a pair imenter’s acquisition parameters (which better for each researcher to collect the of classes in a given row, it is an indication could be subjective) can lead to a detect- entire set of treatments so that their data that these two classes are more similar to able difference between the sets of images can be equally pooled into classes for each other than the other classes in the where no biological difference exists. classifier training. The effectiveness of this row. These similarities can have interest- The potential for observer bias skewing approach can be confirmed experimental- ing biological implications. Depending on PR results means that little or no quality ly by testing the classification accuracy of the classifier and the imaging problem, control should be done during manual acquisition controls from the different however, these similarities can also be a acquisition or subsequent to automated experimenters pooled together. property or limitation of the classifier itself. acquisition. Traditionally, images are se- Image classifiers differentiate image Therefore these types of similarities need lected manually for being representative of classes based on the strongest morpholog- to have independent confirmation—either the biological treatment. In contrast, when ical signal, which for various reasons may biological, or using a different approach to applying PR, it is the entire set of images not be of interest to the experimenter. An PR, and ideally both. for a particular treatment that represent example of this is a cell growth effect that In some cases, the differences between the class, rather than individual images. is not of interest combined with a the classes reflect an inherent order. For Manual selection of images introduces morphological effect that may be of instance, if each successive class is a considerable observer bias, which may greater interest. One option for eliminat- treatment with an increasing dose, or a skew the PR results. ing the growth effect is to use segmenta- time course as in Table 1, the confusion Since image analysis using PR is tion to identify individual cells followed by between neighboring pairs of cells in a row sensitive to artifacts, image collection PR on classes composed of balanced cell is expected to be higher than cells farther should be as consistent as possible to numbers. When segmentation is not apart. In this case, the highest value of reduce the number of non-biological possible or undesirable, an alternative is each row in the confusion matrix is differences. For this reason, it is important to force the classifier to disregard effects expected to be on the diagonal, and the to collect control images in every session of that are considered unimportant. One other values in each row should decrease image collection or for each experimental example of this was discussed above, for cells further away from the diagonal. batch. Control images can be images of where data collected by different research- subjects that do not reflect any biological ers is mixed together in each of the defined Experimental Considerations differences (e.g., untreated cells). If the classes. An undesired growth effect can for Effective PR classifier is able to differentiate between similarly be eliminated from consideration the sets of control images from the by defining each experimental class using Experiments that utilize PR for micros- different sessions or experimental batches, several different cell densities. A third copy image analysis introduce several the analysis may be affected by artifacts. If option was used by our group to reduce important considerations. As described in the classifier is not able to differentiate variation between experimenters [57], as the Computing Image Features and Feature between the sets of control images, but can well as eliminating recognition of individ- Selection amd Classification sections, the classify between the different treatments, ual mice when analyzing the gender or age image features are selected automatically then it can be deduced that the different of liver sections [58]. Here, we trained a by their discriminative power, and the treatments are reflected in the image classifier to discriminate classes composed classification rules are determined by the content. of the artifact we wanted to eliminate (i.e., system. Therefore, if two sets of images Consider a classifier that can differenti- images collected by one experimenter have biologically irrelevant differences ate between biologically equivalent con- versus images collected by someone else; between them (i.e., due to systematic trols as well as between treatments. For liver sections from individual mice to train errors), the PR analysis could classify the example, an accuracy of 55% between a one mouse per class classifier). We two sets accurately, but the classification controls compared to 85% between treat- eliminated the undesired classification would be based on artifacts. Image ments indicates that though systematic signal from the experimental classifier by

PLoS Computational Biology | www.ploscompbiol.org 6 November 2010 | Volume 6 | Issue 11 | e1000974 subtracting the feature weights of the instance, if a classifier of two classes has 40 Software Tools artifact classifier from the experimental test images of class A and 10 test images of one. For mouse livers, we were able to class B, correct classification of all class A While there are numerous publicly show that this corrected classifier could images will lead to an accuracy of 80%, available stand-alone software tools that resolve gender equally well, but could no even if the classifier misclassified all test can perform specific tasks in the process of longer identify individual mice [58]. Sim- images of class B. These results might PR-based image analysis such as segmen- ilarly, using this approach to eliminate a mislead the experimentalist to believe that tation, feature selection, classification, etc., growth effect would involve training an the classifier is performing adequately, using these together may require program- artifact classifier composed of classes with even though it classifies all images as class ming skills for their integration. Fortunate- different cell densities, where each class A. Another approach to address this ly, some software packages have been contained the full range of experimental problem is to measure the mean classifi- developed to provide a start-to-finish effects. This type of correction is highly cation accuracy for each class separately solution for bioimage analysis and HCS, dependent on the type of classifier being [32] rather than relying solely on the and are often equipped with user-friendly used, and is not feasible in most types of overall percentage of images that were graphical user interfaces targeted at bench classifiers. classified correctly. For instance, in the biologists. Unfortunately, not all PR soft- When testing a classifier for its ability to case above, the 100% accuracy of the test ware is well integrated and user friendly, differentiate between sets of images, the images of class A will be balanced by the and in these cases some additional help classification accuracy should be measured 0% accuracy of class B, providing a should be sought from bioinformaticians, in several runs, where different images are per-class average classification of 50%, or the growing number of biologists with used for training and testing in each run. clearly indicating that the classifier does significant expertise in computing and These multiple trials test whether the not work. information technology. classifier’s performance is overly depen- The image classifier must be trained The software discussed in this section dent on the specific images used in with a sufficient number of sample images was selected based on four parameters: training. When the number of control for each of the predefined classes. There- usability without further software develop- images is extremely limited, validation can fore, an experiment that is based on PR ment, integration of PR techniques dis- also be performed in a ‘‘leave one out’’ (or requires a significantly larger number of cussed above, an established user commu- round-robin) manner, where training is images than an experiment in which the nity, and open-source code. Although performed using all but one of the images, conclusions are made by manual inspec- availability of source code would seem to and the left-out image is used to validate tion or by the use of segmentation tools be of little consequence to non-program- the classifier. This is normally systemati- alone. Normally, accuracy increases as the mers, it is an important consideration. The cally repeated, such that each image in the training set gets larger, eventually reaching foremost reason scientifically is that at least dataset is tested in turn. a plateau where the classifier is said to be in principle, the implementation of the It should also be noted that it is ‘‘saturated’’. This number can be deter- algorithms by the software is independent- important to have the same number of mined empirically by running the classifier ly verifiable. There are also practical training images in each class to avoid repeatedly with different numbers of considerations. If the original authors potential bias caused by an unbalanced training images, plotting the classification abandon the software project without image distribution. If the classifier was accuracy against the number of training providing the source code, then the capable only of random guessing, then it images. This classifier analysis can also be software may soon stop running on new should assign test images to the defined used to determine whether poor classifica- versions of operating systems and hard- classes with equal probability. If one of the tion performance is due to insufficient ware. If the software was an integral part training classes was much larger than the training images, or due to the classes being of the processing pipeline, then previous others, a classifier may assign test images indiscernible by the chosen classifier. experiments may need to be repeated with to the larger class at a rate higher than The number of training images required a new software package in order to expected for random guessing, while the for accurate classification can vary de- compare them to new results. The avail- smaller classes would be assigned with a pending on the difficulty of distinguishing ability of source code usually also means less-than-random probability. There are the classes, and the variability within each that there is a widely distributed pool of several mechanisms that could lead to this class. In our experience with wndchrm, if experts that can modify the software or result, and some classifiers are more prone the classes are easily distinguishable by eye just keep it updated. Even when this pool to this bias than others. The safest and the images within classes are visually doesn’t exist, a professional programmer approach is to use the same number of consistent, generally no more than a dozen can be hired to fix, modify, or update the images in each class for training. If this is images are required for training. An software if this becomes necessary. not practical, a negative control experi- extreme example is identifying binucleate In this section we mention one of the ment can reveal if the classifier suffers phenotypes. Here a classification accuracy most popular image processing programs, from this bias. In this case, the class of 98% can be achieved using a single ImageJ, and discuss four complete systems assignments of the training images should training image. In contrast, our study of C. for biological imaging that rely on PR be randomly scrambled, and the resultant elegans muscle degeneration throughout techniques: CellProfiler-Classifier, PSLID, classifier should be checked to report the lifespan [57] used 85 training images for wndchrm, and CellExplorer, listed in expected random distribution of class each of seven classes, and could have used Table 2. Although ImageJ is not specifi- assignments. more. In this case, human observers could cally designed for PR, there are many While it is important to train a classifier reliably distinguish only very young worms plug-ins available for segmentation, which using an equal number of images per class, from very old ones. In cases where smaller can be valuable for data reduction prior to using the same number of test images can training sets can provide reasonable per- classification as discussed above. Manipu- also be important to obtain an unbiased formance, using larger training sets was lation of image groups is an integral part of assessment of classifier performance. For not found to be deleterious. analysis by PR, and ImageJ does not

PLoS Computational Biology | www.ploscompbiol.org 7 November 2010 | Volume 6 | Issue 11 | e1000974 Table 2. Publicly Available Image Analysis Software Tools Employing or Useful for PR in Biological Microscopy.

Graphical ROI User Open Required Tool Detection Classification Interface Source Language Platforms Software Microscopy Web Site

ImageJ Yes n.a. Yes Yes Java Linux, None All http://rsbweb. (plugin) MacOS nih.gov/ij/ Windows PSLID/SLIC Yes ANN, SVM No Yes Matlab, C, Linux Postgres, Fluorescence http://pslid.cbi. Python tomcat cmu.edu/release/ Matlab CellProfiler Yes GentleBoosting Yes Yes Python, Linux, None Fluorescence http://www. Matlab MacOS cellprofiler.org/ Windows wndchrm No WND No Yes C Linux, None All http://ome.grc.nia. MacOS nih.gov/wnd-charm/ Windows CellExplorer Yes SVM Yes Yes Matlab Linux, Matlab Confocal/ http://penglab. MacOS 3-D janelia. Windows org/proj/cellexplorer/

doi:10.1371/journal.pcbi.1000974.t002 provide a mechanism for associating In contrast to ImageJ, CellProfiler is including phase-contrast, differential-inter- images with each other. The grouping of designed around image processing pipe- ference contrast, and histological stains, as images would allow consequent grouping lines, where many thousands of images well as fluorescence microscopy [34]. of the ROIs produced by various segmen- can be analyzed in batch. The recent Other than dividing the images into tiles, tation plug-ins. A PR plug-in could then addition of CellProfiler-Classifier [28] WND-CHARM does not supply any use these multi-image ROIs to define introduces PR techniques to classify cells segmentation tools of its own, although classes for training. There are several into user-defined phenotypes. Interesting- any software that can produce cropped scripts available for batch processing ly, CellProfiler-Classifier can also find cells images of segmented cells can be used to images together in ImageJ, so the notion that do not fit into any of the predefined provide images to WND-CHARM. De- of image groups may be implemented in phenotypes, making it useful for identify- spite the lack of segmentation tools, WND- the future. ing rare or low-penetrance phenotypes. CHARM has been shown to accurately The PSLID [33] was the first applica- Similarly to PSLID, CellProfiler places score imaging assays that have been tion of PR for microscopy images [59]. constraints on how the imaging experi- traditionally analyzed by segmentation This project aims to eventually enumerate ment is conducted—mainly that the cells algorithms, such as scoring a screen for and discern all subcellular localization must be easily identifiable by the segmen- binucleate phenotypes with 100% accura- patterns. Even though this may seem quite tation algorithms used. Generally this cy [34]. WND-CHARM is self-contained specialized, localization is not limited to requires staining with a fluorescent cyto- and does not rely on extensive external identifying organelles, but can be used to plasmic marker as well as a fluorescent math software such as Matlab, or database describe any type of subcellular distribu- nuclear marker in addition to any markers infrastructure such as Oracle and MySQL. tion for a protein, stain or other biomark- being used in the actual experiment. In This portability allows it to be easily er. PSLID has evolved over the years to light of these considerations, the work integrated into other software that pro- analyze patterns in multiple fluorescence done with CellProfiler and PSLID thus far vides segmentation and database function- channels as well as in 3-D and over time. is limited to fluorescence microscopy. In alities. PSLID can be used with a database to contrast to PSLID, CellProfiler is meant to Another useful tool for automatic anal- manage large image collections. A full be used on a user’s desktop rather than ysis of biological images is CellExplorer installation can also include a Web service provide a centralized Web-based service, [61]. CellExplorer was designed for the to perform image-based or localization- so it is somewhat easier to install and use. analysis of C. elegans images, but was also based searches. The WND-CHARM [30] project was found effective for other model organisms The goal of the CellProfiler project is to initiated to provide PR tools for the such as drosophila. The package includes provide a user-friendly image processing analysis of a broad variety of image types, advanced segmentation, annotation and environment for HCS [60]. In HCS, high- and provide a means to explore different straightening algorithms [62] of 3-D resolution imaging of cells is used as an classifiers trained by grouping the same microscopy images. The segmented nuclei assay in a screen of chemical compounds images in various ways. This software is a can also be classified automatically using or RNAi libraries. These experiments command-line tool [32] that processes an SVM classifier. CellExplorer is freely easily involve tens or hundreds of thou- images arranged into folders representing available for download; however, it re- sands of images, where manual scoring is the image classes, and produces reports in quires the installation of Matlab, which is impractical. Similarly to ImageJ, CellPro- HTML format that are viewable in any commercial software. filer includes tools to identify (segment) Web browser. Unlike PSLID and Cell- The software tools described above and cells and nuclei, and report various Profiler, WND-CHARM has been exten- the segmentation tools described in the statistics on the objects found in an image. sively tested on a variety of image types, Finding Regions of Interest section can be

PLoS Computational Biology | www.ploscompbiol.org 8 November 2010 | Volume 6 | Issue 11 | e1000974 tested using publicly available biological pose questions about potential new rela- within an experimental context. For example, image datasets, which include images of tionships within these image collections. a mis-classification rate was used to measure different organisms acquired using differ- Although the techniques outlined in this cellular response to varying drug doses [47], ent types of microscopy, magnifications, review can lead to general image compar- and a direct measurement of similarity using etc. Some useful publicly available image ison algorithms, image data poses several a linear classifier was used to measure datasets include the PSLID datasets challenges that are only now beginning to sarcopenia in the C. elegans pharynx [57]. (http://murphylab.web.cmu.edu/data/), be addressed: quantitative image compar- Currently, direct measurements of contextu- the IICBU benchmark suite [34] (http:// isons within multiple contexts, relevant alized image similarity using general PR ome.grc.nia.nih.gov/iicbu2008/), and the ranking algorithms, and integrated image techniques is an area of active and ongoing Broad Bioimage Benchmark Collection repositories. Context can be understood in research. http://www.broadinstitute.org/bbbc/. text searching by considering the query Ultimately, for image informatics to ‘‘Orange’’. The results of this query will mature, it requires both contextual image Discussion depend on whether the context is colors, comparison algorithms and fully integrated fruit, computer companies, or cellular In this review we describe the basic image repositories. The last decade has service providers. This level of ambiguity concepts, terminology and software tools seen the creation of several specialized is typical for image-based queries where for PR-based imaging assays for biology. image repositories, some examples of which every image can be viewed in several The information provided is directed include the Visible Human Project (http:// distinct contexts. The practical implication towards the bench scientist looking for an www.nlm.nih.gov/research/visible/visible_ for PR is that a given image in a repository alternative to traditional image processing human.html), the Biomedical Informatics may be used for training several different approaches. Although most of the current Research Network (BIRN, http://www. classifiers, or alternatively a group of applications of PR are used for analyzing birncommunity.org/), the cancer Biomedi- very large image datasets (e.g. PSLID, unrelated classifiers may analyze an image cal Informatics Grid (caBIG, https://cabig. CellProfiler-Classifier [28,33]), these tech- along several distinct contexts. Search nci.nih.gov/), the JCB Data Viewer (http:// niques can be applied just as easily to more interfaces typically allow only a limited jcb-dataviewer.rupress.org/), and a new conventional imaging assays performed in specification of context if they allow one at initiative called The Cell: An Image Library non-specialist laboratories. The principal all (e.g., Google Images, Videos, Maps, (http://cellimagelibrary.org/). Currently, advantage of the approach is its potential News, etc.). This is not adequate for these are collections of images that can be for processing a broad variety of image image-based search due to the high degree searched solely by their annotations and types without requiring customized soft- of ambiguity in the search context, the used to exemplify various biological process- ware or parameter tuning for each imag- difficulty of defining an implied context, es and features. The software infrastructure ing experiment. and its complete dependence on the for these image repositories has matured in The ability to compare images to each experimental question being asked. parallel over the past decade from early other regardless of image type can lead to A relevant ranking algorithm is a key projects such as OME [63,64] and PSLID the discovery of new knowledge from characteristic of a useful search interface [33] to projects like Bisque [65] and existing data. General sequence compari- because the results most relevant to the user OMERO [5] that are maintained by full- son algorithms such as BLAST have are presented first. In scientific image-based time developers and used by an increasing transformed the archiving and retrieval searching, the ranking of search results should number of imaging labs. These types of of sequence data in public repositories be based on image similarity measured along image data management systems are pri- such as GenBank into a field (genomics) one or more biologically relevant contexts, marily concerned with the definition and where new knowledge is routinely synthe- specified by the user. Bisque and PSLID structure of imaging metadata, and provide sized from existing sequence collections. provide image-based search algorithms that interfaces for annotation, search, and brows- By analogy, the integration of generalized return sets of images from the database that ing. Furthermore, these systems allow que- image comparison algorithms with large, are most similar to the query image. ries based either on comparisons between diverse, and well-annotated public image However, it is not possible to refine the entered text and textual annotations in the repositories is an essential step toward search context (i.e., similar in what way?), and database, or image features extracted from a more complete data extraction from the biological relevance of the result ranking query image compared to those of the biological images. For example, metadata cannot be easily evaluated. Quantitative archived images. The integration of image fields used to annotate images either image similarity is an important tool for repositories like these with universal, con- manually or by using specialized algo- imaging assays such as dose-response, time textualized image comparison algorithms rithms can serve as the basis for defining courses, and comparisons of phenotypes. In will substantiate the premise of image training classes for PR algorithms. The addition, measures of similarity can form the informatics: that pre-existing image datasets resulting classifiers can then annotate basis for clustering algorithms, where new can be analyzed in-silico to find new images where these fields haven’t been groupings of images can be discovered based relationships between images, leading to defined. While this process can be fully on existing or related contexts using objective new knowledge and discovery. automated, the tools developed for this statistical criteria. PR techniques have been approach can also be used interactively to previously used to quantify image similarity

References 1. Peng H (2008) Bioimage informatics: a new area 3. Tarca AL, Carey VJ, Chen X-w, Romero R, cence microscopy images for cell-based screening. of engineering biology. Bioinformatics 24: Draghici S (2007) Machine learning and its applica- PLoS Comput Biol 5: e1000603. doi:10.1371/ 1827–1836. tions to biology. PLoS Comput Biol 3: e116. journal.pcbi.1000603. 2. Jones TR, Carpenter AE (2007) Open-source doi:10.1371/journal.pcbi.0030116. 5. Swedlow JR, Goldberg IG, Eliceiri KW (2009) software automates high-throughput imaging. 4. Ljosa V, Carpenter AE (2010) Introduction to the Bioimage informatics for experimental biology. BioPhotonics 14: 31–32. quantitative analysis of two-dimensional fluores- Annu Rev Biophys 38: 327–346.

PLoS Computational Biology | www.ploscompbiol.org 9 November 2010 | Volume 6 | Issue 11 | e1000974 6. Fuller CJ, Straight AF (2009) Image analysis 26. Pool M, Thiemann J, Bar-Or A, Fournier AE 46. Misselwitz B, Strittmatter G, Periaswamy B, benchmarking methods for high content screen (2008) NeuriteTracer: A novel ImageJ plugin for Schlumberger MC, Rout S, et al. (2010) En- design. J Microsc 17: 145–161. automated quantification of neurite outgrowth. hanced CellClassifier: a multi-class classification 7. Shamir L, Eckley DM, Delaney J, Orlov N, J Neurosci Methods 168: 134–139. tool for microscopy images. BMC Bioinformatics Goldberg IG (2009) An image informatics 27. Peng H, Long F, Liu X, Kim S, Myers E (2008) 11: 30. method for automated quantitative analysis of Straightening C. elegans images. Bioinformatics 47. Loo LH, Wu LF, Altschuler SJ (2007) Image- phenotype visual similarities. IEEE NIH Life Sci 24: 234–242. based multivariate profiling of drug responses Syst Appl Workshop. pp 96–100. 28. Jones TR, Carpenter AE, Lamprecht MR, from single cells. Nat Methods 4: 445–453. 8. Coelho LP, Glory-Afshar E, Kangas J, Quinn S, Moffat J, Silver SJ, et al. (2009) Scoring diverse 48. Joachims T (2000) Estimating the generalization Shariff A, et al. (2010) Principles of bioimage cellular morphologies in image-based screens performance of a SVM efficiently. Proc Int Conf informatics: focus on machine learning of cell with iterative feedback and machine learning. Mach Learn. pp 431–438. patterns. Lect Notes Comput Sci 6004: 8–18. Proc Natl Acad Sci U S A 106: 1826–1831. 49. Joachims T (2006) Training linear SVMs in linear 9. Gooding AR, Christlieb M, Brady M (2008) 29. Peng H, Long F, Myers EW (2009) VANO: a time. KDD. pp 217–226. Advanced phase-based segmentation of multiple volume-object image annotation system. Bioinfor- 50. Fan RE, Chen PH, Lin CJ (2005) Working set cells from brightfield microscopy images. Proc matics 25: 695–697. selection using the second order information for IEEE 5th Int Symp Biomed Imaging. pp 30. Orlov N, Shamir L, Macura T, Johnston J, training SVM. J Mach Learn Res 6: 1889–1918. 181–184. Eckley DM, et al. (2008) WND-CHARM: Multi- 51. Carpenter AE, Sabatini DM (2004) Systematic 10. Boland MV, Murphy RF (1999) Automated purpose image classification using compound genome-wide screens of gene function. Nat Rev analysis of patterns in uorescence-microscope image transforms. Pattern Recognit Lett 29: Genet 5: 11–22. images. Trends Cell Biol 9: 201–202. 1684–1693. 52. Carson JP, Ju T, Lu HC, Thaller C, Xu M, et al. 11. Sahoo PK, Soltani S, Wong AKC (2008) Survey: 31. Lamprecht MR, Sabatini DM, Carpenter AE (2005) A digital atlas to characterize the mouse a survey of thresholding techniques. Comput Vis (2007) CellProfiler: free, versatile software for brain transcriptome. PLoS Comput Biol 1: e41. Graph Image Process 41: 233–260. automated biological image analysis. BioTechni- doi:10.1371/journal.pcbi.0010041. 12. Vincent L, Soille P (1991) Image-proc skeleton ques 42: 71–75. 53. Peng H, Long F, Eisen M, Myers E (2006) segmentation. IEEE Trans Pattern Anal Mach 32. Shamir L, Orlov N, Eckley DM, Macura T, Clustering gene expression patterns of y embryos. Intell 13: 583–598. Johnston J, et al. (2008) Wndchrm an open source Proc IEEE Int Symp Biomed Imaging. pp 13. Lindeberg T (1993) Detecting salient blob-like utility for biological image analysis. Source Code 1144–1147. image structures and their scales with a scale- Biol Med 3: 13. 54. Zhou J, Peng H (2007) Automatic recognition space primal sketch: A method for focus-of- 33. Boland MV, Murphy RF (2001) A neural network and annotation of gene expression patterns of fly attention. Int J Comput Vis 11: 283–318. classifier capable of recognizing the patterns of all embryos. Bioinformatics 23: 589–596. 14. Roerdink BTM, Meijster A (2001) The watershed major subcellular structures in uorescence micro- 55. Phillips PJ, Moon H, Rauss PJ, Rizvi S (2000) transform: Definitions, algorithms and paralleli- scope images of HeLa cells. Bioinformatics 17: The FERET evaluation methodology for face zation strategies. Fundam Inform 41: 187–228. 1213–1223. recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22: 1090–1104. 15. Cong G, Parvin B (2000) Model-based segmen- 34. Shamir L, Orlov N, Eckley DM, Macura T, 56. Gardner MJ, Altman DG (1989) Calculating tation of nuclei. Pattern Recognit 33: 1383–1393. Goldberg IG (2008b) IICBU 2008: a proposed confidence intervals for proportions and their 16. Vromen J, McCane B (2006) Red blood cell benchmark suite for biological image analysis. differences. In: Gardner MJ, Altman DG, eds. segmentation using guided contour tracing. 18th Med Biol Eng Comput 46: 943–947. Statistics with confidence. London: BMJ Publish- Proc Annu Colloq Spatial Inform Res Centre. pp 35. Ding C, Peng H (2005) Minimum redundancy ing Group. pp 28–33. 25–29. feature selection from microarray gene expression 57. Johnston J, Iser WB, Chow DK, Goldberg IG, data. J Bioinform Comput Biol 3: 185–205. 17. Li G, Liu T, Nie J, Guo L, Chen J, et al. (2008) Wolkow CA (2008) Quantitative image analysis 36. Knudsen S (2002) A Biologist’s Guide to Analysis Segmentation of touching cell nuclei using reveals distinct structural transitions during aging of DNA Microarray Data. New York: John Wiley gradient flow tracking. J Microsc 231: 47–58. in Caenorhabditis elegans tissues. PLoS ONE 3: & Sons. 18. Wang Q, Niemi J, Tan CM, You L, West M e2821. doi:10.1371/journal.pone.0002821. (2009) Image segmentation and dynamic lineage 37. Huang K, Velliste M, Murphy RF (2003) Feature 58. Macura TJ (2009) Automating the quantitative analysis in single-cell uorescent microscopy. reduction for improved recognition of subcellular analysis of microscopy images [PhD thesis]. Cytometry A 77A: 101–110. location patterns in uorescence microscope imag- Cambridge: University of Cambridge Faculty of 19. Yoo T, Metaxas D (2005) Open science - es. Proc SPIE. pp 307–318. Computer Science and Technology. combining open data and open source software: 38. Basavanhally AN, Ganesan S, Agner S, 59. Boland MV, Markey MK, Murphy RF (1998) Medical image analysis with the Insight Toolkit. Monaco JP, Feldman MD, et al. (2010) Com- Automated recognition of patterns characteristic Med Image Anal 9: 503–506. puterized image-based detection and grading of of subcellular structures in uorescence microscopy 20. Schroeder W, Martin K, Lorensen B (2006) The lymphocytic infiltration in HER2+ breast cancer images. Cytometry 33: 366–375. visualization toolkit: an object-oriented approach histopathology. IEEE Trans Biomed Eng 57: 60. Jones TR, Kang IH, Wheeler DB, Lindquist RA, to 3D graphics. 4th edition. Clifton Park (New 642–653. Papallo A, et al. (2008) CellProfiler Analyst: data York): Kitware, Inc. 39. Tenenbaum JB, de Silva V, Langford JC (2000) A exploration and analysis software for complex 21. Kapelner A, Lee P, Holmes S (2007) An global geometric framework for nonlinear dimen- image-based screens. BMC Bioinformatics 9: 482. interactive statistical image segmentation and sionality reduction. Science 290: 2319–2323. 61. Long F, Peng H, Liu X, Kim SK, Myers EW visualization system. MEDIVIS 2007: 81–86. 40. Roweis ST, Lawrence KS (2000) Nonlinear (2009) A 3D digital atlas of C. elegans and its 22. Platani M, Goldberg I, Swedlow JR, Lamond AI dimensionality reduction by locally linear embed- application to single-cell analyses. Nat Methods 6: (2000) In vivo analysis of Cajalbody movement, ding. Science 290: 2323–2326. 667–673. separation, and joining inlive human cells. J Cell 41. Mierswa I, Wurst M, Klinkenberg R, Scholz M, 62. Peng H, Long F, Liu X, Kim SK, Myers EW Biol 151: 1561–1574. Euler T (2006) YALE: rapid prototyping for (2008) Straightening Caenorhabditis elegans im- 23. Schiffmann DA, Dikovskaya D, Appleton PL, complex data mining tasks. KDD. pp 935–940. ages. Bioinformatics 24: 234–242. Newton IP, Creager DA, et al. (2006) Open 42. Holmes G, Donkin A, Witten IH (1994) Weka: A 63. Swedlow JR, Goldberg IG, Brauner E, Sorger PK Microscopy Environment and FindSpots: inte- machine learning workbench. ANZIIS 94: (2003) Informatics and quantitative analysis in grating image informatics with quantitative mul- 357–361. biological imaging. Science 300: 100–102. tidimensional image analysis. BioTechniques 14: 43. Friedman JH, Hastie T, Tibshirani R (2000) 64. Goldberg IG, Allan C, Burel JM, Creager D, 199–208. Additive logistic regression: a statistical view of Falconi A, et al. (2005) The Open Microscopy 24. Peng H, Ruan Z, Long F, Simpson JH, Myers EW boosting. Ann Stat 28: 337–407. Environment (OME) Data Model and XML file: (2010) V3D enables real-time 3D visualization 44. Vapnik VN (1995) The nature of statistical open tools for informatics and quantitative and quantitative analysis of large-scale biological learning theory. New York: Springer-Verlag. analysis in biological imaging. Genome Biol 6: image data sets. Nat Biotechnol 28: 348–353. 45. Ben-Hur A, Ong CS, Sonnenburg S, Scholkopf B, R47. 25. Meijering E, Jacob M, Darria P, Hirling H, Ratsch G (2008) Support vector machines and 65. Kvilekval K, Fedorov D, Obara B, Singh A, Unser M (2004) Design and validation of a tool kernels for computational biology. PLoS Comput Manjunath BS (2009) Bisque: a platform for for neurite tracing and analysis in uorescence Biol 4: e1000173. doi:10.1371/journal. bioimage analysis and management. Bioinfor- microscopy images. Cytometry 58A: 167–176. pcbi.1000173. matics. pp 544–552.

PLoS Computational Biology | www.ploscompbiol.org 10 November 2010 | Volume 6 | Issue 11 | e1000974