CHAPTER 65,000 Shades of Grey: Use of Digital Image Files in Light Microscopy 13 Sidney L. Shaw* and Edward H. Hinchcliffe{ *Biology Department, Indiana University, Bloomington, Indiana, and Department of Biology, University of North Carolina, Chapel Hill, North Carolina, USA { Cellular Dynamics Section, Hormel Institute, University of Minnesota, Austin, Minnesota, USA CHAPTER OUTLINE Introduction............................................................................................................ 318 13.1 What is An Image File?................................................................................. 320 13.2 Bit Depth ..................................................................................................... 320 13.3 File Formats................................................................................................. 322 13.4 Sampling and Spatial Resolution................................................................... 325 13.5 Color........................................................................................................... 325 13.6 Converting RGB to CMYK .............................................................................. 327 13.7 Compression................................................................................................ 329 13.8 Video Files .................................................................................................. 329 13.9 Video Codecs .............................................................................................. 331 13.10 Choosing a Codec ........................................................................................ 332 13.10.1 Real Results ......................................................................... 334 Conclusions............................................................................................................ 335 Acknowledgments ................................................................................................... 335 References ............................................................................................................. 336 Abstract Computers dominate image capture and analysis in modern light microscopy. The out- put of an imaging experiment is a binary coded file, called an image file, which contains the spatial, temporal and intensity information present in the sample. Understanding what comprises an image file, and how these files are generated is necessary in order to optimize the use of the digital light microscope. In this chapter, we discuss image file formats, and the various components of these files, such as bit-depth, sampling rate, color theory, and compression, from the perspective of the non-computer scientist. Methods in Cell Biology, Volume 114 ISSN 0091-679X 317 Copyright © 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/B978-0-12-407761-4.00013-0 318 CHAPTER 13 Use of Digital Image Files in Light Microscopy We also discuss the problem of proprietary file formats, and how these often are incom- patible with certain types of imaging software. We present several solutions to this issue. Finally, we present the use of digital movie formats, compression routines, and provide some real world examples for optimizing the generation of digital movies. INTRODUCTION Digital microscopy is the recording of an image generated through the microscope into a binary coded file. Like photomicrography and cinemicrography, digital image files provide a permanent record of observations made in the microscope (Furness, 1997). Unlike these analog recording modes, digital micrographs can easily be as- sembled into sequences, to represent motion, or into projections to visualize spatial information in three dimensions, without having to first copy and convert these images into a digital format. The ultimate expression of the medium is the presen- tation of time-lapse motion picture sequences of 3D reconstructions (so-called 4D microscopy; see Salmon et al., 1998; Thomas, DeVries, Hardin & White, 1996). In addition to representing purely visual information from a specimen, digital microscopy image files also contain quantitative temporal and spatial information, relating to the behavior, and/or specific events within cells. This is particularly pow- erful when using sophisticated fluorescent probes to detect and analyze individual molecules in vivo. The images contain quantitative data relating to protein concen- tration, enzyme kinetics, cellular dynamics, etc. To maximize the use of the digital microscope as a quantitative instrument, the temporal, spatial, and biochemical in- formation obtained must be output into a form that can be analyzed, searched, shared, published in journals, displayed on websites, and presented at meetings. Mishandling the collection and storage of information contained within digital image files often leads to unfortunate outcomes. Thus, it becomes an important task to plan for the generation, manipulation, and storage of image-based data. Most biologists have limited training in computer science or in areas of image pro- cessing. Often, the use of computers for biological imaging follows an “oral” tradition based on colleagues or collaborators. While such an approach can propagate useful information, it can also perpetuate misconceptions leading to bad digital habits. In many settings, standard protocols for a laboratory are developed without planning for storage, analysis, or eventual publication. The result is that digital imaging is not optimized, when it easily could be. The following chapter will deal with image files from the standpoint of a noncomputer scientist/graphics artist. The emphasis will be on understanding basic concepts and providing the individual biology researcher with the tools to make informed decisions about generating, manipulating, and storing graphics image files. To begin, it is useful to consider the trajectory of your image data from initial collection to final publication. The majority of the images taken in the research lab- oratory are still used for the purpose of visually comparing a control to an experimen- tal condition. The images can be from live cells or fixed material, 2D or 3D volumes, single frames, or time-lapse series. In each case, a collection of images will be taken Introduction 319 and stored as digital files for later review where the conditions of the experiment and details of the microscopy should be recorded, information that is referred to as “meta- data,” a topic that will be touched on later in this chapter. From these stored files, some level of analysis will be performed through human evaluations, often with con- siderable postprocessing of the original images for contrast and noise abatement or for creating 3D projections or movies. In limited cases, the image data will be sub- jected to more complicated analytical schemes using computational tools to extract numerical data (Cardullo & Hinchcliffe, 2007; Jaqaman et al., 2008; Walczak, Rizk, & Shaw, 2010; Waters, 2009). It is important that the processing and analytical steps be carefully recorded, as these need to be reported during publication. Finally, a select set of micrographs will be developed for presentations and for publication, with explicit instruction from a journal about format and resolution. All of the images should then be archived in a manner that permits search and retrieval in a manner consistent with laboratory notebooks and other data. A key point to consider when generating and using digital image files is that the original, unprocessed images are data, a record of experimental results. These should be maintained as such, in an unmanipulated form. Any image-processing steps should be saved as a separate file, containing a processed image or a set of images. The same holds true for figures for presentation and/or publication, which are in es- sence “graphics.” Though these graphics will contain images and measurements made from the data, they are separate and must be treated as such. In other words, files generated as graphics should never be used to generate data. Measurements should be made only from primary data files. All of this data are collected using imaging software, and the software available has an enormous impact on how the data are captured, analyzed, and stored. In broad terms, there are two types of software commercially available for digital microscopy. The first is purpose written to drive an individual instrument and is usually provided as part of a “turn-key” imaging system. Often such software is tied to a particular instru- ment manufacturer and can only rarely interface with other instruments. This type of software can range from the basic, picture-taking packages designed for general dig- ital photomicrography use, to highly sophisticated software, capable of analytical and quantitative measurements. The second type of software is designed primarily as an analytical tool and purpose built to work with most imaging platforms. This type of software is often capable of driving numerous devices from a wide range of manufac- turers, such as microscopes, automated stages, cameras or other detectors, illumina- tion and shuttering units, galvanometers, and laser-bleaching and -ablation devices. Because of their ability to interface with multiple instruments, such software packages offer great flexibility, and are used to drive many custom-built imaging systems. The choice of which type of system to choose is often based on capabilities and expertise in