A Short Course on Bioimage Informatics Lecture 1: Basic Concepts and Techniques
Ge Yang (杨戈) Department of Biomedical Engineering & Department of Computational Biology Carnegie Mellon University
August 10, 2015
1 Acknowledgements
• Dr. Chao Tang • Dr. Luhua Lai • Students, faculty, and staff of the Center for Quantitative Biology, Peking University
• Dr. Jing Yuan (袁静) • Youjun Xu (徐优俊)
• 111 Visiting Scholar Program, Ministry of Education • College of Chemistry and Molecular Engineering, Peking University • School of Engineering & School of Computer Science, Carnegie Mellon University
2 Outline
• Background
• What is bioimage informatics?
• Where does it come from?
• Where is it going?
• Basic concepts of image formation and collection
• Overview of bioimage informatics techniques
• Summary
3 • Background
• What is bioimage informatics?
• Where does it come from?
• Where is it going?
• Basic concepts of image formation and collection
• Overview of biological image analysis techniques
• Summary
4 What information can we extract from images?
刘奇
5 More Examples of Biological Images Typical Readouts from Images
• From static images, e.g. immunofluorescence images - Location - Shape/morphology - Spatial distribution: e.g. spatial patterns - Spatial relation: e.g. protein colocalization
• From dynamic images, e.g. live cell time-lapse images - All the readouts above, which now may be time-dependent - In other words, spatiotemporal dynamics/processes
• Computational analysis techniques are required to extract these quantitative readouts from images.
7 Relations of Imaging and Image Analysis with Other Biological Research Tools • None of these readouts can be directly provided by e.g. genetics or biochemistry.
• However, understanding these readouts often requires close integration of computational image analysis with experimental analysis using genetics and/or biochemistry techniques.
Alberts, MBoC, 5/e, 2007
8 Some Practical Questions
• Q1: If I have an image analysis problem to solve, where do I look for information and solutions?
• Q2: What techniques are available? Which technique should I choose? How can I be sure that I am getting the correct results?
• Q3: How can I go from image analysis results to molecular mechanisms?
• We will address these questions in this course.
9 • Background
• What is bioimage informatics?
• Where does it come from?
• Where is it going?
• Basic concepts of image formation and collection
• Overview of biological image analysis techniques
• Summary
10 What is Bioimage Informatics?
• To give a simple definition, bioimage informatics is the computational analysis and understanding of biological images.
• Bioimage informatics emerged as a new technical area primarily over the past ten years. - First meeting held in 2005 at Stanford University.
https://sites.google.com/site/hanchuanpeng/bioimageinformatics_resource
11 • Background
• What is bioimage informatics?
• Where does it come from?
• Where is it going?
• Basic concepts of image formation and collection
• Overview of biological image analysis techniques
• Summary
12 Looking into History: Robert Hooke’s Microscope
• Microscope was invented more than 300 years ago.
1) http://micro.magnet.fsu.edu/index.html Molecular expressions: microscopy world 2) http://www.history-of-the-microscope.org/
Robert Hooke (1635-1703) 13 Evolution of Microscopes
http://www.microscopyu.com/museum/index.html
14 Pioneering Work of Computational Analysis of Biological Images
• Shinya Inoue pioneered the use of video recording devices and digital image processing in microscopy in late 1970s to early 1980s.
• It was found that digital image processing can improve image quality.
• Today, we use the term “computational image analysis” interchangeably with “digital image processing”.
Inoue, S. 1981. Video image processing greatly enhances contrast, quality Shinya Inoue, and speed in polarization based microscopy. J. Cell Biol. 89:346–356. Marine Biology Lab, Woods Hole, MA
15 Development of Image Processing and Computer Vision Techniques
• Early work in digital image processing started in 1960s.
• Computer vision techniques started as an area of robotics and artificial intelligence around the same period of time.
• The work of Shinya Inoue is another example of interdisciplinary science.
16 Automation of Image Collection
http://micro.magnet.fsu.edu/index.html Molecular expressions: microscopy world
17 High-Throughput Microscopy
Pepperkok & Ellenberg, 2006
Multi-well plates
18 Advances in Fluorescence Microscopy
Livet J, Weissman TA, Kang H, et al. Nature 450: 56–62, 2007 http://www.fda.gov/forconsumers/consumerupdates/ucm107783.htm
19 Systems Level Study of Biological Processes
June 26, 2000
• International Human Genome Sequencing Consortium (2001). "Initial sequencing and analysis of the human genome" Nature 409 (6822): 860–921.
• Venter, JC et al. (2001). "The sequence of the human genome" . Science 291 (5507): 1304– 1351.
20 Advances in Computer Vision
Takeo Kanade lab, CMU
21 Advances in Medical Image Analysis
Heimann et al, Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets, IEEE Trans. Medical Imaging, vol. 28, no. 8, 1251-1265, 2009.
22 Summary
• Development of bioimage informatics techniques is driven by the synergy of several factors: - The need for quantitative and systems-level study biological processes - Availability of computational image analysis techniques - Availability of automated image collection - Interdisciplinary collaboration
•Q: If I have an image analysis problem to solve, where do I look for information and solutions? A: Many techniques have been developed specifically for biological image analysis. In addition, areas such as computer vision, medical image analysis are also important sources of techniques. 23 • Background
• What is bioimage informatics?
• Where does it come from?
• Where is it going?
• Basic concepts of image formation and collection
• Overview of biological image analysis techniques
• Summary
24 Important Trends in Fluorescence Microscopy (I): Fluorescence Biosensors
Rho GTPase sensor in mouse embryonic fibroblast, Machacek et al, Nature, 2009
• Spatiotemporal cell dynamics visualized by fluorescence biosensors. Important Trends in Fluorescence Microscopy (II): Super-Resolution Microscopy
Chen et al, IEEE Int. Sym. Biomed Imaging, 2014
26 Important Trends in Fluorescence Microscopy (III): Light-Sheet Microscopy
B.-C. Chen et al, Science, 346: 439, 2014.
27 Summary
• Bioimage informatics techniques are required for understanding complex spatiotemporal dynamics revealed by fluorescence biosensors.
• Bioimage informatics techniques are required for further improve the resolution of super-resolution microscopy techniques.
• Light sheet microscopy will drive the transition of cellular imaging from 2D to 3D. Bioimage informatics techniques are essential for 3D image analysis.
28 • Background
• What is bioimage informatics?
• Where does it come from?
• Where is it going?
• Basic concepts of image formation and collection
• Overview of biological image analysis techniques
• Summary
29 General Comments
• To analyze biological images, it is often necessary to have at least some knowledge about the image formation process.
• It is often feasible to significantly simplify image analysis by optimizing the imaging process. collaboration between image collection and image analysis
• It is critical to choose imaging settings properly. Erroneous setting can make the collected images unusable. Nyquist/Shannon sampling criterion
30 Spectrum of Visible Light
Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm red 620–750 nm
http://en.wikipedia.org/wiki/Visible_spectrum http://science.hq.nasa.gov/kids/imagers/ems/visible.html
31 Microscope as a Linear System
• A light microscope can be considered as a linear system.
http://micro.magnet.fsu.edu/primer/java/imageformation/airydiskformation/index.html
• A linear system is characterized by its impulse response. For a microscope, this is its point spread function (PSF). 32 Airy Disk • Airy (after George Biddell Airy) disk is the diffraction pattern of a point feature under a circular aperture.
• It has the following form
2 2Jx1 y x
J1(x) is a Bessel function of the first kind.
• Detailed derivation is given in Born & Wolf, Principles of Optics, 7th ed., pp. 439-441.
33 Microscope Image Formation
• Microscope image formation can be modeled as a convolution with the PSF.
I x,y O x,y psf x,y F I x,y F O x,y F psf x,y
http://micro.magnet.fsu.edu/primer/java/mtf/airydisksize/index.html
34 Resolution Limit of Light Microscopy
061. • Rayleigh limit D NA
• Sparrow limit
047. D NA
http://www.microscopy.fsu.edu/primer/java/imageformation/rayleighdisks/index.html
35 Nyquist/Shannon Sampling Theorem • The Nyquist-Shannon sampling theorem: The sampling frequency should be at least twice the highest frequency contained in the signal.
• This is true for both spatial and temporal sampling.
• Spatial sampling is determined by the effective pixel size. It should be no larger than one half of the smallest feature size.
• Temporal sampling is determined by frame rate. It should be at least twice as fast as the dynamic biological process to be visualized.
36 • Background
• What is bioimage informatics?
• Where does it come from?
• Where is it going?
• Basic concepts of image formation and collection
• Overview of biological image analysis techniques
• Summary
37 From Biological Images to Biological Knowledge
38 How are image analysis problem solved?
• Define the problem.
• Formulate the problem mathematically, typically in the form of an optimization.
• Find a numerical approach to solve the problem.
• Validation/benchmarking
39 Image Preprocessing: Image Registration
http://www.cs.cmu.edu/~kangli/code/Image_Stabilizer.html
40 Image Preprocessing: Image Filtering • Image filtering for noise suppression
original noise σ=1 added
σ=2 σ=5 σ=10 41 Image Feature Detection: Particle Detection
42 Image Feature Detection: Region Detection
Chen K.-C. et al, 2012 IEEE International Conference on Image Processing (ICIP).
43 Image Feature Tracking: Single Particle Tracking
44 • Background
• What is bioimage informatics?
• Where does it come from?
• Where is it going?
• Basic concepts of image formation and collection
• Overview of biological image analysis techniques
• Summary
45 Summary
• Bioimage informatics techniques are basic tools for quantitative analysis and understanding of spatiotemporal cellular processes.
• As a technological area, bioimage informatics is closely connected with areas such as computer vision, medical image analysis, machine learning, statistical data analysis, etc.
• The core techniques of bioimage informatics are well defined.
• As a technological area, bioimage informatics is undergoing rapid development.
46 Resources
47 Additional References for This Lecture
• Download link for course related information
http://www.andrew.cmu.edu/user/geyang/BI_ShortCourse/
• Additional download links using Baidu cloud storage will be announced by email.
48 Resources: Journals
• IEEE Trans. Image Processing • IEEE Trans. Pattern Analysis & Machine Intelligence (PAMI) • International Journal of Computer Vision (IJCV) • Computer Vision and Image Understanding • Pattern Recognition
• IEEE Trans. Medical Imaging • Medical Image Analysis
49 Resources: Journals (II)
• Journal of Microscopy http://www.wiley.com/bw/journal.asp?ref=0022-2720
• Biophysical Journal http://www.cell.com/biophysj/
• Nature Methods http://www.nature.com/nmeth/index.html
50 Resources: Further Reading
• Textbook - Gonzalez & Woods, Digital image processing, 3rd ed., Prentice Hall, 2007. - Gonzalez & Wood, Digital image processing using MATLAB, Gatesmark Publishing, 2009.
• General references - Szeliski, Computer vision: algorithms and applications, Springer, 2010. - Snyder & Qi, Machine vision, Cambridge University Press, 2004. - Sonka, Hlavac, & Boyle, Image processing, analysis, and machine vision, CL-Engineering, 2007. 51 Thank You! Questions?
52 Scales of Cellular Structures
• Above 200nm, conventional light microscopy.
• Below 200nm, electron microscopy; assay must be fixed in preparation.
53