Dissertation zur Erlangung des Doktorgrades der Technischen Fakultät der Albert-Ludwigs-Universität Freiburg im Breisgau
Total internal reflection microscopy: super-resolution imaging of bacterial dynamics and dark field imaging
Philipp von Olshausen
Dezember 2012
Albert-Ludwigs-Universität Freiburg im Breisgau Technische Fakultät Institut für Mikrosystemtechnik Dekan Prof. Dr. Yiannos Manoli
Referenten Prof. Dr. Alexander Rohrbach Jun.-Prof. Dr. Maximilian Ulbrich
Datum der Disputation 28. Februar 2013
Contents
Abstract 5
Zusammenfassung 7
Publications 9
Abbreviations 11
1. Introduction 13 1.1. High-resolution microscopy ...... 13 1.2. Super-resolution microscopy ...... 14 1.3. Label-free microscopy ...... 16 1.4. The bacterial cytoskeleton ...... 17 1.5. Outline ...... 18
2. Theoretical background 19 2.1. Basic optical concepts ...... 19 2.1.1. Fourier transformation ...... 19 2.1.2. Coherent and incoherent image formation ...... 20 2.1.3. Point spread function and optical transfer function ...... 23 2.1.4. Resolution ...... 24 2.2. Total internal reflection microscopy ...... 24 2.2.1. Total internal reflection ...... 25 2.3. Theory of structured illumination microscopy ...... 29 2.3.1. The moiré effect ...... 29 2.3.2. Illumination patterns in SIM ...... 30 2.3.3. Image formation in SIM ...... 30 2.3.4. Resolution enhancement ...... 32 2.4. Numerical image reconstruction in structured illumination microscopy . . 35 2.4.1. Preprocessing of the raw images ...... 36 2.4.2. Separation of spectral components ...... 37 2.4.3. Recombination of shifted spectral components ...... 39
3. Experimental TIRF-SIM set-up 43 3.1. Optical set-up ...... 44 3.1.1. Technical details of the illumination beam path ...... 47 3.1.2. Excitation patterns ...... 49
1 Contents
3.1.3. Epi-fluorescence ...... 50 3.2. Spatial light modulator ...... 51 3.2.1. Holograms ...... 52 3.2.2. Polarization effects of the SLM ...... 56 3.2.3. Correction of non-flatness ...... 57 3.3. Polarization in the illumination beam path ...... 58 3.3.1. Pellicle ...... 59 3.3.2. Dichroic mirror ...... 60 3.4. Drift ...... 61 3.5. Alignment of the optical set-up ...... 63 3.6. Image acquisition ...... 64 3.6.1. Speed ...... 65 3.7. Imaging results with TIRF-SIM ...... 65 3.7.1. Modulation contrast ...... 66 3.7.2. Polystyrene beads ...... 67 3.7.3. Biological samples ...... 70 3.8. Discussion ...... 74 3.8.1. Alternative experimental realizations ...... 74 3.8.2. Strengths and limits of the presented optical set-up ...... 76 3.8.3. Comparison with competitive techniques ...... 77
4. Dynamics of the actin-like protein MreB in Bacillus subtilis 81 4.1. Biological background ...... 81 4.1.1. MreB ...... 82 4.1.2. Cell wall ...... 83 4.2. Results ...... 84 4.2.1. 3D distribution of MreB ...... 84 4.2.2. Filamentous Structure of MreB ...... 84 4.2.3. Dynamics of MreB ...... 86 4.2.4. Tracks of MreB motion ...... 89 4.2.5. Transport velocities of MreB ...... 90 4.2.6. Stationary filaments in protoplasts ...... 93 4.3. Discussion ...... 94 4.3.1. Overexpression ...... 94 4.3.2. TIRF-SIM as the ideal super-resolution microscopy technique to study MreB in B. subtilis ...... 95 4.3.3. Structure and dynamics of MreB - a comparison with recent pub- lications ...... 96 4.3.4. MreB filaments are transported by coupled molecular motors that likely belong to the cell wall synthesis machinery ...... 97 4.3.5. Reversal and orientational change of the filament transport direction 99 4.3.6. Coupled molecular motors might explain filament length depen- dent transport velocity ...... 99
2 Contents
4.3.7. MreB filaments organize the synthesis of the cell wall ...... 100 4.4. Outlook ...... 103
5. Total internal reflection dark field microscopy 105 5.1. Imaging Theory ...... 107 5.1.1. Coherent scattering of an evanescent wave ...... 107 5.1.2. 2D Coherent optical transfer function ...... 109 5.1.3. Image formation ...... 110 5.1.4. Two-point resolution in coherent imaging ...... 115 5.2. Experimental set-up ...... 117 5.2.1. Imaging modalities ...... 119 5.2.2. Corrections for an imperfect system ...... 120 5.2.3. Effects of image averaging on the background ...... 121 5.3. Results ...... 123 5.3.1. Images of polystyrene beads ...... 123 5.3.2. Images of biological samples ...... 129 5.4. Discussion ...... 132 5.4.1. Imaging theory ...... 132 5.4.2. Experimental set-up ...... 133 5.4.3. Results: TIRDF images of polystyrene beads and biological samples134
6. Conclusions 137
A. Appendix 139 A.1. Circ function ...... 139 A.2. Preparation of agarose pads ...... 139 A.3. Determination of the illumination intensity in the focal plane ...... 139 A.4. Measurement of filament length from 3D data ...... 140 A.5. Length correction of MreB filament lengths ...... 140
Bibliography 143
3
Abstract
High resolution light microscopy is a valuable tool of modern biology as it allows the in- vestigation of living cells. In particular, fluorescence microscopy enables the observation of virtually any cellular component that can be labeled with a fluorescent dye. A great part of this thesis comprises the construction of a super-resolution fluores- cence microscope and its application to living cells. The built microscope combines the advantages of total internal reflection fluorescence (TIRF) and structured illumina- tion microscopy (SIM) and thus yields a twofold increased lateral resolution (≈ 100 nm) compared to the classical Rayleigh limit (≈ 200 nm) at very high contrast. A spatial light modulator is used as diffractive opitcal element, which allows a fast and flexible switching of the illumination patterns resulting in frame rates of about 1 Hz. In particular, TIRF-SIM was used to image the actin-like protein MreB in the bacteria Bacillus subtilis. MreB is part of the bacterial cytoskeleton and its dynamics are tightly linked to cell wall synthesis. TIRF-SIM imaging confirmed that MreB forms filaments which run on straight tracks. Special events like changes of tracks or reversals in direction could be observed. The transport velocity of MreB was found to depend on the filament length. The super-resolved time series obtained by TIRF-SIM enabled the deduction of a mechanistic multi motor model that accounts for the observed phenomena. In this model MreB filaments couple motors that drive the cell wall synthesis which leads to a parallelized and structured growth of new cell wall material. A second part of this thesis comprises the development of a label-free TIR microscopy technique. The sample is illuminated by an evanescent field but instead of fluorescence the light coherently scattered by the object is used for imaging. The illumination light is blocked in the detection path as in dark field (DF) microscopy, which significantly reduces the background. An imaging concept was developed that exploits multiple illumination directions. A theory for the formation of a TIRDF image is introduced and first results show that scattering samples can be imaged with great resolution and high contrast due to the special illumination scheme. This represents a new coherent imaging technique that is label-free and potentially very fast.
5
Zusammenfassung
Hochauflösende Lichtmikroskopie erlaubt die Untersuchung lebender Zellen und ist somit ein wertvolles Werkzeug der modernen Biologie. Insbesondere die Fluoreszenzmikrokopie erlaubt die Beobachtung praktisch jedes zellulären Bestandteils, welches mit einem Farb- stoff makiert werden kann. Ein Großteil dieser Arbeit umfaßt den Aufbau eines überauflösenden Fluoreszenz- mikroskops und dessen Verwendung zur Untersuchung lebender Zellen. Dieses Mikroskop kombiniert die Konzepte der Totalreflektions-Fluoreszenzmikroskopie und der struk- turierten Beleuchtung (TIRF-SIM). Dadurch wird eine verdoppelte laterale Auflösung (≈ 100 nm) im Vergleich zur klassichen Rayleigh Auflösungsgrenze (≈ 200 nm) erreicht bei gleichzeitig hohem Kontrast. Die Verwendung eines spatialen Lichtmodulators als optisches Beugungselement erlaubt ein schnelles und flexibles Schalten der Beleuchtungs- muster. Somit können Bildraten von bis zu 1 Hz erreicht werden. TIRF-SIM wurde insbesondere angewendet um das Aktin-ähnliche Protein MreB in den Bakterien vom Typ Bacillus subtilis zu beobachten. MreB is Teil des Zytoskeletts von Bakterien und die Dynamik von MreB ist eng verbunden mit der Synthese der Zell- wand. Mittels TIRF-SIM konnte bestätigt werden, dass MreB Filamente formt, welche sich auf geraden Trajektorien bewegen. Besondere Ereignisse konnten beobachtet wer- den, wie z.B. Richtungswechel und -änderungen. Desweiteren wurde herausgefunden, dass die Transportgeschwindigkeit von MreB von der Filamentlänge abhängt. Die über- aufgelösten Bildfolgen, welche mit TIRF-SIM aufgenommen wurden, ermöglichten die Konzeption eines mechanischen Modells von gekoppelten Motoren, welches die Beobach- tungen qualitativ erklären kann. In diesem Modell koppeln MreB Filamente mehrere Motoren, welche die Zellwand-Synthese antreiben. Diese Kopplung führt zu einem paral- lelisierten und strukturierten Wachstum von neuem Zellwandmaterial. In einem zweiten Teil dieser Arbeit wurde eine weitere Mikroskopiemethode entwickelt, welche auf Totalreflektion basiert. Hierbei muss die Probe jedoch nicht (fluoreszenz-) markiert werden, sondern das kohärent gestreute Licht aus dem evaneszenten Beleuch- tungsfeld wird detektiert. Diese Detektion wurde kombiniert mit dem Prinzip der Dunkelfeldmikroskopie, d.h. das ungestreute Beleuchtungslicht wird detektionsseitig ge- blockt, was den Hintergrund im Bild erheblich verringert. Es wurde ein Konzept ent- wickelt, welches die Beleuchtung der Probe aus vielen Richtungen ausnutzt. Ebenfalls wurde eine Theorie eingeführt, welche die Bildentstehung beschreibt. Es wurden Auf- nahmen von streuenden Proben gemacht, welche die hohe Auflösung und den hohen Kontrast dieser Mikroskopietechnik beweisen. Diese neue, kohärente Abbildungstechnik bedarf keiner Markierung oder Manipulation der Probe und ermöglicht eine potentiell sehr schnelle Bildaufnahme.
7
Publications
The following publications contain results from this thesis:
F. Spira, N. S. Mueller, G. Beck, P. von Olshausen, J. Beig, and R. Wedlich-Söldner, Patchwork organization of the yeast plasma membrane into numerous coexisting domains, Nature Cell Biology, 14 (2012), pp. 640-648.
Further publications have been submitted:
P. von Olshausen, H. J. D. Soufo, P. L. Graumann, and A. Rohrbach, Super-resolution imaging of dynamic MreB filaments in B. subtilis - a multiple motor driven transport?
M. C. Huber, A. Schreiber, P. von Olshausen, and S. M. Schiller, Synthetic Biology Inside the Cell: Programmed Engineering of Genetically Encoded Artificial Organelles in vivo
9
Abbreviations
2/3D two/three-dimensional AOD acousto optical deflector AOI area of interest AOTF acousto optical tunable filter a.u. arbitrary units B. subtilis Bacillus subtilis BE beam expander BFP back focal plane CCD charge coupled device coh coherent (s)CMOS (scientific) complementary metal-oxide-semiconductor det detection DF dark field DNA deoxyribonucleic acid DO diffraction order E. coli Escherichia coli EM electron microscopy ev evanescent FTIR frustrated total internal reflection FWHM full width at half maximum (E)GFP (enhanced) green fluorescent protein i incident ill illumination inc incoherent LCOS liquid crystal on silicon max maximum min minimum NA numerical aperture OTF optical transfer function PALM photo-activated localization microscopy PBP penicillin binding protein PG peptidoglycan pol polarization PSF point spread function ROI region of interest SIM structured illumination microscopy
11 Contents
SLM spatial light modulator SNR signal-to-noise ratio STED stimulated emission depletion STORM stochastic optical reconstruction microscopy TIRDF total internal reflection dark field TIR(F) total internal reflection (fluorescence)
12 1. Introduction
Understanding the intricate mechanisms regulating living cells is among the major chal- lenges of modern biology and medicine. Light microscopy has become a valuable tool that facilitates the investigation of living cells. One of the most important light microscopy techniques is fluorescence microscopy. The possibility to specifically label almost any cellular component has proven to be enormously helpful as it allows to observe the desired component without distracting signals from other parts of the cell. The discovery of the green fluorescent protein (GFP), honored with the Nobel prize in 2008, together with the advancements in microbiology allow a simple modification of the DNA such that most proteins can be made fluorescent, which gave fluorescence microscopy another boost. Among the vast amount of research carried out on living cells with the help of light microscopy there is one field that is of particular interest to biophysicists: the cytoskele- ton, which is a highly dynamic filament system present in most cells. The cytoskeletal filaments appear on length scales from 10 nm to several micrometers and thus necessitate high resolution microscopy, sometimes even beyond the diffraction limit (≈ 200 nm). This thesis aims at the implementation of a fluorescence microscope that can im- age dynamic biological samples beyond the diffraction limit with a resolution of about 100 nm. This microscopy technique is then applied to image the bacterial cytoskeleton. Also, the development of a new variation of dark field microscopy, label-free and based on coherent scattering, is pursued.
1.1. High-resolution microscopy
A major quality feature of every microscope is its optical resolution. Optical resolution means how well two neighboring features in an image can be separated. Resolution is always tightly linked to the contrast of an image, which defines how much the structure of interest differs in intensity from the background. Both a good resolution and contrast are highly desirable when investigating cells. Electron microscopy (EM) has been shown to achieve resolution down to the molecular or even atomic scale [1]. However, it suffers from practical limitations when it comes to the study of living cells. In order to get reasonable contrast the samples must be stained, usually with heavy salts or metals like, e.g., gold. Furthermore, the samples are mounted in a vacuum chamber. Both staining and a vacuum surrounding are not even close to physiological conditions and are thus not suitable for the study of living cells. The situation is comparable with the decryption of the DNA: to understand the meaning of any genetic sequence not only its structure must be known but also its function. As light
13 1. Introduction microscopy allows to follow the dynamics of cellular components it is an indispensable tool for the deciphering of their function. The investigation of many processes in cells necessitates a high optical resolution, ultimately down to the nanometer scale of single molecules. Though single molecules can already be observed in fluorescence microscopy, the resolution is fundamentally limited by the physics of light. As first discovered by Abbe in 1873 diffraction limits λ the resolution of a normal light microscope to about 2NA [2]. Here, λ is the wavelength of the light and NA = n · sin θ is the numerical aperture of the microscope’s objective, where n is the refractive index of the immersion medium (n ≈ 1 in air) and θ is the maximum angle that can be captured by the objective lens. For high-end objectives with NA ≈ 1.3 and a typical wavelength of λ ≈ 510 nm (GFP) this results in a resolution of about 200 nm.
1.2. Super-resolution microscopy
During the last decades several techniques have been developed to circumvent this res- olution limit. These techniques can be classified into two groups [3]. First, there are localization-based techniques, which base on the detection and precise localization of single fluorophores. Many raw images, each containing a subset of single molecules that can be localized with high precision, have to be acquired so that these techniques are limited in acquisition speed. However, the resolution is theoretically only limited by the signal-to-noise ratio (SNR). Photoactivated localization microscopy (PALM) and stochastic optical read-out microscopy (STORM) follow this approach and a lateral res- olution < 10 nm could be shown in fixed biological samples [4, 5, 6]. The second group of techniques bases on a spatially non-uniform illumination that can additionally be combined with a nonlinear photoresponse. Among these is stimulated emission depletion (STED) microscopy, a point-scanning technique similar to confocal microscopy [7]. The difference to a confocal microscope is that in STED a second doughnut-shaped depletion beam is overlayed to the scanning focus which depletes the excited fluorophores in the periphery of the focus by the process of stimulated emission. As the depletion process is nonlinear the remaining excitation volume can be made arbitrarily small. A lateral resolution of 15 nm to 20 nm has been shown in biological samples [8]. However, no axial super-resolution is achieved and high laser intensities have to be used to deplete the excited fluorophores. Also, due to the point-scanning nature of this technique the image acquisition rate decreases dramatically with increasing sample size. In this thesis another technique of the second group is exploited, which is structured illumination microscopy (SIM). SIM is a widefield technique, where the whole sample is illuminated by a light pattern. Several raw images illuminated by modified light patterns must be acquired. A final image with increased lateral resolution is then reconstructed from these raw images in a numerical postprocessing step. The light pattern is usually a sinusoidal grating, which can by produced by the interference of two plain waves. As long as the fluorescence signal is proportional to the intensity of the exciting light
14 1.2. Super-resolution microscopy pattern, an increase in resolution of about a factor of two can be achieved [9]. Lateral and Axial resolutions of 100 nm and 360 nm, respectively, have been shown using linear SIM [10, 11]. Exploiting additionally non-linear fluorophore responses the resolution can be further increased (< 50 nm) and is then only limited by the SNR [12, 13]. However, more raw images have to be acquired which limits the overall imaging speed and increases the problem of photobleaching. This thesis aims at an implementation of linear SIM to study dynamic biological processes. For this purpose, a set-up is built using a spatial light modulator (SLM). A SLM is a diffractive optical element in form of a pixelated display, where each pixel can be controlled and shift the phase of the reflected light. Displaying different phase distributions (as, e.g., phase gratings in the case of SIM) on the SLM leads to different beam shapes without mechnically moving any parts. This allows fast, precise, and flexible switching of illumination patterns and is thus well suited for SIM.
cell
glass coverslip
objective lens
fluorescence light
Figure 1.1.: Scheme for TIRF-SIM microscopy. TIRF-SIM is a fluorescence technique, where two counter-propagating evanescent waves create a sinusoidal illumination grating that illuminates only parts of the sample (here a cell with fluorescently labeled filaments). From several raw images, illuminated with different gratings, a final image with a twofold increased lateral resolution can be reconstructed.
The combination of SIM with total internal reflection fluorescence (TIRF) microscopy, referred to as TIRF-SIM, is ideally suited to observe events near the coverslip. In TIRF microscopy the illumination light is totally internally reflected at the coverslip-sample interface and only a thin evanescent field penetrates and illuminates the sample [14]. As the penetration depth is typically on the order of λ/4 the axial sectioning is very good. Furthermore, hardly any background signal is obtained which leads to very high contrast. Figure 1.1 shows the scheme for TIRF-SIM, where two evanescent waves create a thin sinusoidal light grid. Thus only parts of the sample are excited and give fluorescent signal. Although TIRF microscopy is limited to imaging near the coverslip, its two dimen- sional nature also limits the number of necessary raw images. In TIRF-SIM only nine raw images with modified illumination patterns are needed to reconstruct a final super- resolved image with a resolution of about 100 nm (≈ˆ λ/4). It is thus a good compromise between acquisition speed and lateral resolution. For this reason, TIRF-SIM was applied to study the dynamics of MreB filaments in living bacteria (see sec. 1.4).
15 1. Introduction
1.3. Label-free microscopy
Although the advantages of fluorescence microscopy for biological investigations are nu- merous, there is still a need to observe cells with label-free techniques. These techniques do not suffer from photobleaching and there is no need to change the sample’s native state by introducing fluorescent dyes. Also, short exposure times are often sufficient due to high signal which allows greater imaging speeds. For this reason the set-up built in this thesis was extended to combine the concept of TIR with dark field (DF) microscopy, referred to as TIRDF.
Label-free microscopy techniques like Zernike phase contrast and differential inter- ference contrast microscopy translate phase changes into modulations of the detected intensity [15]. These techniques are employed in transmission mode which limits the axial sectioning capabilities and also the signal-to-noise ratio (SNR) due to significant background light. Another label-free technique is digital holography, where the light transmitted through the sample is overlayed with an undisturbed reference beam. Sev- eral images with phase-stepped reference beams are acquired which allows the extraction of the phase of the light. From this the differences in optical path length can be cal- culated. Digital holography has also succesfully been combined with TIR by Ash et al. [16, 17]. However, the maximum NA they used was 0.4 which constricts the lateral resolution. A further techniqe for label-free imaging of cells is tomographic phase mi- croscopy as done by Choi et al. [18]. They could reconstruct the 3D distribution of the refractive index in a cell with a lateral and axial resolution of 0.5 µm and 0.75 µm, re- spectively. However, image acquisition speed is limited and takes about 10 s for a volume of 20 µm in diamter.
In dark field microscopy the illumination light is blocked in the detection path and only light scattered by the sample is used for imaging [19]. This eliminates the background light and yields very high contrast. In normal implementations of dark field microscopy white light sources are used and the oblique illumination light penetrates the sample. This impedes the use of detection objectives with very high NA and thus limits the lateral resolution. Furthermore, light scattered in defocused axial planes contributes to the signal and decreases the SNR.
Figure 1.2 illustrates TIRDF as employed in this thesis. The sample is illuminated by an evanescent field which is created by TIR of a spatially coherent light source. The illumination light is blocked on its way back to the imaging detector. Only the light coherently scattered by the sample within the evanescent field is used for imaging. This approach yields excellent axial sectioning and high SNR as no out-of-focus parts of the sample are illuminated. Also, a detection objective with high NA can be used and imaging is potentially fast. This thesis presents how this technique can be used to yield images of high resolution and contrast with minimized coherent artifacts.
16 1.4. The bacterial cytoskeleton
scattered light
sample
glass coverslip
objective lens
block Figure 1.2.: Scheme for TIRDF microscopy. The illumination light gets blocked in the detection path and only the light scattered by objects in the evanescent field is used for imaging.
1.4. The bacterial cytoskeleton
The cytoskeleton of eukaryotic cells consists of three ubiquitous biopolymers made up of protein subunits, namely actin filaments, microtubules and intermediate filaments [20]. These protein structures are highly dynamic and are constantly adapting to environmen- tal influences. They often serve mechanical purposes as they play a key role in cell shape, migration, adhesion, cell division, and serve as tracks for molecular motors [20]. The constant reformation of the cytoskeleton consumes a large part of the energy available to a cell which underlines its importance. In the meantime it is also known that proteins similar to those in eukaryotes form a cytoskeleton in bacteria, which is supposed to be equally important [21]. The proteins ParM and MreB, e.g., have a similar structure to monomeric actin and FtsZ resembles the eukaryotic tubulin. These similarities suggest analog functions of these proteins in procaryotes as their eukaryotic counterparts. Deciphering their function is not simply fundamental research but might help to tackle diseases and design new medicaments against bacterial infections. Applying modern optical microscopy with high temporal and spatial resolution to living bacteria enables the precise observation of cytoskeletal dynamics. Quantitative data about filament distributions, sizes, transport velocities and dynamic changes can be gained which allows for the design of sophisticated biophysical concepts and models. In the case of MreB, e.g., dynamics are tightly linked to the synthesis of the cell wall which is important for the shape and proliferation of bacterial cells like Bacillus subtilis and Escherichia Coli. To understand the mechanics of the growth and the structure of the cell wall the microscopic observation of MreB with high temporal and spatial resolution is of uttermost importance. Thus, the major aim of this thesis is the application of super-resolution microscopy to the living bacteria B. subtilis to better understand the dynamics of the cytoskeletal element MreB.
17 1. Introduction
1.5. Outline
Here, the outline of this thesis is presented.
Chapter 2: Theoretical background The theoretical background necessary to under- stand this thesis is introduced. This covers basic optical concepts like a Fourier-optical description of image formation and the phenomenon of total internal reflection. A the- oretical explanation of 2D linear structured illumination microscopy is also provided. Furthermore, the code used for the numerical reconstruction of the SIM images is sum- marized.
Chapter 3: Experimental TIRF-SIM set-up The experimental set-up that was built during this thesis to do TIRF-SIM imaging is described in great detail. The spatial light modulator (SLM) is presented as the central part of this set-up which controls the illumination pattern. Effects of polarization are treated explicitly due to their strong influence on the illumination grating. A scheme for the compensation of axial drift is described that enables long-term measurements. The alignment of the set-up and the task-flow during image acquisition are explained. Super-resolved images of small polystyrene beads and of biological samples are shown to demonstrate the high-resolution capabilities of TIRF-SIM and to prove its applicability to biological samples. Finally, the experimental set-up as well as the achieved results are discussed which includes a comparison with competitive techniques and an outlook on future improvements and applications.
Chapter 4: Dynamics of the actin-like protein MreB in Bacillus subtilis This chap- ter presents results on the structure and dynamics of the protein MreB in the bacteria Bacillus subtilis that have been obtained with the TIRF-SIM set-up. First, some bio- logical background about the function of MreB as part of the bacterial cytoskeleton is provided. Then the obtained results, mainly time-lapse TIRF-SIM data, are presented. These results are extensively discussed and a multi motor model is introduced for their explanation. Finally, an outlook on promising future experiments is provided.
Chapter 5: Total internal reflection dark field microscopy This chapter presents the combination of TIR microscopy with the dark field (DF) principle. First, an imaging theory is derived, which explains the formation of an image in TIRDF and bases upon the theoretical foundations given in chap. 2. Then the modified experimental set-up is presented. In the results section images of polystyrene beads and of biological samples acquired by TIRDF microscopy are shown. The discussion of this chapter covers the derived theory, the experimental set-up, and the achieved results as well as an outlook an future improvements and experiments.
Chapter 6: Conclusions A summary of the most important achievements of this thesis is given together with some concluding remarks on the great potential of TIR microscopy.
18 2. Theoretical background
This chapter introduces the theoretical concepts that are necessary for the understanding of this thesis. The presentation of these concepts is compact and the reader is referred to publications and textbooks for a more detailed derivation. Section 2.1 covers basic optical concepts and presents fundamental descriptions of the image formation in a microscope which form the basis for structured illumination microscopy (SIM) and total internal reflection dark field microscopy (TIRDF). In sec- tion 2.2 the concept of total internal reflection is introduced and the relevant properties of the evanescent field are explained. Section 2.3 covers the theory of SIM and how lateral resolution enhancement is achieved. The numerical reconstruction that produces the final super-resolved image from the raw images acquired in SIM is explained in the last section 2.4.
2.1. Basic optical concepts
This section introduces a definition of the Fourier transformation and gives a mathemat- ical formulation of coherent and incoherent image formation in terms of Fourier optics. The point spread function (PSF) and the optical transfer function (OTF) are introduced as important characteristics of an imaging system. Also, the Rayleigh criterion for the lateral resolution of an image is explained.
2.1.1. Fourier transformation
The electric field E(x, y) in the front focal plane of a lens is related to the electric field in the back focal plane, Ee(kx, ky), by a two-dimensional Fourier transformation. The forward Fourier transformation used in this thesis is defined as
Ee(kx, ky) = FT {E(x, y)} ZZ = E(x, y)eı(kxx+kyy) dx dy, (2.1) and the corresponding backward (or inverse) Fourier transform is
−1 n o E(x, y) = FT Ee(kx, ky) ZZ 1 −ı(kxx+kyy) = Ee(kx, ky)e dkx dky. (2.2) (2π)2
19 2. Theoretical background
The spatial frequency coordinates kx-ky in the back focal plane are related to the real space coordinates x0-y0 in the same plane by
x0 y0 k = k · and k = k · , (2.3) x 0 f y 0 f
f k 2π where is the focal length of the lens and 0 = λ0 is the vacuum wavenumber with the vacuum wavelength λ0 [22].
2.1.2. Coherent and incoherent image formation
Almost all imaging presented in this thesis is done in TIR-mode, which is inherently two dimensional (2D). However, the description of image formation given here as according to Singer et al. is equally valid in 3D [23]. Effects of magnification are omitted for clarity. The imaging of an object f(r) is possible because electric fields of the illuminating light interact with the object. In the most simple case the object is illuminated by a −ık r single plane wave si(r) = s0(ki)·e i from the direction ki and with amplitude s0(ki), which represents a spatially coherent illumination. The coherent point spread function (PSFcoh) describes how a point source is imaged into the image plane. For a linear shift invariant system the electric field distribution in the image plane is then described by a convolution of the object with the PSFcoh Z 0 0 0 0 E(r) = f(r ) · si(r ) · PSFcoh(r − r ) dr (2.4)
= f(r) · si(r) ⊗ PSFcoh(r), (2.5) where ⊗ denotes the convolution operator. An equivalent description of eq. 2.5 in Fourier space can be given using the convolution theorem [24]
h i Ee(k) = f˜(k) ⊗ s˜i(k) · OTFcoh(k) (2.6)
= f˜(k − ki) · OTFcoh(k). (2.7)
Here, f˜(k) is the Fourier transform of the object f(r) describing the object field spectrum in k-space. Furthermore,s ˜i(k) = δ(k − ki) is a delta function, which leads to a shift of the object spectrum f˜(k) upon convolution for oblique incidence (ki =6 0). The coherent optical transfer function (OTFcoh) is given by the Fourier transform of the coherent point spread function
OTFcoh(k) = FT {PSFcoh(r)} . (2.8)
The OTFcoh describes how the spatial frequencies of the object field spectrum f˜(k) are transmitted through the optical system (see sec. 2.1.3). In the following sections it will be considered that typical imaging devices measure intensities and not electric fields.
20 2.1. Basic optical concepts
Spatially coherent image formation
For the formation of a spatially coherent image two things must be considered. First, the illuminating light source must be spatially coherent. In microscopy this is usually achieved by either closing the Köhler aperture or by using a laser and illuminating the sample with a single, spatially coherent plane wave. Second, every imaging device records intensities. In the most simple approximation the intensity is given by the square of the absolute value of the electric field. Using eqs. 2.4 and 2.5 the coherent image formation can be described in real space by
I(r) = |E(r)|2 Z 2 0 0 0 0 = f(r ) · si(r ) · PSF (r − r ) dr coh 2 = |(f(r) · si(r)) ⊗ PSFcoh(r)| (2.9)
Thus, the coherent image is linear in terms of the object field. In Fourier space the image intensity spectrum, I˜(k), can be expressed using eq. 2.6 and applying the autocorrelation theorem [24]
nh i o I˜(k) = AC f˜(k) ⊗ s˜i(k) · OTFcoh(k) , (2.10) where AC denotes the autocorrelation. The corresponding description of the real space image, calculated in Fourier space, is
n o I(r) = FT −1 I˜(k) −1 n nh i oo = FT AC f˜(k) ⊗ s˜i(k) · OTFcoh(k) n o 2 −1 ˜ = FT f(k − ki) · OTFcoh , (2.11) where again the autocorrelation theorem was applied. The object field spectrum gets shifted by the illuminating field vector and then filtered by the OTFcoh of the system. The absolute value of the filtered, backward Fourier transformed field spectrum must then be squared to yield the intensity image.
Spatially incoherent image formation
In incoherent imaging an extended light source is assumed that illuminates the object from many directions. Integrating over a large number of different illumination direc- tions ki renders the image spatially incoherent. The formation of an incoherent image is of special importance in this work as it also describes a fluorescence image. The spa- tially uncorrelated emission of light by many fluorophores is equivalent to an incoherent
21 2. Theoretical background illumination from many directions. Using eq. 2.4 the final intensity image is given by
ZZ 2 0 0 0 0 I(r) = f(r ) · si(r ) · PSF (r − r ) dr dki coh ZZ 0 0 = f(r ) · PSFcoh(r − r )· Z ? 00 ? 00 0 ? 00 0 00 f (r ) · PSFcoh(r − r ) · si(r ) · si (r ) dki dr dr , (2.12) | {z } eq. 2.13
? where denotes complex conjugation. Assuming s0(ki) = 1 for all illumination directions −ık r and using si(r) = s0(ki) · e i the underbraced integral becomes a delta function
Z 0 00 ? −ıki(r −r ) 0 00 s0(ki) · s0(ki) · e dki = δ(r − r ). (2.13)
Consequently, eq. 2.12 can be simplified to
2 2 I(r) = |f(r)| ⊗ |PSFcoh| (2.14) 2 = |f(r)| ⊗ PSFinc, (2.15) where the incoherent point spread function is given by
2 PSFinc = |PSFcoh| . (2.16)
The incoherent image is linear in terms of the intensity of the object. Fourier transforming eqs. 2.14 and 2.15 and applying the convolution theorem and the autocorrelation theorem [24] yields a description of the image intensity spectrum n o I˜(k) = AC f˜(k) · AC {OTFcoh} n o = AC f˜(k) · OTFinc, (2.17) where the incoherent OTFinc can be expressed as the autocorrelation of the OTFcoh
OTFinc = AC {OTFcoh} . (2.18)
The corresponding description of the real space image, calculated in Fourier space, is n o I(r) = FT −1 I˜(k) −1 n n o o = FT AC f˜(k) · OTFinc . (2.19)
Thus, the transmitted spatial frequency components that contribute to an incoherent n o image are calculated by multiplying the object’s intensity spectrum, AC f˜(k) , with the incoherent optical transfer function, OTFinc.
22 2.1. Basic optical concepts
2.1.3. Point spread function and optical transfer function The point spread function (PSF) and the optical transfer function (OTF), as introduced in the previous section, are important parameters characterizing any imaging system. The PSF represents the image of a point source. The OTF, the Fourier transform of the PSF, describes the transmission of spatial frequencies through the imaging system. These properties strongly influence resolution and contrast in the final image. As this thesis covers microscopy techniques in the TIR-mode, a two-dimensional description of the PSF and OTF in the x-y- and kx-ky-plane, respectively, is sufficient. Furthermore, rotational symmetry around the optical axis is assumed. Prefactors have been largely omitted, instead the functions are normalized so that their maximum value is 1. The factor that defines the size and shape of both the PSF and the OTF is the numerical aperture (NA) of the imaging system. The NA is defined as
NA = n · sin(θmax), (2.20) where n is the refractive index of the immersion medium and θmax is the maximal angle of the light that is collected by the imaging system.
Coherent PSF and OTF
The coherent optical transfer function, OTFcoh, is given by the circular aperture of the imaging system in Fourier space. In kx-ky-coordinates it is given by ! k⊥ OTFcoh(k⊥) = circ , (2.21) k0 · NA
q 2 2 where k⊥ = kx + ky and k0 · NA is the radius of the aperture [25]. The definition of the circ-function is given in the appendix (A.1). A radial line profile of eq. 2.21 is shown in red in fig. 2.1-b. From eqs. 2.21 and 2.8 if follows that the coherent PSFcoh is 2 · J1 r⊥ · k0NA PSFcoh(r⊥) = , (2.22) r⊥ · k0NA
p 2 2 where J1 denotes the first order Bessel function and r⊥ = x + y is the radial coordinate in the x-y-plane [26]. Figure 2.1-a shows a radial line profile of the coherent PSF in red.
Incoherent PSF and OTF
According to eq. 2.18 the incoherent OTFinc is given by the autocorrelation of the coherent OTFcoh, which yields [27] v ! u !2 2 k⊥ k⊥ u k⊥ t OTFinc(k⊥) = arccos − 1 − . (2.23) π 2k0NA 2k0NA 2k0NA
23 2. Theoretical background
I (a.u.) I (a.u.) a) b) 1.0 PSFcoh 1.0 OTFcoh PSFinc OTFinc
(λ/NA) r k (k0 NA) -2 -1 1 2 -2 -1 1 2
Figure 2.1.: Radial line profiles of the coherent and incoherent PSF (a) and OTF (b).
It is a tent-like function that drops to zero at k⊥ = 2k0NA and is shown in blue in fig. 2.1-b. Combining eqs. 2.22 and 2.16 yields the incoherent PSFinc, 2 2 · J r · k NA 1 ⊥ 0 PSFinc(r⊥) = . (2.24) r⊥ · k0NA Figure 2.1-a shows a radial line profile of the incoherent PSF in blue. A characteristic measure of the incoherent PSF that will be used throughout this thesis is its lateral full width at half maximum (FWHM) given by
λ0 ∆r = 0.51 · (2.25) ⊥,F W HM NA which depends on the vacuum wavelength of the light, λ0, and the NA of the imaging system.
2.1.4. Resolution The incoherent lateral resolution of an optical microscope can be defined by the Rayleigh criterion. It says that two point sources are resolvable in an image when their distance is not smaller than dR given by
λ0 d = 0.61 · . (2.26) R NA This is exactly the distance at which the maximum of one point image is at the position of the first minimum of its neighboring point image. Figure 2.2 illustrates the Rayleigh criterion graphically. The dip in intensity between the two maxima of the point images is ∆I = 0.26.
2.2. Total internal reflection microscopy
This section presents the process of total internal reflection and introduces the properties of its associated evanescent field. For the microscopy techniques presented in this thesis the evanescent field constitutes the illumination.
24 2.2. Total internal reflection microscopy
I
PSFinc sum ΔI
r λ NA d NA R λ
Figure 2.2.: Rayleigh criterion for the lateral resolution of an incoherent image. The image of two neighboring point sources is the sum of the PSFs of the imaging system. According to the Rayleigh criterion two neighboring point images are resolvable if their λ0 distance is at least dR = 0.61 · NA .
2.2.1. Total internal reflection Whenever a light wave encounters the interface of two media with different refractive indices it undergoes refraction according to Snell’s law [28]. Let ni and nt be the re- fractive index of the incident and transmitting medium, respectively. If a plane wave hits the interface under an angle greater than the critical angle θc it will undergo total internal reflection (TIR). It occurs when the exit angle in the transmitting medium, θt, becomes 90◦. Using Snell’s law the critical angle is then described by nt θc = arcsin (2.27) ni Due to the boundary conditions set by Maxwell’s equations a thin evanescent field will form in the transmitting medium right at the interface [28]. The situation is illustrated in fig. 2.3, where a plane wave with wave vector ki undergoes TIR and the parallel wave fronts of the evanescent field are drawn in the transmitting medium. The following explanations are restricted to the x-z-plane as shown in fig. 2.3 but are equally applicable to three dimensions. The components of the wave vector of the evanescent field depend on the incident wave vector ki and the indices of refraction of the two media. From Maxwell’s equations it can be deduced that the tangential component of the electric field at an interface is conserved [29]. Thus, the tangential component of the evanescent field vector, ktx, is given by
ktx = kix
= k0 · ni · sin(θi), (2.28) where θi is the angle of incidence as shown in fig. 2.3. Equation 2.28 defines the wave- length of the evanescent wave, λev, traveling along the interface by
λ0 λev = . (2.29) ni · sin(θi)
25 2. Theoretical background
z z
λev nkt ktz d I(z) tx pd ni x
I0/e I0
kix
θc kiz kr ki θi
Figure 2.3.: Scheme for total internal reflection (TIR) at a plane interface. A plane wave incident under the angle θi > θc (critical angle for TIR) with wave vector ki is totally internally reflected at a plane interface with refractive indices ni > nt. All intensity is reflected (kr) but an evanescent field penetrates into the transmitting medium. To the left the axial decay of the intensity is shown. See main text for details
The axial component of the wave vector of the evanescent field, ktz, can be deduced from 2 2 2 q 2 2 the condition kt = ktx + ktz which results in ktz = ± kt − ktx and thus
q 2 2 2 ktz = −ık0 ni · sin (θi) − nt . (2.30)
Here, kt = nt · k0 is the wave vector in the transmitting medium. Equation 2.30 reveals that ktz is imaginary for θi > θc. Consequently, the evanescent electric field is described by −ktz·z −ıktx·x E(x, z ≥ 0) = E0 · e · e . (2.31) This field travels along the interface and decays exponentially in axial direction. From equation 2.31 it follows that the intensity does also decay exponentially in axial direction which can be expressed by
z − d I(z ≥ 0) = I0 · e pd (2.32) with the penetration depth
λ0 dpd = . (2.33) q 2 2 2 4π ni · sin (θi) − nt
At a distance z = dpd from the interface the intensity has dropped to I0/e. The axial decay of the evanescent intensity is illustrated in the left part of fig. 2.3 .
Polarization dependent field components of the evanescent wave
In order to also consider effects of polarization all three dimensions must be taken into account. According to fig. 2.3 a P-polarized incident wave has electric field components in the x-z-plane, whereas S-polarization corresponds to an electric field in the y-direction
26 2.2. Total internal reflection microscopy
(the third dimension that is not shown in fig. 2.3). Assuming an incident wave in the x-z- plane with electric field amplitudes Ei0P and Ei0S for P- and S-polarization, respectively, the electric field components of the evanescent field are, according to Axelrod et al. [30], given by
" 2 2 1/2 # 2 cos(θi)(sin (θi) − n ) E ti · E · e−ı(δP +π/2) x = 4 2 2 2 1/2 i0P (2.34) (nti cos (θi) + sin (θi) − nti) " # 2 cos(θi) E · E · e−ıδS y = 2 1/2 i0S (2.35) (1 − nti) " # 2 cos(θi) sin(θi) E · E · e−ıδP , z = 4 2 2 2 1/2 i0P (2.36) (nti cos (θi) + sin (θi) − nti) where
" 2 2 1/2 # −1 (sin (θi) − nti) δP ≡ tan 2 (2.37) nti cos(θi) " 2 2 1/2 # −1 (sin (θi) − nti) δS ≡ tan (2.38) cos(θi)
n nt E 6 and ti = ni . Remarkably, the evanescent field has a component x = 0 for the case of P-polarization. In this case, the evanescent wave is not a transversal wave but has a field component along its direction of propagation (eq. 2.34). The P-polarized field “cartwheels” along the surface. For the case of S-polarization the orientation of the incident electric field is maintained in the evanescent field. This polarization-dependent behavior of the evanescent field has two important con- sequences. First, the intensity at the interface, I0, depends on the polarization and is, according to Axelrod et al. [30], given by
2 2 2 2 4 cos (θi) · (2 sin (θi) − nti) I0P = |Ei0P | · 4 2 2 2 (2.39) nti cos (θi) + sin (θi) − nti 2 2 4 cos (θi) I0S = |Ei0S| · 2 (2.40) 1 − nti for P- and S-polarization, respectively. According to eqs. 2.39 and 2.40 there is a field enhancement for θi > θc. For θi being only slightly larger than the critical angle the evanescent intensity can become several times greater than the incident intensity. For θi approaching 90◦ the intensity drops towards zero for both polarizations. In this thesis ◦ S-polarization and an angle of incidence θi ≈ 68 were used which, according to eq. 2.40, 2 yields I0S = 2.4 · |Ei0S| . The second important consequence of the polarization-dependence of the evanescent field is that two counter-propagating evanescent waves can only interfere with full mod- ulation contrast if they are S-polarized. This is illustrated in fig. 2.4, where one di- mensional line profiles of the intensity at the interface (z = 0) of the resulting standing
27 2. Theoretical background
a) b) IS(x) IP(x)
IP0 IS0
x (nm) x (nm) -400 -200 0 400200 -400 -200 0 400200 Figure 2.4.: One dimensional line profiles of the polarization-dependent contrast of a standing evanescent wave. The interference pattern is assumed to be formed by two counter-propagating waves along the x-axis. (a) Intensity profile for the case of S- polarization. The resulting modulation contrast is Cmod = 1. (b) Intensity profile for the case of P-polarization, where the modulation contrast is reduced to Cmod = 0.89 (with the chosen parameters as given in the text). Also, in the case of P-Polarization the intensity is slightly higher than in the S-polarized case, IP 0 > IS0. waves are plotted for S- and P-polarization. The intensity of the evanescent standing wave is calculated according to eq. 2.43 but here the vectorial character of the electric field was considered. Subfigure (a) shows the case of S-polarization, where the modula- C Imax−Imin C tion contrast, calculated as mod = Imax , is mod = 1. P-polarization only yields Cmod = 0.89, as shown in subfigure (b). Also, the maximum intensity is higher in the case of P-polarization according to eqs. 2.39 and 2.40. Here, the parameters were chosen ◦ to be θi = 68 , nti = 1.33/1.52 and λ0 = 488 nm. The polarization-dependent contrast must be considered when doing TIRF-SIM as will be shown in the next chapter 3.
Frustrated total internal reflection
A special situation relevant to TIR microscopy comes up when an object with refractive index nt2 gets into the evanescent field at a distance ≤ dpd. Assuming the most simple case, this object consists of a flat surface that is parallel to the interface, where TIR occurs. In order to describe the behavior of the evanescent light at the new interface nt-nt2 the axial component of the wave vector in the object medium, kt2z, must be evaluated. As the tangential component of the wave vector is conserved the boundary 2 2 2 condition is kt2 = kt2z + kix, equivalent as for eq. 2.30. The axial component of the wave vector is then given by
q 2 2 kt2z = (k0nt2) − (k0ni sin(θi)) . (2.41)
From eq. 2.41 it results that kt2z becomes imaginary if nt2 < ni sin(θi), which means that the field in the object with nt2z is also evanescent. If, on the other hand, nt2 > ni sin(θi) then the field in the object becomes propagating. This latter effect is called frustrated total internal reflection.
28 2.3. Theory of structured illumination microscopy
Conclusions for TIR microscopy
Summarizing the effects of TIR at a given interface with ni and nt the following conclu- sions are important for TIR microscopy as presented in this thesis:
• A lower angle of incidence θi leads to a higher penetration depth of the evanescent field, a stronger field enhancement and frustrated TIR is more likely to occur. Accordingly, a higher angle of incidence leads to a smaller penetration depth, weaker field enhancement and FTIR becomes less likely.
• FTIR is generally more likely to occur the greater the index of refraction nt2 of the object intruding into the evanescent field.
• For proper interference of two counter-propagating evanescent waves these waves must be S-polarized.
2.3. Theory of structured illumination microscopy
This section covers the theoretical concepts behind structured illumination microscopy (SIM). First, an illustrative description of SIM in real space is given by introducing the moiré effect (sec. 2.3.1). Second, the creation of a sinusoidal illumination pattern by the interference of two plane waves is briefly explained in sec. 2.3.2. Then the formation of a two-dimensional fluorescence image with a sinusoidal illumination is described in sec. 2.3.3. In sec. 2.3.4 it is explained how this leads to an increase in lateral resolution. As SIM was done in TIRF-mode in this thesis, all descriptions are constrained to two dimensions (x-y-plane). However, the concept of SIM is also applicable in 3D which then leads to an increase in resolution also in axial direction [31]. Furthermore, only the case of linear SIM is explained, which assumes that the fluorescence intensity is proportional to the exciting light intensity and the density of fluorophores. For the case of nonlinear SIM it is referred to the work by Heintzmann [32].
2.3.1. The moiré effect A descriptive illustration of why SIM can enhance the lateral resolution is given by the moiré effect in real space which is illustrated in fig. 2.5. Subfigure (a) shows an object that contains high spatial frequencies. Let’s assume that these frequencies can not be resolved by the imaging system. When this object is illuminated by a periodic pattern (b) this results in an overlay image which contains lower spatial frequencies, as shown in (c). These lower frequencies are the result of frequency mixing and encode the higher frequency components which have been downsampled into the transmission passband of the imaging system. Thus, the observed image contains information about high frequency components which are not accessible under normal illumination. Knowing precisely the regular illumination pattern, the high frequency components of the object can in principle be extracted. These higher frequencies correspond to a higher spatial resolution.
29 2. Theoretical background
a) b) c)
Figure 2.5.: Structured illumination in real space: the moiré effect. If an object contain- ing high frequencies (a) is illuminated by a periodic pattern (b) the resulting image (c) contains low spatial frequencies encoding the higher ones. A microscope image contains only the lower spatial frequencies due to the limited support of the optical transfer func- tion. Knowing the illumination pattern, the object’s high spatial frequency components can in principle be extracted from the image.
2.3.2. Illumination patterns in SIM The illumination patterns in SIM are usually created by the interference of two plane waves (or three in 3D-SIM [31]). Interference of two plane waves leads to a sinusoidal pattern. In TIRF-SIM two counter-propagating evanescent waves are used to create the illumination pattern. According to eq. 2.31 the two evanescent waves are described by
−ıkevr⊥+ϕ01 Eev1(r⊥, z = 0) = E01(r⊥) · e (2.42) ıkevr⊥+ϕ02 Eev2(r⊥, z = 0) = E02(r⊥) · e , ! x where k has been replaced by ±k which lies in the plane of r = and ϕ are tx ev ⊥ y 01/2 constant phase offsets. S-polarization must be assured so that these evanescent waves can interfere with maximum contrast (see sec. 2.2.1). The intensity of the interference pattern, Iev, is then given by 2 Iev(r⊥) = |Eev1(r⊥) + Eev2(r⊥)| 2 2 = |E01(r⊥)| + |E02(r⊥)| + 2E01(r⊥)E02(r⊥) · cos(2kevr⊥ + ϕ0), (2.43) where ϕ0 = ϕ01 + ϕ02 is the global phase of this cosine grating. Alternatively, Iev(r⊥) can be written as
Iev(r⊥) = I0(r⊥) (1 + Cmod(r⊥) · cos(2kevr⊥ + ϕ0)) , (2.44)
2 2 2E01(r⊥)E02(r⊥) where I0(r⊥) = |E01(r⊥)| + |E02(r⊥)| and Cmod(r⊥) = is the modula- I0(r⊥) tion contrast of the interference pattern.
2.3.3. Image formation in SIM For the description of the image formation eq. 2.44 is rewritten to introduce a more n convenient expression for the intensity of the illumination grating Gd (r⊥), n Gd (r⊥) = 1 + Cmod · cos(kg,d · r⊥ + n · ∆ϕ + ϕ0). (2.45)
30 2.3. Theory of structured illumination microscopy
Here, n · ∆ϕ with n = −1, 0, 1 represent the phase steps of the illumination pattern and the index d refers to the direction of the grating. The grating vector kg,d = 2kev determines the period and orientation of the illumination pattern. The illumination grating has been normalized by the intensity I0(r⊥) and it is furthermore assumed that Cmod does not depend on r⊥. The Fourier transform of eq. 2.45 is given by
n ı(n·∆ϕ+ϕ0) Ged (k⊥) = 2π · δ(k⊥)+πCmod · δ(k⊥ + kg,d) · e +
−ı(n·∆ϕ+ϕ0) πCmod · δ(k⊥ − kg,d) · e ). (2.46)
In the case of fluorescence microscopy the object f(r⊥) represents the distribution of fluorophores. As already mentioned, it is assumed that the measured intensity is pro- portional to the excitation intensity and the fluorophore density. This means that f(r⊥) represents the intensity object which must be multiplied by the illumination grating. According to eq. 2.15 the image is then given by
n n Imd (r⊥) = (f(r⊥) · Gd (r⊥)) ⊗ PSFinc(r⊥) (2.47)
A more illustrative understanding of SIM is obtained in Fourier space. The intensity spectrum of the image, Img(k⊥), is then obtained by the Fourier transform of eq. 2.47
n n Imgd (k⊥) = f˜(k⊥) ⊗ Ged (k⊥) · OTFinc(k⊥), (2.48) where f˜(k⊥) is the intensity object spectrum and the convolution theorem was applied. Plugging in eq. 2.46 for the illumination grating yields
n ı(n·∆ϕ+ϕ0) Imgd (k⊥) = 2π · δ(k⊥) + πCmod · δ(k⊥ + kg,d) · e +
−ı(n·∆ϕ+ϕ0) πCmod · δ(k⊥ − kg,d) · e ⊗ f˜(k⊥) · OTFinc(k⊥) (2.49)
Performing the convolution leads to three copies of the object spectrum f˜(k⊥) at the positions of the delta-functions and eq. 2.49 simplifies to
n C mod ın·∆ϕ ıϕ0 Img (k⊥) = 2π · f˜(k⊥) + · f˜(k⊥ + kg,d) · e · e d 2 C mod −ın·∆ϕ −ıϕ0 + · f˜(k⊥ − kg,d) · e · e · OTFinc(k⊥). (2.50) 2
Figure 2.6 illustrates the effects of a structured illumination in real space and in k- space with one dimensional line profiles along the grating direction d. To the left, a box-like object, a cosinusoidal illumination structure and the product of these two are shown in red from top to bottom. Upon imaging the object structure is convolved with the PSF of the imaging system which blurs the image but maintains a certain degree of modulation that comes from the illumination (not shown). To the right of fig. 2.6, the absolute values of the corresponding Fourier transforms are plotted in blue. The
31 2. Theoretical background
f(r ) f(k )
OTF object inc FT r k
-kg,d kg,d n n Gd(r ) Gd(k )
illumination FT r k
-kg,d kg,d n f(k ) n f(r ) Gd(r ) Gd(k )
illuminated object FT r k
-kg,d kg,d
Figure 2.6.: One dimensional illustration of how a structured illumination shifts high frequency information of an imaged object into the transmission passband of the OTF. To the left, a box-like object, a cosinusoidal illumination structure and the product of these two are shown in red from top to bottom. To the right, the absolute values of the corresponding Fourier transforms are plotted in blue. The incoherent OTF of the imaging system is shown in gray. The spectrum of the structurally illuminated object contains high spatial frequency components which have been shifted into the transmitting band of the OTF. incoherent OTF of the imaging system is shown in gray. According to eq. 2.48 in Fourier space the object spectrum must be convolved with the illumination spectrum, which consists of three delta functions. This leads to two copies of the object spectrum at the position of the grating vector ±kg,d, plus the central copy. From the two shifted copies high frequency spectral components are now in the transmitting passband of the OTF that are usually outside. This high frequency information corresponds to a higher spatial resolution and can be extracted as shown in secs. 2.3.4 and 2.4.
2.3.4. Resolution enhancement
As shown in fig. 2.6 and by eq. 2.50 the cosinusoidal illumination leads to two extra copies of the object spectrum which have high spatial frequency components within the OTF support. In order to gain lateral resolution, these high frequency components must be extracted and shifted to their proper position in Fourier space. As there are three spectral copies that have to be separated, three images with differently modulated illuminations are required to solve the equation system. This is achieved by shifting
32 2.3. Theory of structured illumination microscopy
2π the phase of the illumination pattern. Phase shifts are chosen to be 0 and ±∆ϕ = 3 , corresponding to n = −1, 0, 1, so that the sample gets homogeneously illuminated over one grating orientation. Applying three phases for one grating direction d to get three raw images the equations according to eq. 2.50 can be written in matrix notation
Im (k ) f d ⊥ M f˜ (k ) z }| { c d ⊥ −1 z }| { z }| { Im k Cmod −ı·∆ϕ Cmod ı·∆ϕ ˜ gd ( ⊥) 1 2 · e 2 · e f(k⊥) 0 C C mod mod · ˜ ıϕ0,d · Imgd(k⊥) = 1 2 2 f(k⊥ + kg,d) · e 1 C ı·∆ϕ C −ı·∆ϕ −ıϕ 1 mod · e mod · e f˜(k⊥ − kg,d) · e 0,d Imgd(k⊥) 2 2
2π · OTFinc(k⊥), (2.51)
n where Imgd (k⊥) are the Fourier transforms of the acquired raw images. Using the com- pact vector notation eq. 2.51 breaks down to
Img d(k⊥) = Mc · f˜d(k⊥) · 2π · OTFinc(k⊥) (2.52) with the mixing matrix Mc, the image vector Img d(k⊥) and the object spectrum vector f˜d(k⊥) that also contains the global phase ϕ0,d for the corresponding direction. Inverting Mc yields the separation matrix Mc−1, which allows to separate the central copy and the two shifted copies of the object spectrum
−1 Mc · Img d(k⊥) = f˜d(k⊥) · OTFinc(k⊥) · 2π. (2.53)
The three separated spectral components, described by the right side of eq. 2.53, are in the following abbreviated by
ı·m·ϕ0,d Sem,d(k⊥) = f˜(k⊥ − m · kg,d) · e · OTFinc(k⊥), (2.54) where m = −1, 0, 1 is the order of the component. The constant factor 2π is omitted for clarity. The three spectral components Sem,d(k⊥) contain different parts of spectral information that partly overlaps. Correcting for the global phase ϕ0,d and putting the three frequency components to their proper places in k-space yields an effectively enlarged image spectrum along the direction of the grating. For an isotropic increase in resolution in the x-y-plane the above procedure is repeated three times along three symmetrically distributed grating π ◦ directions so that φ = d · 3 ˆ=d · 60 with d = −1, 0, 1, where φ is in the x-y-plane [33]. This yields an effectively enlarged image spectrum,
1 1 X X −ı·m·ϕ0,d Imgeff (k⊥) = Sem,d(k⊥) · e d=−1 m=−1 1 1 X X = f˜(k⊥ − m · kg,d) · OTFinc(k⊥) (2.55) d=−1 m=−1
33 2. Theoretical background
n f(k ) Gd(k ) a)
3 x with n = -1, 0, 1
k
-kg,d kg,d
b) S-1,d(k ) S0,d(k ) S1,d(k )
k k k
-kg,d kg,d -kg,d kg,d -kg,d kg,d
c) Imeff(k )
OTFeff
k
-2kg,d -kg,d kg,d 2kg,d
Figure 2.7.: One dimensional scheme along one grating direction d that illustrates the enhancement of lateral resolution in Fourier space. The OTF of the imaging system is shown in gray. (a) Three raw images with different phases n provide three image spectra. Each of these image spectra contains three copies of the object spectrum as shown in fig. 2.6. The spectral parts shown in light blue are not contained in the raw images. (b) Application of the separation matrix Mc−1 separates the three spectral copies. (c) Putting these copies to their true positions in Fourier space yields an effectively enlarged image spectrum Imf eff (k⊥) and consequently an image with increased lateral resolution. The effective OTFeff is shown in gray and indicates the enlarged support.
Alternatively, an effective OTFeff can be defined
1 1 X X Imgeff (k⊥) = f˜(k⊥) OTFinc(k⊥ + m · kg,d) d=−1 m=−1
= f˜(k⊥) · OTFeff (k⊥) (2.56)
The final super-resolved image is then obtained by an inverse Fourier transformation of the enlarged image spectrum
−1 n o Imfinal = FT Imgeff (k⊥) (2.57)
Figure 2.7 illustrates the reconstruction process and the corresponding enhancement of resolution in a one dimensional scheme along one grating direction d in Fourier space.
34 2.4. Numerical image reconstruction in structured illumination microscopy
The OTF of the imaging system is shown in gray. Three raw images with different phases ϕ = n · ∆ϕ + ϕ0 of the illumination grating provide three image spectra as indicated in subfigure (a). Each of these image spectra contains three copies of the object spectrum as explained in fig. 2.6. Applying the separation matrix Mc−1 according to eq. 2.53 separates the three spectral copies (b). Putting these copies to their true positions in Fourier space, as illustrated in subfigure (c), yields an effectively enlarged image spectrum Img(k⊥) and consequently an image with increased lateral resolution. The effective OTFeff , as defined in eq. 2.56, is shown in gray and indicates the enlarged support. In TIRF-SIM the grating vector kg lies right outside of the OTF support. Thus, one half of the shifted copies of the spectrum is also contained in the central part of the spectrum, whereas the other half is new, additional information at higher frequencies outside of the normal OTF support. Putting these copies to their true positions in Fourier space enlarges the effective OTF support by a factor of two along the applied grating orientation. Consequently, the expected increase in lateral resolution in the final real space image is also a factor of two.
2.4. Numerical image reconstruction in structured illumination microscopy
The previous section explained theoretically how SIM leads to an increase in resolution. In practice the reconstruction must be performed numerically in a postprocessing step. Some parameters like, e.g., the global phase ϕ0 and the grating vector kg,d are not known or only known with insufficient precision so that they have to be determined during the postprocessing. This necessitates an elaborate reconstruction code. The reconstruction code used in this thesis was kindly provided by Rainer Heintzmann from the Institute of Photonic Technology in Jena. It is a Matlab ( c The Mathworks) code mainly written by Rainer Heintzmann, Kai Wicker and Ondrej Mandula. This section aims at giving an overview and a principle understanding of most of the steps performed during the reconstruction. The explanations given here base fundamentally on the PhD thesis by Kai Wicker and the Master thesis by Ondrej Mandula [34, 35]. A brief summary of their code is described by Hirvonen et al. [36]. A quite detailed description of the numerical image reconstruction is also given by Gustafsson et al. [31]. The explanations of the image reconstruction in this section refer to how the code was usually employed in the frame of this thesis. Additional, optional capabilities of the code that may sometimes be relevant are briefly mentioned. A mathematically strict derivation of the applied numerical steps is not provided but is rather found in the above mentioned literature. With the TIRF-SIM set-up presented in this thesis (chap. 3) nine raw images were acquired to reconstruct a super-resolved image of one time point. Three orientations π π 2π 2π (φ = − 3 , 0, 3 ) with three phases each (∆ϕ = − 3 , 0, 3 ) made up the raw data, which is referred to as one set of raw images. The reconstruction code uses one such set of digitally acquired raw images as input, extracts the high frequency information and pops
35 2. Theoretical background out a final image. In order to do so some input parameters must be provided by the user: • estimate of the grating period and orientations • number and magnitude of phase steps for each grating orientation • pixel size of the raw images • experimentally measured 2D PSF; alternatively the PSF/OTF is calculated from the numerical aperture (NA) of the objective lens, the index of refraction of the immersion medium and the emission wavelength of the fluorophores This section is subdivided into three parts, similar to how the code is structured. In the first section 2.4.1 the preprocessing of the raw images is explained. Section 2.4.2 describes the separation of the various spectral components which is done independently for each grating orientation. In the last section 2.4.3 the recombination of the shifted frequency components is described. This includes a weighted average that accounts, e.g., for effects of the OTF. Also, a deconvolution is done before the final output image is generated.
2.4.1. Preprocessing of the raw images All raw images are preprocessed to correct for experimental imperfections and then Fourier transformed. Also, the OTF of the imaging system is calculated as it is needed in the following steps.
Backround subtraction Every raw image is background corrected by subtracting a dark image to get rid of the camera offset. The dark image is an average of ≥ 10 images that were acquired with the illumination laser switched off and the same exposure time as the actual data.
Miscellaneous options Optionally the intensity between single raw images can be adjusted to compensate for fluctuations of the laser intensity. Every raw image is normalized by its overall intensity, norm Imn(r⊥) Imn (r⊥) = R . This correction assumes that the overall intensity should Imn(r⊥) dr⊥ be the same for all raw images even though they are illuminated by different gratings. This approximation might only hold for large, dense samples and should be used with care on sparse samples. Another option is to correct the raw images for lateral drift between single images of one grating orientation. Due to the modulated illumination the raw images contain different information. By low-pass filtering the raw images in Fourier space most of the contributions from the grating are suppressed as this information is mainly around ±kg,d. The low-pass filtered real space images are then cross-correlated to yield the lateral drift.
36 2.4. Numerical image reconstruction in structured illumination microscopy
Fourier transformation
All raw images are dimmed down at their edges using a 2D Hanning window with a flat central area. Approximately the 10 outermost pixel in each direction are damped. Then the images are Fourier transformed using a fast Fourier transformation (FFT).
Calculate OTF
Ideally, an image of an experimentally measured 2D PSF is provided for the reconstruc- tion. The PSF is centered, rotationally averaged and then Fourier transformed to yield the OTF of the system, which is needed throughout the reconstruction. If no PSF im- age is provided the OTF is theoretically calculated from the NA, the index of refraction of the immersion medium, and the emission wavelength. For most of the reconstruc- tions performed in this work an experimentally measured PSF was provided that was measured by imaging a small fluorescent bead of 92 nm diameter.
2.4.2. Separation of spectral components
Once the raw images are preprocessed the next step is to separate the spectral compo- nents as described in sec. 2.3.4. This is done separately for each orientation d of the illumination grating.
Separation matrix n Three raw images Imgd (r⊥)(n = −1, 0, 1) along one direction d are multiplied with the separation matrix Mc−1 according to eq. 2.53 to yield the three separate spectral components. For the unmixing it is assumed that Cmod = 1. It must be mentioned that the global phase ϕ0 is not yet known but will be corrected for later. Optionally, an iterative matrix optimization can be performed. In this process the phase steps ∆ϕ and the intensities Cmod are varied for the individual images and the resulting separation is evaluated by the suppression of residual orders in the spectral components. A detailed description of the iterative matrix optimization can be found in the works by K. Wicker and O. Mandula [34, 35].
Grating period
The grating period (or grating vector kg,d) of the illumination pattern is defined by the illumination beam path. For the TIRF-SIM set-up built in this thesis the grating period is dictated by the pixel pitch of the spatial light modulator, the pixel period of the hologram and the magnification of the illumination beam path (see chap. 3). These parameters are known with some accuracy. However, to shift the separated spectral components to their true positions in k-space and to avoid artifacts in the reconstruction due to destructive interference of mismatched spectral information the grating period has to be determined with very high precision.
37 2. Theoretical background