Wide-field Structured Illumination Microscopy for Fluorescence and Pump-Probe Imaging
By
Yang-Hyo Kim
B.S., Mechanical and Aerospace Engineering Seoul National University (2005)
S.M., Mechanical Engineering Massachusetts Institute of Technology (2007)
Submitted to the Department of Mechanical Engineering in Partial Fulfillment of the Requirements for the Degree of
Doctor of Philosophy
at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY
February 2019
C 2018 Massachusetts Institute of Technology. All rights reserved Signature redacted
Signature of Author: Department of Mechanical Engineering Signature redactedIan i5, 2019 Certified by: Peter T. C. So Professor of Mechanical Engineering and Engineering .s ervisor Signature redacted Accepted by: MASSACHUSETS INS1TTWE Nic las Hadjiconstantmou OF TECHNOLOGY Chairman of the Departmental Committee for Graduate Students
IEB 25 2019 LIBRARIES ARCHIVES Wide-field Structured Illumination Microscopy for Fluorescence and Pump-Probe Imaging
By
Yang-Hyo Kim
Submitted to the Department of Mechanical Engineering on Jan 15, 2019 in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Mechanical Engineering
Abstract
The optical resolution of microscopy is limited by the wave-like characteristic of the light. There are many recent advances in overcoming this diffraction limited resolution, but mostly focused on fluorescent imaging. Furthermore, there are few non-fluorescence wide-field super-resolution techniques that can fully utilize the applicable laser power to optimize imaging speed. Structured illumination microscopy is a super-resolution method that relies on patterned excitation. This thesis has presented novel applications of structured illumination microscopy to surface plasmon resonance fluorescence and pump-probe scattering imaging. First, structured illumination microscopy was introduced to surface plasmon resonance fluorescence imaging for high signal-to- noise and high resolution. Secondly, a theoretical framework for three-dimensional wide-field pump-probe structured illumination microscopy has been developed to increase the lateral resolution and enable depth sectioning. Further, structured illumination wide-field photothermal digital phase microscopy is proposed as a high throughput, high sensitivity super-resolution imaging tool to diagnose ovarian cancer. Finally, I have derived the exact analytical solution to the heat conduction problem in which a sphere absorbs temporally modulated laser beam for photothermal microscopy. The proposed method also has a great potential to be applied to other pump-probe modalities such as transient absorption and stimulated Raman scattering.
Thesis Supervisor: Peter T. C. So Title: Professor of Mechanical Engineering and Biological Engineering
2 Acknowledgement
First, I would like to thank my thesis advisor, Prof. Peter So for his generosity and guidance. His door was always open and I could ask whatever silly question to him. Whenever discussion needed, I could talk fact-to-face physically or through Skype meeting even when he was waiting for the airplane in the international airport. I also would like to thank Prof. Shyamsunder Erramilli and Prof. Mi Hong for their advice and help. They were another greatest support for my study and life at MIT. I would like to express my gratitude to Prof. George Barbastathis and Prof. Nicholas Xuanlai Fang for their advice, guidance and generous support as my committee members. I appreciate Prof. Colin J. R. Sheppard, Prof. K. C. Toussaint, Jr., Prof. Conor L. Evans, and Prof. Aydogan Ozcan for their academic and personal support.
I also thank all the former and current So Lab members and LBRC members, especially Daekeun Kim, Hyuksang Kwon, Euihean Chung, Maxine Jonas, Heejin Choi, Jae Won Cha, Dimitris Tzeranis, Hayden Huang, Ki Hean Kim, Christopher Rowlands, Elijah Yew, Vijay Raj Singh, Yun-Ho Jang, Jong Kang Park, Barry Masters, Yi Xue, Baoliang Ge, Dushan Wadduwage, Murat Yildirim, Sossy Megerdichian, Zahid Yaqoob, Jeong Woong Kang, Sungsam Kang, Surya Pratap Singh, Renjie Zhou, Luis H. Galindo, and Christine Brooks for their academic advice, numerous help, and warm-hearted support.
All of my collaborators, Wai Teng Tang and Sinyoung Jeong, sincerely worked together with me and their positive energy always encouraged me to move forward. I want to thank all the people who have been with me in my Boston life including Korean Graduate Student Association members in Mechanical Engineering. The administrators of Mechanical Engineering and Biological Engineering, Leslie Regan and Olga Parkin, took care of various issues I generated with magically smooth and timely manner.
I cannot imagine how I can express my appreciation to my family: Hong Sik Kim, Gi Soon Lee, Mi Yeon Kim, Hee Chang Jeong, Young Hee Kim, Kyung Suk Jeong, and Buhyun Yun. Especially I thank my wife, Hyung Mee Jeong for her constant support, sacrifice and endless love.
I would like to acknowledge my funding sources and sponsors: National Institutes of Health (9P41EB015871-26A1, 5R01NS051320, 4R44EB012415, and 1R01HL121386-OlAl), National Science Foundation (CBET-09395 11), Hamamatsu Corporation, Singapore-Massachusetts Institute of Technology Alliance for Research and Technology (SMART) Center, BioSystems and Micromechanics (BioSyM), and Samsung Scholarship.
3 Table of Contents
1. Introduction ...... 10 1.1. M icroscopic image formation...... 10 1.2. Super-resolution m icroscopy ...... 11 1.3. Imaging speed improvem ent in m icroscopy ...... 14 1.4. 3D wide-field m icroscopy technologies ...... 15 1.5. Review of super-resolution m icroscopy...... 17. 1.6. Objectives ...... 17
2. Wide-field extended-resolution fluorescence microscopy with standing surface-plasmon- resonance waves...... 20 2.1. Introduction...... 20 2.2. Principle of SW -SPRF m icroscopy...... 22 2.2.1. Fluorescence excitation in SW -SPRF microscopy ...... 22 2.2.2. Fluorescence detection in SW -SPRF m icroscopy...... 24 2.2.3. Im age formation in SW -SPRF microscopy ...... 27 2.3. M ethods and Experimental Setup ...... 30 2.4. Signal intensity comparison of different im aging modes...... 31 2.5. Calculation of the transmitted intensity and collected emission photons ...... 33 2.6. Point spread function measurement of SW-SPRF microscope ...... 34 2.7. Conclusion ...... 36
3. Theoretical framework for three-dimensional wide-field pump-probe structured illum ination m icroscopy ...... 39 3.1. Introduction...... 39 3.2. Principle of 3D SIM pump-probe m icroscopy...... 41 3.2.1. Configuration of a wide-field pump-probe structured illumination microscope ...... 41 3.2.2. Theoretical fram ework...... 43 3.2.2.1. 3D CTF and getting a 3D object information with a 2D detector ...... 43 3.2.2.2. Principle of 3D wide-field pump-probe structured illumination microscope. 48 3.2.2.3. Ideal choice of structured illumination phase set for 3D wide-field pump- probe structured illum ination m icroscope ...... 54 3.3. M ethods for num erical simulation ...... 56
4 3 .4 . R esu lts...... 5 7 3.4.1. Imaging of a planar target: Calibration Chart ...... 57 3.4.2. Imaging of a non-planar target: a 3D M IT Logo ...... 60 3.4.3. Imaging of Biomolecules in HeLa Cells...... 61 3.5. Discussion...... 62 3.6. Conclusion ...... 66
4. Structured illumination wide-field photothermal digital phase microscopy ...... 70 4.1. Introduction...... 70 4. 1. 1. Motivation ...... 70 4.1.2. Photothermal microscopy ...... 71 4.2. M ethods and Experimental Setup ...... 74 4.2.1. Structured illumination photothermal digital phase microscopy ...... 74 4.2.2. Experimental setup ...... 75 4.2.3. Structured illumination of pump beam ...... 77 4.2.4. Sample preparation ...... 79 4.3. Image acquisition and processing flow ...... 81 4.3.1. 1/f noise correction and lock-in amplification...... 83 4.3.2. Holographic reconstruction, SIM reconstruction, and CTF equalization ...... 86 4.3.3. Aberration correction ...... 88 4.4. Results and discussion ...... 95 4.4.1. Signal-to-background ratio (SBR) enhancement ...... 95 4.4.2. Resolution enhancement (Point spread function measurement)...... 96 4.5. Conclusion...... 98
5. Analytical derivation of temperature field by a spherical particle that absorbs temporally m odulated radiative energy...... 101 5.1. Introduction...... 101 5.2. Analytical derivation of temperature field ...... 101 5.3. Temperature distribution in the experimental condition ...... 108 5.4. Conclusion ...... 109
6. Conclusion and Outlook ...... 110
5 List of Figures
Fig. 1.1. Far-field super-resolution imaging principles. (a) Structured illumination microscopy (SIM). (b) Stimulated emission depletion (STED). (c) Photoactivated localization microscopy (PALM) / Stochastic optical reconstruction microscopy (STORM)...... 12 Fig. 2.1. (a) Schematic representation of SPR. (b) Schematic of a typical SPR bio-sensing device. (c) SPR microscope utilizing a high numerical aperture objective lens replacing a prism.. 20 Fig. 2.2. (a) Excitation of dipole by a P-polarized incident plane wave in SPRF. The standing evanescent wave excitation from two electromagnetic plane waves in SW-TIRF/SW-SPRF: (b) S-polarization and (c) P- polarization...... 22 Fig. 2.3. (a) A schematic view of the SPRF imaging process within a 4f optical system. (b) Axis convention used in the derivation of the field in medium 3...... 25 Fig. 2.4. Concept of resolution enhancement by standing-wave surface plasmon resonance fluorescence (SW-SPRF) microscopy in 2D reciprocal space (spatial Fourier space)...... 27 Fig. 2.5. SW -SPRF microscopy experimental setup ...... 30 Fig. 2.6. Comparison of various imaging modes ...... 32 Fig. 2.7. (a) Transmitted intensities on the gold surface with respect to the incidence angle for the cases of SPRF and TIRF with S- and P-polarization incident light. (b) Fluorescence emission intensities collected by objective from fluorophores at different distance from the substrate. (c) AFM image of gold surface (bump view)...... 33 Fig. 2.8. Extended-resolution imaging with SW-SPRF microscopy in vertical direction: (al) original SPRF image with doughnut-shape PSF, (bl) deconvolved SPRF image, (cl) SW-SPRF image after applying the SW-TIRF algorithm on three deconvolved SPRF images, (dl) SW- SPRF image with linear deconvolution to reduce side lobes; (a2)-(d2) comparison of PSF profiles of various imaging methods at a selected region of interest (ROI)...... 35 Fig. 3.1. Wide-field pump-probe structured illumination microscope (ppSfIM) design...... 42 Fig. 3.2. Definition of the diffraction plane and the observation plane...... 44 Fig. 3.3. 4f optical imaging system with a thin object ...... 44 Fig. 3.4. 4f optical imaging system with a thick object...... 45 Fig. 3.5. 4f optical imaging system with a shifted thick object...... 46 Fig. 3.6. Schematic diagram of the 3D coherent transfer function for coherent imaging with a circular lens in 4f system ...... 47 Fig. 3.7. Pump beam configuration and probe beam extended transfer function for pump-probe structured illumination microscopy (ppSIM )...... 48
6 Fig. 3.8. (a) The two wave vectors corresponding to the each pump beam direction. Two wave vectors have the same magnitude. (b) Three complex numbers equally distributed on the circle whose center is located in the origin. (c) Arbitrary number of complex numbers equally distributed on the circle whose center is located in the origin...... 54 Fig. 3.9. Numerical simulation results of USAF 1951 test chart ...... 59 Fig. 3.10. 3D M IT logo image simulation result ...... 60 Fig. 3.11. The HEC 1 (a-c) and DNA (e-h) in HeLa cells ...... 61 Fig. 4.1. (a) Dark field image of anti-CD4 plasmonic nanoparticles (PNPs) (80 nm) in surfactant solution added to whole blood. (b) ratio of scattering to absorption cross-sections of spherical nanoparticles with various diameters...... 70 Fig. 4.2. Typical photothermal microscopy setups for (a) point-scanning configuration and (b) wide- field configuration ...... 73 Fig. 4.3. Schematic diagram of structured illumination photothermal digital phase microscopy..... 74 Fig. 4.4. Experimental setup for structured illumination photothermal digital phase microscopy ... 75 Fig. 4.5. Structured illumination pump beam by SLM as a phase grating ...... 78 Fig. 4.6. Effect of structured illumination parameter errors to point spread functions ...... 79 Fig. 4.7. Image acquisition and processing flow for structured illumination photothermal digital phase m icroscopy ...... 81 Fig. 4.8. (al) Schematic diagram of an optical setup for collection of light. (a2) Patterns on DMD. Fourier space diagram for (bl) original sample information, (b2) pump beam intensity (transient grating), (b3) sample information modulated by the pump beam intensity, (b4) DMD plane, and (b5) acquired image filed by the camera ...... 81 Fig. 4.9. Qualitative noise spectrum of a typical photothermal experiment...... 83 Fig. 4.10. Probe beam intensity fluctuation compensation ...... 84 Fig. 4.11. (a) Camera samples and integrates the signal synchronously to the pump beam modulation at four points per sinusoidal waveform. (b) Four-step modulation signal recon struction ...... 85 Fig. 4.12. Image reconstruction in Fourier space ...... 86 Fig. 4.13. Spherical (sp) and aberrated (ab) wavefronts ...... 88 Fig. 4.14. Aberration correction for structured illumination digital phase microscopy ...... 92 Fig. 4.15. Aberration corrected imaging with structured illumination photothermal digital phase m icro scop y ...... 93 Fig. 4.16. Step by step procedure of pattern search (PS) algorithm...... 94 Fig. 4.17. (a) Schematic diagram of an optical setup for SBR enhancement. (b) Before SBR
7 enhancement. (c) After SBR enhancement. 60 nm gold nanoparticles in 2% agarose gel were used for a phototherm al sample...... 95 Fig. 4.18. Extended-resolution imaging with structured illumination photothermal digital phase microscopy in horizontal and vertical directions ...... 97 Fig. 5.1. Schematic diagram for a heat flow problem ...... 102 Fig. 5.2. C ontour of integration...... 105 Fig. 5.3. Temperature increase for gold nanoparticle in water ...... 108
List of Tables
Table. 1.1. Definitions of lateral resolution of an optical system as a function of the illumination ty p e ...... 12 Table. 4.1. The first 17 Zernike polynom ials ...... 90
Abbreviations
1D one-dimensional 2D two-dimensional 3D three-dimensional ab aberrated AFM atomic force microscope AOI area of interest ATF amplitude transfer function CARS coherent anti-Stokes Raman scattering CCD charge-coupled device CMOS complementary metal-oxide-semiconductor CTF coherent transfer function DMD digital mirror device EMCCD electron multiplied charge-coupled device FOV field-of-view FWHM full-width at half-maximum GPS generalized pattern search GSD ground-state depletion iCCD intensified charge-coupled device
8 MMM multifocal multiphoton microscopy NA numerical aperture ND neutral density NSOM or SNOM near-field scanning optical microscopy OC ovarian cancer OD optical density ODT optical diffraction tomography OPD optical path difference OTF optical transfer function PALM photoactivated localization microscopy PNP plasmonic nanoparticle ppSIM pump-probe structured illumination microscopy PSF point spread function RESOLFT reversible saturable optical fluorescence transitions SBR signal-to-background ratio sCMOS scientific complementary metal-oxide-semiconductor SIM structured illumination microscopy SLM spatial light modulator SNR signal-to-noise ratio sp spherical SPCE surface plasmon-coupled emission S-Pol, P-Pol S-polarization, P-polarization SPR surface plasmon resonance SPRF surface plasmon resonance fluorescence SRS stimulated Raman scattering STED stimulated emission depletion STORM stochastic optical reconstruction microscopy SW-SPRF standing-wave surface plasmon resonance fluorescence SW-TIRF standing-wave total internal reflection fluorescence TA transient absorption TIR total internal reflection TIRF total internal reflection fluorescence XP exit pupil
9 Chapter 1
Introduction
1.1. Microscopic image formation
The development of optical microscopes has enabled the successful observation of fine structures in nature and contributed significantly to science and technology development in many areas. For example, fluorescence microscopy has been a powerful tool for biologists to observe a broad range of sample types from single molecules to whole organisms [1] and pump-probe microscopy has enabled molecular characterization in material science, medicine, and art conservation [2]. Unless otherwise specified, our discussion will be limited to the fluorescence microscopy and pump-probe microscopy in the following sections.
Optical microscopic imaging is measuring light distribution at the image space emitted/scattered from the sample after passing through an optical imaging system. To understand the image formation of an optical imaging system, wave optics is needed for the diffraction properties of light, where diffraction refers to the bending of a wave when it encounters an obstacle. In the optical imaging theory based on wave optics, the relation between object and image can be described as a convolution of the light source distribution in the sample with the point spread function (PSF) of the optical system [3, 4]. The point spread function (PSF) is the three-dimensional diffraction pattern of light emitted/scattered from an infinitely small point source in the sample. The PSF provides a measure of imaging performance and characteristics of an optical system combining the effects such as optical resolution and aberrations. There is another way to describe how the information content of an object is transferred through an optical system. In a spatial Fourier space, the relation between object and image can be described as a multiplication of the light source distribution in the sample with the transfer function of the optical system, which is the Fourier transform of PSF.
10 An optical system behaves differently depending on whether the light source is coherent or incoherent.
In fluorescence microscopy (incoherent), the image intensity is described as a convolution of the object intensity with the "incoherent" PSF in the spatial domain and a multiplication of the object intensity with the optical transfer function (OTF) in the spatial Fourier domain. In pump-probe microscopy (coherent), the image field is described as a convolution of the object field with the "coherent" PSF in the spatial domain and a multiplication of the object field with the coherent transfer function (CTF) in the spatial
Fourier domain. CTF is also called as amplitude transfer function (ATF).
Despite the success of optical microscopy as a wonderful tool to explore the nature, there are several challenges that limit the usage of optical-microscopy in answering scientific questions and in meeting industrial demands [5]. In the following chapters, I will shortly review how people have developed various methods to overcome some of these challenges related to this thesis: optical resolution, throughput
(temporal resolution), and optical sectioning (3D imaging).
1.2. Super-resolution microscopy
The optical resolution of microscopy is limited by the wave-like characteristic of the light, more specifically diffraction [6]. Ernst Abbe (1873) first defined how the diffraction of light determines the image resolution according to the wavelength and the NA of an objective lens in Table 1.1. His resolution definition corresponds to the spatial frequency limit of the transfer function. Resolution is also often defined as the largest distance at which the image of two point-like objects seems to be resolved (for example, Rayleigh limit, Sparrow limit, and full width at half maximum of the PSF). Roughly speaking, these limit state that it is impossible to resolve two elements of a structure which are closer to each other than about half the wave length (k) in the lateral direction (several hundred nm). People have developed various methods to overcome this diffraction-limited resolution of optical microscopy.
11 . I
Table. 1.1. Definitions of lateral resolution of an optical system as a function of the illumination type [7]. Rayleigh Sparrow Sparrow Abbe Abbe (incoherent) (coherent) (incoherent) (coherent) (incoherent) 0.61 A/NA 0.73 A/NA 0.47 A/NA AlNA 0.5 ANA
Total internal reflection fluorescence (TIRF) microscopy utilizes near-field evanescent excitation of a very thin region on the order of 100 nm behind the surface of the cover glass [8]. This selective excitation of TIRF provides intrinsic optical sectioning (removes the out-of-focus noise) and reduces photobleaching of fluorophores outside the focal volume. Although TIRF microscopy has become a valuable tool to study cellular processes near the basal plasma membrane and single molecule near the surface [8], it doesn't provide imaging into the interior of cells and the lateral resolution still remains the same as conventional wide-field microscopy.
A near-field scanning optical microscopy (SNOM, NSOM) scans samples with a very small physical aperture (for example, the tip of a tapered glass fiber) instead of objective lenses. The evanescent wave is limited laterally as well as axially, thus bypassing the diffraction limit in all three spatial dimensions, bringing the resolution to below 20 nm [9]. However, these scanning probe methods shows significant resolution degradation in soft biological specimens; they can only image surface structures and suffer from slow scanning speed with minute scale frame rate.
(a) (b) (c)
Interference of Excitation PSF exciting light T with sample + structure Time Gauss fitting (Mowr effect) STED pulse PSF series of individual spots (few -o Projection 1,0- (PointIllism) Mathem0atic Mathmati ~expo- .. reconstruction Effective PSF * sures) (PSF shaping)
Fig. 1.1. Far-field super-resolution imaging principles [10]. (a) Structured illumination microscopy (SIM). (b) Stimulated emission depletion (STED). (c) Photoactivated localization microscopy (PALM) / Stochastic optical reconstruction microscopy (STORM).
12 Structured illumination microscopy (SIM) illuminates a sample with a series of sinusoidal patterns as
shown in Fig. 1.1(a) [11]. When a sinusoidal pattern illuminates a sample containing finer structures than the diffraction limit, coarser interference patterns arise in the emission distribution. These coarse fringes
are coarser than the diffraction limit and can be transferred to the image plane by the microscope. By processing all acquired images, we can get a high-resolution image of the underlying structure. With this approach, the resolution can increase by a factor of two beyond the diffraction limit. Compared to other far-
field super-resolution techniques, SIM has an advantage that any fluorescent label can be used because SIM process does not rely on any photo-physical characteristics of the fluorophores.
Stimulated emission depletion (STED) microscopy selectively de-excites fluorophores with a doughnut shape depletion beam to minimize the effective illumination area at the focal point as shown in Fig. 1.2(b)
[12]. STED forces the excited electrons in fluorophores to relax into a ground state releasing fluorescence photons with the same color as the de-excitation beam. Current experimental setups routinely achieve a resolution in the range of 30-80 nm [10]. One of the most important considerations for STED microscopy is the choice of the right fluorophore to achieve efficient stimulated emission. The wavelength of the stimulated emission photon should be in the range of the emission wavelength of the fluorophore and should not be too close to the tail of the absorption spectrum. In addition, because STED happens efficiently when
STED laser intensity is more than a certain threshold, the fluorophore should also have good photostability.
Photo-activated localization microscopy (PALM or FPALM) [13, 14] and stochastic optical reconstruction microscopy (STORM) [15] are wide-field fluorescence imaging methods that utilize stochastic activation of photoswitchable fluorophores and estimates the centroid of point emitters. Only sparse subset of fluorophores are activated/excited, the centroid of each fluorophore is determined by fitting the emission profile to a known distribution, and the current set fluorophores are deactivated subsequently.
This process repeats for different subset of fluorophores until a final image of the sample is generated typically reaching a resolution in the range of 30 nm [10]. Because of extended timescales compared to traditional imaging, these PALM/STORM techniques are more applicable to fixed cell or tissue or
13 observation of relatively slow dynamic phenomena. In addition, these methods may perform worse with dense and bulky structures than sparse and smaller objects.
1.3. Imaging speed improvement in microscopy
Imaging developmental or dynamic processes often requires adequate time resolution like heart development (50-130 frames per second (fps)), cilia beating (900 fps) or fluid flow in developing embryos
(44 fps) [16]. A high pixel rate is also required to study slower large-scale processes such as collective cell migration or cell division patterns with a large number of pixels per image (100 million voxels per 3D- image stack in less than a minute) [16]. For live animal imaging, fast imaging speed becomes critical for minimizing animal stress due to prolonged anesthesia and overcoming motion artifacts.
A conventional point-scanning confocal or two-photon microscopy is sometimes too slow to observe aforementioned dynamic processes or to study large volume samples since image is recorded one pixel at a time with a pixel rates of only 105~106 pixels/s [16]. One approach to improve imaging speed beyond conventional point scanning approach is utilizing higher speed scanners such as polygonal mirrors, resonant mirror scanners, or acousto-optic deflectors [16]. These high-speed scanners can typically achieve frame rate up to about 1 kHz in tissues without losing imaging depth compared to the conventional point scanning microscopy. However, the higher speed scanning requires increasing the excitation laser power to compensate decreased pixel dwell time and the pixel rate is fundamentally limited by fluorophore saturation or photodamage [16].
Another strategy to circumvent this limitation is to parallelize the sample illumination and the signal detection with multifocal point-scanning mechanism. Several approaches have been explored to improve the imaging speed up to 10' pixels/s, including spinning-disk confocal microscopy [17], multifocal multiphoton microscopy (MMM) [18], and line-scanning confocal microscopy [19]. Using multifocal excitation, overall pixel rate can be increased while maintaining the same illumination time per pixel. To implement multifocal point-scanning microscopy, we need multifocal excitation device (e.g., a lenslet array,
14 a diffractive optical element), a scanner, and often area type detector (such as a CCD or a CMOS sensor)
for simultaneous acquisition of data from multiple foci. For a turbid or thick specimen with multifocal point-scanning microscopy, the scattered emission photons can deviate from the correct location to adjacent pixels (phenomenon called as crosstalk) resulting in increased background, image blurring, and sometimes
severe ghost images.
The wide-field microscopy is the ultimate parallelization of image acquisition where exciting and imaging occur across the whole field of view simultaneously. The development of high speed, low noise, and high sensitivity area type detectors like electron multiplied (EMCCD), intensified CCD (iCCD), and scientific CMOS (sCMOS) has made even single molecule observation become a routine using wide-field microscopy. We can easily find commercial near single photon sensitivity sCMOS cameras with -5 x 108 pixels/s in the market and this pixel rate is keep increasing with the development of electronics for data transfer from the camera to the storage device. However, the crosstalk issue is the most severe compared to above-mentioned single point-scanning and multi-focal scanning microscopies. In addition, compared to previous two single/multiple point-scanning methods, much higher peak power laser pulses are needed for non-linear contrast generation microscopies like multi-photon, second harmonic generation, and stimulated
Raman scattering.
1.4. 3D wide-field microscopy technologies
Another major disadvantage of basic wide-field microscopy is that it has no (coherent) or incomplete
(incoherent) depth sectioning capability because the light from the sample is integrated through the Z dimension. This limitation is critical for samples where out of focus light obscures details or when we need to know the 3D distribution/structure information of the sample.
The same principle of the structured illumination microscopy (SIM) for lateral resolution enhancement can be applied to the axial dimension for complete optical sectioning wide-field microscopy: the spatially structured excitation intensity in axial dimension causes normally unreachable depth sectioning information
15 to become encoded into the observed images through spatial frequency mixing [20]. The main advantage of SIM is that we can optically section the sample with super-resolution images. On the other hand, SIM requires multiple images to provide optical sectioning, which may cause photo-bleaching/damage of the sample or blurred image from the sample movement.
Light-sheet microscopy illuminates the sample perpendicularly to the direction of observation with a laser light-sheet for optical sectioning [21]. One way of generating light-sheet is focusing a laser beam only in one direction (e.g. using a cylindrical lens) and the other way is using a beam with long depth of field and scanning in an orthogonal direction. The separation of the illumination and detection beam paths in most light-sheet microscopes (except oblique plane microscopy [22]) requires specialized sample mounting methods. The advantage of light-sheet microscopy is that it provides reliable high frame rate imaging over long periods with minimal sample exposure. There is a trade-off between axial resolution and the field of view for light-sheet microscopy because the light sheet has a hyperbolic profile in the xz plane (not perfectly planar).
For wide-field nonlinear microscopies like multiphoton and second harmonic generation, temporal focusing microscopy enables axial localization of excitation to a single plane by temporal focusing of an ultrafast pulsed excitation [23, 24]. Temporal focusing microscopy spectrally disperses the light pulse using dispersive element like a grating and makes it recombined near the focal plane only achieving axial resolution comparable to line-scanning nonlinear microscopy for wide-field excitation. However, wide- field temporal focusing imaging is often limited by the smaller field-of-view (FOV) due to the need for much higher peak power laser pulses than point-scanning method.
For 3D scattering microscopy, optical diffraction tomography (ODT) has been developed to measure the 3D refractive index distribution of optically transparent or semi-transparent samples using interferometric technique [25]. In ODT, 3D refractive index distribution of a weakly scattering sample is reconstructed from the measurements of multiple 2D holograms of the object obtained with various
16 illumination angles or rotation of the sample itself. The 3D distribution of the refractive index can be translated into various useful parameters such as protein concentrations and cellular dry mass [25].
1.5. Review of pump-probe microscopy
Nobel laureate Dr. Ahmed Hassan Zewail pioneered femtochemistry which studies chemical reactions on
femtosecond timescales using femtosecond transient absorption (TA) technique [26]. In a transient absorption experiment, pump laser pulses excite a molecule and a probe laser pulse measures the population in the excited state (femtosecond or picosecond timescale relaxation dynamics) at different temporal delays with respect to the excitation. This experimental method was termed as pump-probe spectroscopy.
However, the concept of pump-probe is very general and can include a variety of linear and nonlinear interactions [2] like linear photothermal effect (heat induced refractive index change resulting from photoexcitation of material) and nonlinear cross-phase modulation (change in the refractive index of a material in response to an applied electric field). Pump-probe microscopy is an emerging spectroscopic optical imaging technique with the following advantages [26]: First, it is nondestructive to the sample. Thus, it can be used as a repeatable diagnostic tool. Second, it can be a label-free technique and doesn't need an exogenous target. Third, compared to scattering measurements, transient absorption based pump-probe method has a weaker dependence on particle size and thus is highly sensitive to nanoscale subjects.
1.6. Objective
First, I will shortly review the history of SIM techniques that our group developed related to this dissertation.
Our group suggested standing wave total internal reflection fluorescence (SW-TIRF) microscopy combining SIM and TIRF microscopy for higher lateral resolution than conventional SIM [27, 28]. In this approach, the excitation standing evanescent wave has an effective wavelength shorter than the free space excitation light by a factor of approximately 2n where the factor of 2 comes from the conventional SIM and n results from the high refractive index of the substrate. A prismless objective launched TIRF
17 configuration is more convenient than a conventional prism based TIR geometry for biological investigation and our group experimentally realized SW-TIRF in an objective-launched geometry for the first time [29].
Our group also demonstrated the performance of the SW-TIRF microscopy with a biological sample of cellular actin cytoskeleton of mouse fibroblast cells as well as single semiconductor nanocrystal molecules
(quantum dots) [30].
This dissertation describes the development of apparatus and methods to provide wide-field super- resolution imaging technique by applying structured illumination microscopy (SIM) to two different imaging modalities, surface plasmon resonance fluorescence (SPRF) and pump-probe microscopies. More specifically, standing-wave surface plasmon resonance fluorescence (SW-SPRF) microscopy demonstrates super-resolution wide-field imaging based on SPR standing waves with signal-to-background ratio compared to conventional wide-field fluorescence microscopy. Our wide-field pump-probe structured illumination microscopy (ppSIM) enables increase in imaging speed by parallelized acquisition, super- resolution wide-field pump-probe imaging, and 3D sectioning.
Several aims will be pursued in this thesis. First, new contrast mechanisms using the standing wave total internal reflection will be explored based on surface plasmon resonance effect on the fluorescence emission.
Second, the theoretical framework of 3D wide-field ppSIM will be discussed and characterized with computer simulation. Third, the implementation and the characterization of wide-field photothermal (one branch of pump-probe imaging modalities) structured illumination microscopy apparatus will be discussed to improve resolution via structured light and increase imaging speed by fully utilizing available laser power.
Finally, I will further derive an analytical solution for heating in nanoscale metal particles that are important to theoretically model photothermal imaging.
References
1. Combs, C.A., FluorescenceMicroscopy: A Concise Guide to Current Imaging Methods, in Current Protocols in Neuroscience. 2001, John Wiley & Sons, Inc. 2. Fischer, M.C., et al., Invited Review Article: Pump-probe microscopy. Review of Scientific Instruments, 2016. 87(3). 3. Gu, M., Advanced OpticalImaging Theory. 2000: Springer, Heidelberg. 4. Goodman, J.W., Introduction to FourierOptics. 2005: Roberts & Company, Greenwood Village, CO.
18 5. Antony, P.M.A., et al., Light microscopy applications in systems biology: opportunities and challenges. Cell Communication and Signaling, 2013. 11. 6. Born, M. and E. Wolf, Principlesof Optics. 1959: Cambridge University Press, Cambridge, UK. 7. Colonna De Lega, X. and P. de Groot. Lateral resolution and instrument transferfunctionas criteriafor selecting surface metrology instruments. in Imaging and Applied Optics TechnicalPapers. 2012. Monterey, California: Optical Society of America. 8m Axelrod, D., Cell-SubstrateContacts Illuminated by Total Internal-Reflection Fluorescence. Journal of Cell Biology, 1981. 89(1): p. 141-145. 9. Betzig, E. and J.K. Trautman, Near-FieldOptics - Microscopy, Spectroscopy, and Surface Modification Beyond the Diffraction Limit. Science, 1992. 257(5067): p. 189-195. 10. Schermelleh, L., R. Heintzmann, and H. Leonhardt, A guide to super-resolutionfluorescence microscopy. Journal of Cell Biology, 2010. 190(2): p. 165-175. 11. Gustafsson, M.G.L., Surpassing the lateralresolution limit by a factor of two using structured illumination microscopy. Journal of Microscopy-Oxford, 2000. 198: p. 82-87. 12. Hell, S.W. and J. Wichmann, Breaking the Diffraction Resolution Limit by Stimulated-Emission - Stimulated-Emission-DepletionFluorescence Microscopy. Optics Letters, 1994. 19(11): p. 780-782. 13. Betzig, E., et al., Imaging intracellularfluorescent proteins at nanometer resolution. Science, 2006. 313(5793): p. 1642-1645. 14. Hess, S.T., T.P.K. Girirajan, and M.D. Mason, Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophysical Journal, 2006. 91(11): p. 4258-4272. 15. Rust, M.J., M. Bates, and X.W. Zhuang, Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nature Methods, 2006. 3(10): p. 793-795. 16. Supatto, W., et al., Advances in multiphoton microscopyfor imaging embryos. Current Opinion in Genetics & Development, 2011. 21(5): p. 538-548. 17. Nakano, A., Spinning-disk confocal microscopy - A cutting-edge toolfor imaging of membrane traffic. Cell Structure and Function, 2002. 27(5): p. 349-355. 18. Bewersdorf, J., R. Pick, and S.W. Hell, Multifocal multiphoton microscopy. Optics Letters, 1998. 23(9): p. 655-657. 19. Masters, B.R. and A.A. Thaer, Real-Time Scanning Slit Confocal Microscopy of the in-Vivo Human Cornea. Applied Optics, 1994. 33(4): p. 695-701. 20. Gustafsson, M.G.L., et al., Three-dimensionalresolution doubling in wide-field fluorescence microscopy by structured illumination. Biophysical Journal, 2008. 94(12): p. 4957-4970. 21. Voie, A.H., D.H. Burns, and F.A. Spelman, Orthogonal-PlaneFluorescence Optical Sectioning - 3- Dimensional Imaging ofMacroscopic Biological Specimens. Journal of Microscopy-Oxford, 1993. 170: p. 229-236. 22. Bouchard, M.B., et al., Swept confocally-alignedplanar excitation (SCAPE) microscopy for high speed volumetric imagingof behaving organisms. Nat Photonics, 2015. 9(2): p. 113-119. 23. Oron, D., E. Tal, and Y. Silberberg, Scanningless depth-resolved microscopy. Optics Express, 2005. 13(5): p. 1468-1476. 24. Zhu, G.H., et al., Simultaneousspatial and temporalfocusingoffemtosecond pulses. Optics Express, 2005. 13(6): p. 2153-2159. 25. Kim, K., et al., Optical diffraction tomography techniquesfor the study of cell pathophysiology. 2016, 2016. 2(2). 26. Dong, P.T. and J.X. Cheng, Pump-ProbeMicroscopy: Theory, Instrumentation, and Applications. Spectroscopy, 2017. 32(4): p. 24-36. 27. So, P.T.C., H.S. Kwon, and C.Y. Dong, Resolution enhancement in standing-wave total internalreflection microscopy: a point-spread-functionengineering approach. Journal of the Optical Society of America a- Optics Image Science and Vision, 2001. 18(11): p. 2 8 3 3 -2 84 5 . 28. Cragg, G.E. and P.T.C. So, Lateral resolution enhancement with standing evanescent waves. Optics Letters, 2000. 25(1): p. 46-48. 29. Chung, E., D.K. Kim, and P.T.C. So, Extended resolution wide-field optical imaging: objective-launched standing-wave total internalreflection fluorescence microscopy. Optics Letters, 2006. 31(7): p. 945-947. 30. Chung, E., et al., Two-dimensional standing wave total internal reflectionfluorescence microscopy: Superresolution imaging ofsingle molecular and biologicalspecimens. Biophysical Journal, 2007. 93(5): p. 1747-1757. 19 Chapter 2
Wide-field extended-resolution fluorescence microscopy with standing surface-plasmon-resonance waves
2.1. Introduction
(a) (b) Analyteo Z Dielectric y y yAntibody xMetal Imaging System Light Prism Metal
Ligho
(c) Fluorescent sample Metal Cover slip
Objective lens
Excitat,
Emission
Fig. 2.1. (a) Schematic representation of SPR. (b) Schematic of a typical SPR bio-sensing device. (c) SPR microscope utilizing a high numerical aperture objective lens replacing a prism.
Surface plasmon resonance (SPR) occurs when energy from incident photons is transferred to electron density oscillations along a metal-dielectric interface (surface plasmons) in Fig. 2.1(a). Since precise momentum matching between photons and plasmons are required, SPR efficiency is sensitively dependent on the refractive index over the metal layer with a given incident angle of the excitation light.
This sensitivity has been utilized to develop optical biomolecular sensors in Fig. 2.2(b) [1, 2]. An
20 important application of these sensors is in proteomics where SPR sensing allows label-free and pico- molar sensitivity detection of protein binding kinetics. As proteomics studies demand higher throughput, high-density SPR sensors have been developed [3-5]. However, the density is limited by the cumbersome reflection geometry and lateral confinement of SPR excitation. Another application of SPR is the development of novel imaging tools where image contrast is produced from variations of dielectric coating thickness on metal surface [6, 7]. However, wide-field SPR microscopes suffer from low resolution similar to SPR sensors. On the other hand, sub-diffraction limited microscopes with tens of nanometer resolution have been developed but they suffer from having limited field of view or requiring unconventional dyes or photobleaching [8-11]. In this thesis, I demonstrate a super-resolving wide-field imaging technique with high signal-to-background ratio based on SPR. This technique has the potential to improve the density of current proteomic sensors by over four orders of magnitude.
Traditionally, SPR imaging has relied on the Kretschmann-Raether configuration using a prism to couple the light with surface plasmon in Fig 2.1(b). However, in recent SPR imaging experiments, the prism has been replaced by a high numerical aperture (NA) objective for compatibility with biological specimens and system simplicity [12-15] in Fig. 2.1(c). The resolution of SPR microscope is fundamentally limited by the lateral confinement of SPR wave and the diffraction limit of the emitted light. Our group developed standing-wave total internal reflection fluorescence (SW-TLRF) microscopy to enhance the lateral resolution of total internal reflection fluorescence (TIRF) microscopy using standing evanescent waves [16, 17]. Here I describe a novel imaging concept, named as standing-wave surface plasmon resonance fluorescence (SW-SPRF) microscopy, with sub-diffraction limit resolution approaching 100 nm by combining the SPR and SW-TIRF methods. This is the first demonstration of super-resolution wide-field imaging based on SPR standing waves.
21 2.2. Principle of SW-SPRF microscopy
2.2.1. Fluorescence excitation in SW-SPRF microscopy
(a) Flunrscernre di nie ip x Id -jMetal >- n2 Cover slip
I- Immersion oil 173 0W
(b) S-pol (c) P-pol
S ~ X S x -A .- - - n,,, nic kine kc2 kiwi 6,,, ,, 0m kinc2 inc , incl Inc2 Z Uinc2
Fig. 2.2. (a) Excitation of dipole by a P-polarized incident plane wave in SPRF. The standing evanescent wave excitation from two electromagnetic plane waves in SW-TIRF/SW-SPRF: (b) S- polarization and (c) P- polarization.
As shown in Fig. 2.2(a), the optical system used in SPRF to excite the fluorophores by surface plasmons is essentially the same as that of a TIRF microscope [18]. In SPRF, dipoles on the metal-coated glass slide are excited by a P-polarized incident plane wave in Fig. 2.2(a). ni denotes the refractive index of the
object space, n2 is the refractive index of the metal layer, and n3 is the refractive index of the glass slide and immersion medium. The dipole is placed at a distance d from the metal surface. k3 is the wave number of the incident plane wave in medium 3 intersecting the interface at an angle of 6O,, from the surface normal, and is given as k 2;rn,/A,, , where ,, is the wavelength of the incident light in free space.
22 The electric field amplitude of the refracted light in multiple layers can be described by the Fresnel equations [19]. The amplitude transmission coefficients for TIR/SPR with S-polarization and P- polarization are
tSTJR 2n2 cos02 t S, TIR - 2 n 2 1 12
tPTIR = 2n2 cos 02 , 2cos 0, + n, cos O2 (2.1) t 2t exp~ik tsspR- eXp i 2tCOS0 2 ) 2 +tP r 2exp(2ik 2tcos02)
PSPt2 t exp (ik2tcos 0 ) PSPR = 2 2
I r rp2'p2k2tcos 0 2 ) where t32 and t2 are the Fresnel coefficients for each of the respective two-layer interfaces and
02 of tS,SPR and tP,SPR is the angle in the metal layer given by Snell's law. With above Fresnel coefficients, the excitation field of TIR/SPR for S-pol and P-pol is simply given by [20]
Us,TIR(F,) 2 tSTIR exp (ik 2xsin6nc)exp k2z i n 2
2 UP,TIR 0 1O) n tP,TIR isin 2 ( (n ,/n2 ) x sinO inc2) exp(ik2x s @inc)exp (k 2z in 2 2
USSsPRQ,0) = ts sRexp (ik x sinoinc 2 3 exp (k 3z inc 1 3
2 LP,SPR V, 0) = - n3 tPSPR(i s in - (n,/n3 .j+sinci,, ) exp (ik3x sin ) eXp k3z sin inc - (1n3 2 where X and 2 denote the unit vectors in the x and z direction.
Let us restrict our discussion to one-directional resolution enhancement geometry because we can easily extend the same framework for symmetric resolution enhancement. The standing wave excitation intensity distribution for each imaging mode for SW-TIRF/SW-SPRF is obtained by interfering two counter-propagating waves with opposite incident angle in Fig. 2.2(b) and (c) as:
23 2tsTIR 12 [1 + cos(2k2 sin 0 - x)] exp (2k 2z SiIS,SW-TIR 2 0i - (n, /n2 )2 ,
22 IP-TI2 2, (2 sin 2 _(n,/n2 )2)r~ (n, <)) 2 "cos (2k2sin 0 x)1 IP,SW -TIR tPTIR2 22 22 ni (2sin 2 0 -- n, /n2)
x exp (2k z sin 2 0 2 - (n/n2 )2 (2.3)
2 22 3 2 ) SP R I S W -SPR P , (sin 2 0n I/n3 )2)I (n $1n3/) cos (2k'P,SW-SPR3sin 0 -x)j =_2 2 n, 2 sin 20 - (n, /n3)2
2 x exp (2k3 z sin 01 -(n, /n 3 )2
Above equations show exponentially decays with perpendicular distance z from the interface which
provides unique capability of illuminating a very thin region on the order of 100 nm. This near-field
excitation volume allows intrinsic optical sectioning to less than one-fifth of the excitation wavelength.
The selective excitation of TIRF/SPRF removes the out-of-focus noise and reduces photobleaching of
fluorophores outside the focal plan [18]. As a result, TIRF/SPRF is ideal for single molecule imaging to
probe cellular processes near the basal plasma membrane of adherent cells [21, 22], but they don't allow
deeper imaging into the interior of cells.
2.2.2. Fluorescence detection in SW-SPRF microscopy
In this section, I introduce the core concept to explain the collection of fluorescence emission by the
SPRF microscope based on vector field theory. This work was mostly done in collaboration with Mr. Wai
Teng Tang and Prof. Colin Sheppard of the National University of Singapore and the detail can be found
in the paper we published [20].
Due to the highly polarized emission of SPRF and the high NA objective used, a full vectorial
formulation will be derived for the model shown in Fig. 2.3, since a scalar treatment of the model does
not take into account these effects and would be considered inadequate here. We start by considering the
24 field emitted by the dipole in medium 1 which is subsequently transmitted into medium 3, and then propagate the waves through the 4f system into medium 4.
n, n2 n3 n4 (a):
r
-0 X .-- 04 iDipoic :l z
Objective lens Tube lens Image plane
(b) ke
Dipole k I 0
1W I n, n2 n3
z=-d z=O z=t
Fig. 2.3. (a) A schematic view of the SPRF imaging process within a 4f optical system. (b) Axis convention used in the derivation of the field in medium 3 [20].
Figure 2.3(b) shows the axis convention that is used for obtaining the expression for the dipole radiation in medium 3. The complex amplitude of the dipole source field in an infinite medium with index of refraction n is given by [23]
U,(r)=- Jd2k1 [k.k i -[i-k )k ]exp[i -(+d )] . (2.4) 2yrnM koi I
25 for and k, denotes the surfacewhere wave vector. v, =(nf -a2 , a k/ko k = , =(- ak2) k/ for z<-d
In Eq. (2.4), the field in medium 1 is effectively represented by an angular spectrum of plane waves,
where the limits of integration indicate that both evanescent waves and travelling waves are taken into
consideration.
Decomposing the plane waves in Eq. (2.4) into their p and s components and using the Fresnel
transmission coefficient for the three-layer system, the complex amplitude of the field in medium 3 (z > t)
can be expressed in terms of these unit vectors. Next, we propagate these plane waves to medium 4 and
use the vectorial Debye integral of Richards and Wolf [24], the field in the image space of the tube lens is
given by
-. ik ~-, U4 (r,q, z) =--4 fU' sin 4 exp(ik4rsinO 4 cos($ - p))exp(ik4zcosO4 )exp(iD)d 4d# . (2.5)
where the coordinates of a point in the image space are given by (r,qp,z), U' is the field in medium 4
after the objective lens, k4 is the wave number in medium 4, Q is the solid angle of the wave vector in
Debye integral, 04 is the polar angle of the wave vector in medium 4, and qp is the azimuthal angle of the
wave vector. It is assumed that as the wave propagates through the lenses, the meridional plane of the
propagation remains constant. (D denotes the wave front aberration function which is a measure of the
path difference between the wave front at the exit aperture of the tube lens and the wave front of an ideal
spherical wave [25].
Finally, since the detected intensity in the image space depends on the fluorescence dipole orientation
and an ensemble of fluorophores is imaged in a given sample, the total intensity from the fluorescent
sample is obtained by averaging the intensity over all possible dipole orientations.
26 2.2.3. Image formation in SW-SPRF microscopy
(a) n (b) (c) (d)
T Tt
Fig. 2.4. Concept of resolution enhancement by standing-wave surface plasmon resonance
fluorescence (SW-SPRF) microscopy in 2D reciprocal space (spatial Fourier space) [26]. (m, n, s)
is a spatial frequency coordinate corresponding to the spatial coordinate (x,y,z). (a) The optical
transfer function (OTF) for a wide-field conventional microscope limited by diffraction. (b) The spatial frequency components of the excitation intensity for the SW-SPRF. The possible positions of the two side components are limited by the excitation OTF (dashed). (c) The effective OTF for SW-SPRF after the reconstruction is the same as the convolution of the conventional OTF and the excitation intensity. This increased OTF delivers higher resolution in the direction of standing- wave excitation pattern. (d) From a sequence of SW-SPRF images with different orientation of standing-wave excitation pattern, it is possible to recover symmetric resolution-enhanced information.
Standing-wave surface plasmon resonance fluorescence (SW-SPRF) microscopy belongs to incoherent structured illumination microscopy (SIM) technology [16, 26-29]. Here I will introduce the fundamental image formation of our SW-SPRF microscopy in an incoherent SIM framework. This image formation framework is the same as that of SW-TIRF microscopy except some details like excitation intensity, shape of point spread function, and fluorescence emission intensity which I will discuss later. The image formation process in fluorescence microscopy can be described mathematically based on incoherent imaging theory [30, 31]. Assuming unit magnification without loss of generality, the image intensity in an incoherent imaging system can be described as
i(x,y,z)=[o(x,y,z)e(x,y,z)]®h(x,y,z) . (2.6) where (x, y, z) is a spatial coordinate, i(x,y, z) denotes the image field at position (x, y, z), o(x,y, z) is the pump-probe active chemical species concentration distribution in the object, e (x, y, z) is the intensity 27 distribution of excitation pump beam, i (x, y, z) is the incoherent point spread function (PSF) defined by the system's limiting circular aperture, and the symbol 9 stands for the convolution operator. After
Fourier transforming the image field, the image spatial frequency distribution can be described as
I(m,n,s)=[O(m,n,s)& E(rn,n,s)]fI(m,n,s) . (2.7) where (M, n, s) is a spatial frequency coordinate corresponding to the spatial coordinate (x, y, z) and H is called is called as an optical transfer function (OTF). I denote Fourier-transformed entity by changing from lowercase to uppercase font in this thesis.
From Eq (2.3), the evanescent standing wave excitation intensity for our SW-SPRF system (including
SW-TIRF system) can be approximately (neglecting the z directional profile) described as
e(x,y,z)= 2I+ acos(2kn, sin O,,, -x + A012)} (2.8) where a is the contrast of the standing wave, Q, is the incidence angle of the excitation beam at the interface, and k,,, = 2rn,,,4,, , where A., is the wavelength of the incident light in free space. By
Fourier transforming excitation intensity Eq. (2.4) for our SW-SPRF system, F e(x,y, z) , we obtain the excitation intensity spectrum in the spatial-frequency domain as
E(m,n,s) = Fe(x,y, z)} (2.9)
= 25(m,n,s) aexp(-iA~2 )(m+2k,,m,n,s) + aexp(iA 1 )S(m -2k,,,, n,s)
O,, wherewhere kk, k,,= nsinn'", 0,1 = ninc sin " .in Interference between two excitation beams produces one- dimensional excitation intensity pattern that contains three Fourier components at each difference frequency between the two illumination wave vectors in Fig. 2.4(b).
Substituting the previous result into Eq. (2.7), and mathematically expanding, we get the corresponding image intensity in the Fourier space as
28 20 (m,n, s)
J(m,n,s) = +aexp(-iA$2 )O(m+2k,,n,s) H(m,n,s) . (2.10)
L+aexp(iAO1 2)O(m- 2knls) J
This equation shows that the measured data gains new information that is not observable with a conventional uniform illumination microscope: the spatial frequency translation of the original object 0 moves new information into the support of H. The resolution of a microscope is limited by the "support" or "observable region" of the OTF, the region of two-dimensional reciprocal space (spatial Fourier space) where the OTF has nonzero values in Fig. 2.4(a). Our SW-SPRF has a larger effective OTF support (Fig.
2.4 c) than the normal observable region with uniform illumination (Fig. 2.4 a) by shifting higher- resolution information into the OTF support using structured illumination (Fig. 2.4 b), also known as spatial frequency mixing.
Although the image data captured by the SW-SPRF system contains more spatial frequency information compared to the conventional wide-field microscope, the high frequency spatial information is translated and mixed with other components in the raw data. To restore the data, these information components must be separated and shifted back to the original position in the spatial Fourier space. For this image reconstruction, at least 3 raw image data with corresponding independent phase A01 must be acquired because there are 3 spatial frequency shifted image components as unknowns. This formulation gives a lateral resolution enhancement in one direction. We can extend the formulation of ID SW-SPRF theory to achieve uniform resolution enhancement laterally by successive 1D SW-SPRF image acquisition with multiple SW directions [16, 26].
29 2.3. Methods and Experimental Setup
Standing evanescent wave excitation Au coated Cover slip Sample
PRl Objective, 60X, NA1.45 Half-wave plate Collimation Tube lens splitter ple CW laserp e 532nm J
ltrsuer Fiber itniiddtco o Retro tips Eiso reflector CMsso
Piezoelectric plier linear arrary azr ntensified detector for transducer i CCD phase detection
Fig. 2.5. SW-SPRF microscopy experimental setup: The output from the laser is split into two beams, which pass through a linear polarizer to regulate the polarization of the incident beam on a sample. Two collimated beams through the objective lens form standing waves under total internal reflection geometry. The evanescent fields are interfered at the interface and sinusoidal intensity distribution is formed for sample excitation.
The schematic of SW-SPRF microscope is shown in Figure 2.5. The incident angle of the excitation light
can be adjusted over a range of 41.2* to 72.5* by translating the distance between fiber tips. In this paper,
the incidence angle is fixed at SPR angle (44.7*) to excite SPR on the flat gold surface. Light from a
continuous wave laser (532 nm, Verdi-10, Coherent) is delivered to the system through a single-mode
polarization preserving optical fiber (Oz optics, Ottawa, Canada). The beam is divided by a beam splitter.
One beam is reflected from a retro-reflecting mirror while the other beam is allowed to go straight
through. These two beams are coupled into two single-mode polarization preserving optical fibers. The
angle of incidence is adjusted by the separation distance between two fiber tips. The divergent beams
from these fiber tips are focused down to the back focal plane of a high NA objective (Olympus Plan Apo
60X NAl.45) of an inverted microscope (Olympus, IX-71) by two lenses. A linear polarizer between the 30 collimation lens and the tube lens is used to set the illumination S- or P- polarized. The beams from the objective interfere and excite the sample either by SW-TIRF (with bare glass coverslip) or by SW-SPRF
(with metal coated coverslip). The SW phase is controlled by a feedback control system consisting of a linear CMOS array detector (S9227, Hamamatsu, Bridgewater, NJ), an embedded microprocessor
(SBC0486, Micro/sys, Montrose, CA), and a piezoelectric transducer (PZT, P-810.10, Polytec PI,
Auburn, MA) attached to a retro-reflecting mirror regulating the optical path length difference between the two beams. The scattered 532 nim excitation light is blocked by the dichroic mirror (z532dc, Chroma,
Rockingham, VT) and further attenuated by a barrier filter (HQ545LP, Chroma, not shown in the Fig.
2.5). The fluorescence emission passes through the dichroic mirror and a relay optics (x16 magnification).
The images are acquired with an intensified CCD camera (Pentamax, Princeton Instrument now Roper
Scientific, Trenton, NJ).
We prepared thin gold film (40 2.3 nm) on coverslips by vapor deposition (EMF Corp., Ithaca, NY) and covered the gold film with a silica layer (SiOx, 5 nm thickness) by the same method as gold to minimize the quenching effect. The thickness of the film was confirmed by AFM (Dimension 3100,
Veeco, measuring the depth of the groove made by a razor blade) and ellipsometry (M-2000, J.A.
Woollam Co.) Diluted solution containing sub-diffraction size fluorescent beads (nominal diameter 0.04 ptm; F-8792, Molecular Probes, OR) was spread on the gold coated coverslips for SPR and on bare coverslips for TIR and dried overnight. We measured the intensity of the beads on the gold coated coverslips for SPR and that of the beads on the bare coverslip for TIR.
2.4. Signal intensity comparison of different imaging modes
Typical bead images of different imaging modes (SPR and TIR in each P- and S-pol) at the SPR angle are displayed in Figure 2.6(a)-(d). Here we compare the imaging result of non-standing wave cases for clarity by blocking the beam coming from one fiber tip. The point-spread function (PSF) of SPR (P-pol)
31 shows a significantly strong signal with unique doughnut-shape. In contrast, SPR cannot be excited with
S-pol excitation, and so no beads appear in Figure 2.6(b).
We quantified the average intensity values for each imaging mode after background subtraction in Fig.
2.6(e). For this analysis, we used the intensity signals within the first maximum ring of the SPR PSF images. The error bar represents the standard error from the statistics of 48 beads. On average, SPR (P- pol) shows more than four times enhancement compared with TIR (P-pol) while TIR (P-pol) shows about
40% higher intensity than TIR (S-pol). Both of the differences in intensity are statistically significant by
Student's t test (p < .00 1).
4000 SPR (P-pol) SPR (S-pol) TIR (P-pol)' TIR (S-pol) (a) 2u (b) (M (d) 3000
-2000
1000
50+5 (e)
4e+5 .0 E 0 1- 39+5-
2e+5
E1I+5 -6
0 < SPR(P pol) SPR(S pol) TIR(P pol) TIR(S pol)
Fig. 2.6. Comparison of various imaging modes: (a)-(d) images of fluorescent beads under different imaging conditions with the same incident angle, excitation intensity, and exposure time. Scale bar, 2 pm; inset, 2.5 pm across. (e) Comparison of intensity of fluorescent beads under various imaging modes (n=-48 each).
32 2.5. Calculation of the transmitted intensity and collected emission photons
30 T16 (a) -SPRF (P-pol) (b)-SPRF (P-pol) --- TIRF (P-pol) 14 - -SPRF (S-pol) S2-SPRF (S-pol) -TIRF (P-pol) - TIRF (S-pol) 12 1g 20 -- .... -... 1. --...... 15 .1 0 --...-. ....
5 - 0
0 20 40 60 80 100 50 100 150 200 250 incidence angle (deg) Distance (nm)
Fig. 2.7. (a) Transmitted intensities on the gold surface with respect to the incidence angle for the cases of SPRF and TIRF with S- and P-polarization incident light. (b) Fluorescence emission intensities collected by objective from fluorophores at different distance from the substrate. (c) AFM image of gold surface (bump view).
This work was mostly done in collaboration with Mr. Wai Teng Tang and Prof. Colin Sheppard of the
National University of Singapore. Based on SPRF theory in Ch. 2.2, we calculated the transmitted intensities above the substrate, i.e. coverslip glass for TIRF and gold film coating for SPRF in Fig. 2.7(a).
The detailed calculation procedure can be found in the published article [20]. The refractive index for the coverslip glass is assumed to be 1.52 and that of the thin gold film coating (thickness 40 nm) is assumed to be 0.32 + i2.8318 [32]. A dielectric medium on the substrate with refractive index 1.0 and 532 nm laser excitation are used. The figure shows that the normalized transmitted intensity of P-pol SPRF at the SPR angle is about three times higher than that of P-pol TIRF at the SPR angle. The emitted fluorescence is usually proportional to the intensity incident on the fluorophores, but there is a known quenching effect near to metal coatings. In our experiment, a 5 nm silica layer has been deposited on top of the gold layer to attenuate the quenching effect [33].
To examine the number of emission photons collected by the microscope objective, we performed a theoretical simulation in Fig. 2.7(b). The result shows the signal collected by the objective lens from fluorophores at varying distance from the substrate. In this calculation, the point dipole is randomly
33 oriented and placed at a fixed distance above the substrate, and the intensity radiated by the dipole is integrated over the aperture of a high NA TIRF objective lens. The SPRF case shows that quenching is present for distances less than 40nm, where near-field interaction between the dipole and the metal causes non-radiative losses and does not contribute to the far-field detection. To our surprise, within the range of distance equal to the bead diameter (0.04 im), the collected intensity of SPRF (P-pol) and TIRF (P-pol) do not differ much, which is contradictory to our experimental results [34]. With known factors such as reduced lifetime, ohmic loss, and reflection caused by the metal film, we speculate one potential reason
for this discrepancy as the assumption of flatness of metal film in the theoretical simulation. It is known that even slight corrugations can change the electro-dynamic properties of the film significantly and a number of studies have been carried out regarding this surface enhancement effect on Raman
spectroscopy, second harmonic generation, and fluorescence [35-38]. We measured the gold surface with
AFM (Dimension 3100, Veeco, tapping mode) and visualized the surface in Fig. 2.7(c). The root mean
square roughness of the gold surface is 2.28 nm. A theoretical study predicted the electric field
enhancement caused by randomly rough surface is about three times intensity enhancement with less rough surface than our sample under similar conditions [39].
2.6. Point spread function measurement of SW-SPRF microscope
The procedure of generating super-resolution images with SW-SPRF in the vertical direction is shown in
Fig. 2.8. One major complication is that the unique doughnut-shape PSF of SPRF [15] in Fig. 2.8(a) does not allow a direct use of the SW-TIRF algorithm due to the central dip and overall widened PSF full- width-at-half-maximum (FWHM). A careful observation of the SPRF PSF profile reveals that all the major peaks in the SPRF PSF have FWHMs comparable with the FWHM of TIRF PSF. Thus it is feasible to apply a deconvolution algorithm to convert the doughnut-shape SPRF PSF in Fig. 2.8(a) into a more conventional Airy disk shape PSF as in Fig. 2.8(b). Based on our numerical model of the PSF, the
Richardson-Lucy algorithm could be adapted with the theoretical PSF kernel as an input [15].
34 A
al OriVSPRF bi* DecorSPRF C, Decon SWaSPRF d eo WSR 000 0 Rol
2000 1000
1600
1400
48a nm 313lnm 1.24 m 12m]0,
a riinz SRFiae it dgh-she PSF-4 (b2 deconSoledSPRF image ( _)_W-PR ,400~ imae ftriapligteS-IFaloihMntred conve SPRF imge, d SW-SPF
0 6 10 1160 2000 2600 0 600 1000 16002000 26000 600 1000 1600 2000 2600 0 600 1000 1600 2000 0 26oo4 y (nm) y(nm) y (nm) y(nm)
Fig. 2.8. Extended-resolution imaging with SW-SPRF microscopy in vertical direction: (al) original SPRF image with doughnut-shape PSF, (bi) deconvolved SPRY image, (ci) SW-SPRY image after applying the SW-TIRY algorithm on three deconvolved SPRY images, (dlI) SW-SPRY image with linear deconvolution to reduce side lobes; (a2)-(d2) comparison of PSF profiles of various imaging methods at a selected region of interest (ROI).
To generate a SW-SPRF image, three intermediate SPRF images (denoted as "Orig SPRY"; one representative image shown in Fig. 2.8 al) are taken at three SW phases (0, 2n/3, and 4n/3) as in SW-
TIRF imaging. Then, the deconvolution algorithm is applied to convert the original doughnut-shape PSFs into PSFs which are single-peaked ("Decon SPRF"; Fig. 2.8 b2) followed by the application of the SW-
TIRF algorithm [40]. These three deconvolved SPRF images are used to generate one enhanced image
("Decon SW-SPRF"; Fig. 2.8 ci). The side band of SW-SPRF in Fig. 2.8 (c2) can be removed with simple linear deconvolution (Decon SW-SPRD) as shown in Fig. 2.8(d2) [16].
The PSF profiles in Fig. 2.8(c2) and 2.8(d2) demonstrate that FWHMs of both SW-SPRF and SW-
SPRD are more than factor of two narrower than that of the deconvolved SPRF PSF in Fig. 2.8(b2) and approximately four times narrower than that of the original SPRF PSF in Fig. 2.8(a2). Though we demonstrate here one directional resolution enhancement as a proof of concept, general two-dimensional imaging with super-resolution in every direction is also possible [17]. 35 2.7. Conclusion
SW-SPRF imaging holds a promise as a high resolution, high signal-to-noise wide-field imaging technique over conventional SPR imaging methods. While surface plasmon-coupled emission (SPCE) imaging has been demonstrated to give better quality images compared to conventional TIRF in the imaging of single muscle fibers [13, 41, 42], it suffers from poor lateral resolution compared with conventional TIRF due to the wider FWHM of the doughnut-shaped PSF. This can be redeemed by deconvolution. Our method goes one-step further by applying the technique of SW-TIRF to enhance the resolution from the typical diffraction limited resolution down to a sub-diffraction limited resolution on the order of 100 nm. It is notable that our SPR standing waves are not limited by the relatively short propagation length of SPR waves since we generate SW-SPR over several hundred microns wide area which eventually will be beneficial for wide area imaging or sensing applications.
References
1. Malmqvist, M., Biospecific Interaction Analysis Using Biosensor Technology. Nature, 1993. 361(6408): p. 186-187. 2. Wilson, W.D., Analyzing biomolecularinteractions. Science, 2002. 295(5562): p. 2103-2105. 3. Lesuffleur, A., et al., Laser-illuminated nanohole arraysfor multiplex plasmonic microarraysensing. Opt. Express, 2008. 16(1): p. 219-224. 4. Singh, B.K. and A.C. Hillier, Multicolor Surface Plasmon Resonance Imaging of Ink Jet-PrintedProtein Microarrays. Anal. Chem., 2007. 79(14): p. 5124-5132. 5. Jordan, C.E., et al., Surface Plasmon Resonance Imaging Measurements of DNA HybridizationAdsorption and Streptavidin/DNA Multilayer Formation at Chemically Modified Gold Surfaces. Anal. Chem., 1997. 69(24): p. 4939-4947. 6. Rothenhausler, B. and W. Knoll, Surface-PlasmonMicroscopy. Nature, 1988. 332(6165): p. 615-617. 7. Yeatman, E., E. Yeatman, and E.A. Ash, Surface plasmon microscopy. Electronics Letters, 1987. 23(20): p. 1091-1092. 8. Hell, S.W. and J. Wichmann, Breaking the diffraction resolution limit by stimulated emission: stimulated- emission-depletionfluorescence microscopy. Opt. Lett., 1994. 19(11): p. 780-782. 9. Gustafsson, M.G.L., Nonlinear structured-illuminationmicroscopy: Wide-field fluorescence imaging with theoretically unlimited resolution. Proceedings of the National Academy of Sciences of the United States of America, 2005. 102(37): p. 13081-13086. 10. Betzig, E., et al., Imaging IntracellularFluorescent Proteins at Nanometer Resolution. Science, 2006. 313(5793): p. 1642-1645. 11. Rust, M.J., M. Bates, and X. Zhuang, Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat Meth, 2006. 3(10): p. 793-796. 12. Somekh, M.G., Surface plasmon fluorescence microscopy: an analysis. Journal of Microscopy-Oxford, 2002. 206: p. 120-13 1. 13. Borejdo, J., et al., Application ofsurface plasmon coupled emission to study ofmuscle. Biophysical Journal, 2006. 91(7): p. 2626-2635. 36 14. Huang, B., F. Yu, and R.N. Zare, Surface plasmon resonance imaging using a high numerical aperture microscope objective. Analytical Chemistry, 2007. 79(7): p. 2979-2983. 15. Tang, W.T., Chung, E., Kim, Y., So, P. T. C., Sheppard, C. J. R., Investigation of the Point Spread Function of Surface Plasmon-CoupledEmission Microscopy. Optics Express, 2007. 16. Chung, E., D.K. Kim, and P.T.C. So, Extended resolution wide-field optical imaging: objective-launched standing-wave total internal reflectionfluorescence microscopy. Optics Letters, 2006. 31(7): p. 945-947. 17. Chung, E., et al., Two-Dimensional Standing Wave Total Internal Reflection Fluorescence Microscopy: SuperresolutionImaging of Single Molecular and Biological Specimens. Biophys. J., 2007. 93(5): p. 1747- 1757. 18. Axelrod, D., Total internal reflectionfluorescence microscopy in cell biology. Traffic, 2001. 2(11): p. 764- 774. 19. Raether, H., Surface-Plasmonson Smooth and Rough Surfaces and on Gratings. Springer Tracts in Modem Physics, 1988. 111: p. 1-133. 20. Tang, W.T., et al., Investigation of the point spread function of surface plasmon-coupled emission microscopy. Optics Express, 2007. 15(8): p. 4634-4646. 21. Tokunaga, M., et al., Single molecule imaging of fluorophores and enzymatic reactions achieved by objective-type total internal reflection fluorescence microscopy. Biochemical and Biophysical Research Communications, 1997. 235(1): p. 47-53. 22. Webb, S.E.D., et al., Multidimensionalsingle-molecule imaging in live cells using total-internal-reflection fluorescence microscopy. Optics Letters, 2006. 31(14): p. 2157-2159. 23. Arnoldus, H.E. and J.T. Foley, Transmission of dipole radiation through interfaces and the phenomenon of anti-criticalangles. Journal of the Optical Society of America a-Optics Image Science and Vision, 2004. 21( 6 ): p. 1109-1117. 24. Richards, B. and E. Wolf, ElectromagneticDiffraction in Optical Systems .2. Structure of the Image Field in an Aplanatic System. Proceedings of the Royal Society of London Series a-Mathematical and Physical Sciences, 1959. 253(1274): p. 358-379. 25. Wolf, E., ElectromagneticDiffraction in Optical Systems .1. An Integral Representation ofthe Image Field. Proceedings of the Royal Society of London Series a-Mathematical and Physical Sciences, 1959. 253(1274): p. 349-357. 26. Gustafsson, M.G.L., Surpassing the lateralresolution limit by a factor of two using structured illumination microscopy. Journal of Microscopy-Oxford, 2000. 198: p. 82-87. 27. Bailey, B., et al., Enhancement of Axial Resolution in Fluorescence Microscopy by Standing-Wave Excitation. Nature, 1993. 366(6450): p. 44-48. 28. Chung, E., et al., Two-dimensional standing wave total internal reflection fluorescence microscopy: Superresolution imaging of single molecular and biological specimens. Biophysical Journal, 2007. 93(5): p. 1747-1757. 29. Frohn, J.T., H.F. Knapp, and A. Stemmer, True optical resolution beyond the Rayleigh limit achieved by standing wave illumination. Proceedings of the National Academy of Sciences of the United States of America, 2000. 97(13): p. 7232-7236. 30. Goodman, J.W., Introduction to Fourieroptics. 3rd ed. 2005, Englewood, Colo.: Roberts & Co. xviii, 491 p. 31. Gu, M., Advanced OpticalImaging Theory. 2000: Springer, Heidelberg. 32. Palik, E.D., Handbook of Optical-Constants. Journal of the Optical Society of America a-Optics Image Science and Vision, 1984. 1(12): p. 1297-1297. 33. Hellen, E.H. and D. Axelrod, Fluorescence Emission at Dielectric and Metal-Film Interfaces. Journal of the Optical Society of America B-Optical Physics, 1987. 4(3): p. 337-350. 34. Enderlein, J. and T. Ruckstuhl, The efficiency of surface-plasmon coupled emission for sensitive fluorescence detection. Optics Express, 2005. 13(22): p. 8855-8865. 35. Albrecht, M.G. and J.A. Creighton, Anomalously Intense Raman-Spectra of Pyridine at a Silver Electrode. Journal of the American Chemical Society, 1977. 99(15): p. 5215-5217. 36. Chen, C.K., A.R.B. de Castro, and Y.R. Shen, Surface-EnhancedSecond-Harmonic Generation. Physical Review Letters, 1981. 46(2): p. 145-148. 37. Kneipp, K., et al., Single molecule detection using surface-enhanced Raman scattering (SERS). Physical Review Letters, 1997. 78(9): p. 1667-1670.
37 38. Zhang, J. and J.R. Lakowicz, Metal-enhancedfluorescence of an organicfluorophore using gold particles. Optics Express, 2007. 15(5): p. 2598-2606. 39. Arya, K. and R. Zeyher, Optical-Response from a Randomly Rough-Surface - Theory and Numerical Results. Physical Review B, 1983. 28(8): p. 4090-4105. 40. So, P.T.C., H.S. Kwon, and C.Y. Dong, Resolution enhancement in standing-wave total internal reflection microscopy: a point-spread-functionengineering approach. Journal Of The Optical Society Of America A- Optics Image Science And Vision, 2001. 18(11): p. 2833-2845. 41. Burghardt, T.P., et al., In Situ Fluorescent Protein Imaging with Metal Film Enhanced Total Internal Reflection Microscopy. Biophys. J., 2006: p. biophysj. 105.079442. 42. Borejdo, J., et al., Fluorescencecorrelation spectroscopy in surface plasmon coupled emission microscope. Optics Express, 2006. 14(17): p. 7878-7888.
38 Chapter 3
Theoretical framework for three-dimensional wide-field pump-probe
structured illumination microscopy
3.1. Introduction
The development of high-sensitivity pump-probe microscopy has enabled molecular imaging and characterization in the fields of materials science, biomedicine, and art conservation [1]. Many pump-probe microscopy modalities are based on higher order optical processes, such as transient absorption (TA) and stimulated Raman scattering (SRS), require high peak energy excitation. As a result, typical implementation of pump-probe microscopy involves tight focusing of combined "pump" and "probe" laser beams and scanning the localized focal spot to map out the whole sample volume. Using this well-established approach,
SRS and TA based pump-probe microscopy has demonstrated high speed imaging with sub-micrometer scale spatial resolution in visualizing the fine structures of cells and tissues with molecular specificity [1, 2].
However, the optical resolution of microscopy is limited by the wave-like characteristic of the light, more specifically diffraction [3]. Many interesting biological and chemical structures are smaller than this diffraction limit. There are many recent advances in overcoming this diffraction-limited resolution. These efforts have been mostly focused on fluorescent imaging utilizing unique properties of fluorescence processes.
One group of these technologies utilize the photoswitchable fluorophores and the centroid of the individual emitter is localized based on stochastic nature of fluorophore activation including photoactivated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM) [4-6]. Other super- resolution technologies use reversible saturable fluorophores and patterned excitation to decrease the fluorescence emission volume for resolution enhancement. Reversible saturable optical fluorescence transitions (RESOLFT), Stimulated emission depletion (STED), and ground-state depletion (GSD)
39 microscopies are representative technologies in this group [7-9]. Structured illumination microscopy (SIM) is a super-resolution method that also relies on patterned excitation, but is not limited to specific fluorophores
[10-14]. SIM illuminates the sample with spatially structured light to shift the high frequency information into the passband of the imaging system.
While significant progresses have been made in fluorescence imaging, the advances in achieving super- resolution in pump-probe microscopy are more limited. For sub-diffraction resolution, Wang et al. applied
RESOLFT to TA based pump-probe microscopy to image graphene and graphite nanoplatelets [15] and Silva et al. adopted STED concept to SRS based pump-probe microscopy to image diamond [16]. These approaches can only be applied to the sample that can withstand high enough flux of laser beam to generate
'saturated transient absorption' for TA and 'decoherence' for SRS. Lu et al. developed a super-resolution imaging method based on spatiotemporal modulation and it can readily be adopted to the pump-probe microscopy with general samples [17]. However, the imaging speeds of these methods are limited because of its point scanning configuration. As an alternative method to point scanning, there have been publications to implement wide-field pump-probe microscopy based on coherent anti-Stokes Raman scattering (CARS)
[18-20]. People have also applied SIM to pump-probe microscopy to increase the lateral resolution in two dimensions in various configurations including point scanning, line scanning and wide-field [21-24]. In this article, we extend these previous works to three dimensions (3D) in wide-field configuration and propose a scheme that is compatible with pump-probe modalities where signal wavelength is not shifted from those of input beams (for example, signal wavelength is anti-Stokes shifted for CARS, but it is not for SRS). We provide the theoretical framework to implement 3D SIM pump-probe microscopy in the wide-field configuration. To validate the proposed method, numerical simulations are carried out on a planar and a non-planar resolution targets, and 3D HeLa cell data from the online public source [25-27]. The results demonstrate that our scheme for 3D SIM pump-probe can provide three-dimensional imaging with nearly three times higher lateral resolution than that of conventional pump-probe imaging.
40 3.2. Principle of 3D SIM pump-probe microscopy
3.2.1. Configuration of a wide-field pump-probe structured illumination microscope (ppSIM)
We consider a wide-field pump-probe microscope where a sample is illuminated with structured pump beams in Fig. 3.1(a). Pump-probe signal is detected as a small change of probe beam and the signal is a
function of pump beam intensity [28]. The pump and probe beams are arranged in a collinear geometry
and focused at the back aperture of the microscope objective resulting in a uniform illumination of the objective field of view for the probe beam and a structured illumination for the pump beam. The transmitted
signal is imaged by a second objective. A sharp long-pass filter (not shown in Fig. 3.1(a)) separates the pump beam from the probe beam. For lock-in detection of the probe signal, a light chopper is used to gate the pump beam at 50% duty cycle. This modulation transfers from the pump beam onto the probe beam by the nonlinear light-matter interactions and the probe beam passing through the sample object is detected by a camera. The frame rate of the camera will be set at four times higher frequency than the chopper frequency to allow easy extract of the lock-in signal based on homodyne detection. The chopper modulation and the camera sampling are synchronized by phase-locked loop and successive waveforms are synchronously averaged to suppress non-harmonic noise. The pump-probe signal which is directly proportional to the molecular concentration is related to the AC/DC ratio as shown in Fig. 3.1(b) [29]. To detect electromagnetic phase and amplitude of the probe beam, which are needed for image reconstruction, phase- shifting holography technique is adopted [30]. The probe beam is divided into three paths. One beam passes through the sample object, which is being illuminated by the structured pump beam, (Probe 1 in Fig. 3.1(a)).
A second beam (Probe 2 in Fig. 3.1(a)) is delayed by the piezoelectric transducer mirror for phase shifting and split into two. A third probe beam (Probe 3 in Fig. 3.1(a)) freely propagates and it is interfered with a part of Probe 2 on the photo-diode surface to calibrate the motion of the delay mirror. The other part of
Probe 2 acts as a reference wave for Probe 1 and the phase-shifted interference patterns between two beams are recorded by the camera.
41 (a) Pump beam Chopper 1 Grating --- a- Modulated probe beam Probe beam Sample |
h Probe Comera Probe 2M
Probe 3 Photo-diode (b) | V3 DC= VO + V1 + V2 + .a) 4 2 [(Vo - V2) +(Vl - V3)] AC= VO VI V2 V3 VO V1 V2 V3 Time DC
Fig. 3.1. Wide-field pump-probe structured illumination microscope (ppSIM) design: (a) Schematic diagram of an optical setup (b) Synchronous sampling of probe beam signal at four points per sinusoidal wave form in each pixel of the camera (directly applicable to 50% duty ratio on-off modulation).
I represent the first probe beam (object wave) complex amplitude on the camera plane as U(x,y) and the second probe beam (reference wave) as UR (OR). Then the interference intensity captured by the camera is expressed as
I(x,y;#OR (R )+U (x,y) 2 )=UR (3.1) -AR 2 +A+2ARA cos(R-0)
With the intensities with stepped phase, we can derive the object phase as
I(x, y;3; / 2)-I(x,y;;r / 2) (3.2) I(x,y;O)-I(x,y;;r) where the initial reference phase is assumed to be zero. The amplitude A(x,y) of the object wave is derived by blocking the reference wave [30]. To retrieve a 3D object information with a 2D detector, we shift the
42 object in the axial direction and get a 3D data stack. I will provide an explanation on the details about getting the 3D data stack and the 3D coherent transfer function in a 4f system in the next chapter.
3.2.2. Theoretical framework
3.2.2.1. 3D coherent transfer function and getting a 3D object information with a 2D detector
Here, I mathematically derive the 3D coherent transfer function in the 4f system that usually consists of two positive lenses with the input plane located one focal length (fl) in front of the first lens and the output plane located one focal length (f2) after the second lens. There is a derivation for the 3D coherent transfer function for the single lens imaging system in the literature [31] and I will extend its methodology for the
4f system. However, we will use the terminology including the phasor direction for electric field and the various transforms from a different publication [32].
I will use x, y, z for spatial coordinates in the Cartesian coordinate system, spatial frequencies m, n, s in the x, y, z directions for the Fourier spectrum. Radial and angular coordinates in the polar coordinate system are defined as:
r=X2+y21x =rcos0, y =rsin0 (3.3) {=(M2 +n2)1/2 in=/cosip, n-Isin (
T{ } represents the Fourier transform and capital lettered function F is the Fourier transform of small lettered function f [32] as
F {, f (x,y,z)exp[-i2r (mx + ny + sz)]dxdydz. (3.4)
43 ~1
U1(x1,y1) U 2(X2,y2 )
y1 yX
z
Fig. 3.2. Definition of the diffraction plane (the x, - y, plane) and the observation plane (the x2 - y, plane).
Using scalar diffraction theory and the paraxial approximation in Fig. 3.2, the light field in the x2 - y 2 plane
(z distance far away from the x, -y, ) can be calculated as
-)2 2 di (3.5) U2 (X 2 ,y2 )= exp(I (Iy,)exp)U ( which is the Fresnel diffraction integral [32]. With the same method, the complex transmittance of a thin lens can be presented as
Fik(x2 +y2) t(x,y)= exp - 2 (3.6) 1 2f
U1(x1 yy,z1) U 2(x2,Y2) P(X3,y3) U4 (X4,1Y4) Y1 d, Y2 X22 3 X 40X
IVZ3___i ____
object detector
f1 f1 f2 f2
Fig. 3.3. 4f optical imaging system with a thin object.
We apply the previous diffraction integral to the 4f optical imaging system with two lenses in Fig. 3.3.
There is a thin object placed in a plane at a distance z, from the back focal plane of the first lens. Let's assume that we have an area type detector that can measure the light field and the it is placed in the front
44 focal plane of the second lens. There is a pupil mask with transmittance p(x,,y,) in the front focal plane
of the first lens (so called Fourier or pupil plane). The pupil mask determines the maximum spatial
frequency that the optical system can pass and finally the spatial resolution of the optical system. The field
in the detector plane (x 2 - Y 2 plane) can be presented as a convolution integral like
exp[2ik(f +f2)] ,y ,z, )dxidy, (3.7) U4 (x 4 ,y4 ) = A2 exp(-ikz,) o(x,,y)h(x +Mx 4 +M 4
with a new function h(x,y,z) and demagnification factor M as
h(x,y,z)=P' x, 1y,z I ( Af Af,(3.8) M= , X4
where P' is the Fourier transform of modified pupil function p' y3 , z)= x k x (x, y )
U1(x1,y1,z1) U2 (X 2,y2) P(X3,y3) U4 (x4,y4)
Y2 2 3 Y -- d, Z2Z3 z4 __ V
object detector
f, 04 f1 - f2 f2
Fig. 3.4. 4f optical imaging system with a thick object.
Let us now apply the result to an object with a finite thickness in Fig. 3.4. In this case, the object function
is a 3D function o (x, y1 , zi). For each of the vertical sections in the thick object at a given position z, , its
image in the image plane is given by Eq. (3.7). The total field in the image plane is the superposition of the
contributions from the images of all sections. The superposition principle holds only if secondary diffraction
in a thick object is neglected and if the object is semi-transparent. This assumption is called the first Born
45 approximation. Under this approximation, the image of an object with finite thickness, i.e. the image of a 3D object, is the integration of Eq. (3.7) with respect to z, . The final image field in the image plane is
exp[2ik(f + f)] U4 (X4,IY4)= A~2 2 ff2 (3.9) o(x, ,y, z, ) exp(-ikz, )h(x, + Mx 4 ,y1 + My 4 ,zl )dxldyldz1
To retrieve a 3D object information with a 2D detector, we need to scan the object in the axial direction in Fig. 3.5 and the light field in the image plane with a shifted object is represented as:
exp[-2ik(f, + f)] U4 (X4,IY4,IZ5) A2 2f2 (3.10) 0o(x,,y 1 z,, -z )exp(-ikz)h(x, +Mx4,y1 +My4,zl)dxidydz,
exp [-2ik (f + f 2)] ,-y - My ,-z -z)dxdy dz, =2f J f2 0(XIIY 1 ,z1)h'(-x - Mx 4 4 1 with a change of a integration variable in z direction and a new function h'(x,y, z) = exp(ikz)h(-x, -y, -z)
. Therefore, the acquired image field is the 3D convolution of the object function with the point spread function
h'(xy, z). Here the negative sign before the coordinate variable (x 4 ,y 4, and z, ) inside the point spread function h' implies that the image is inverted. The imaging is 3D space-invariant with a transverse magnification factor 1/M and a unity axial magnification.
3D object Shifted 3D object o(x ,y,,z,) o(x ,y,,z - ) 1 1 1 5 U 4 (x4,y4 )
z Z2 Za Z4
detector
f1 . f1, f2 '. f2 .
Fig. 3.5. 4f optical imaging system with a shifted thick object.
46 Performing a 3D Fourier transform on the point spread function h'(x,y,z) gives the 3D coherent transfer function (CTF) in object space:
c(m,n,s) = Jff h'(x,y,z)exp[-i2r(mx +ny + sz)]dxdydz ff 1".. (3.11) f "[J f h'(x,y,z)exp[-i2r(mx+ny)]dxdyjexp(-i2rz)dz
Here we are considering a circular lens and pupil function, h'(x, y, z) shows a circular symmetry, which means that it is independent of 0 .
With circular symmetry and the polar coordinates, 3D CTF becomes
c(1,s)=(Af,)2 p(l) s -- +-- (3.12) A 2 where p(1)= b NA/2 (NA: numerical aperture of the optical system) and 5( ) is a delta function. 0, 1 > NAI A
This 3D CTF is an axially shifted cap of a paraboloid of revolution about the s axis in Fig. 3.6(a). However, this axial shift of the 3D CTF comes from the wave vector of the incident light and the 3D CTF in Fig.
3.6(a) is a 3D Fourier transform of the diffracted light over the object. Therefore, the effective 3D CTF to measure the pass/block of the object information passes the origin without the axial shift in Fig. 1(b) [33].
The 3D CTF for a circular lens should be a cap of a sphere (i.e. the Ewald sphere). This difference is caused by the use of the paraxial approximation in the beginning of the derivation [31].
(a) S (b) S c(l,s) (1,s)
K K6 K ,, w Kobed
Fig. 3.6. Schematic diagram of the 3D coherent transfer function for coherent imaging with a circular lens in 4f system. (a) 3D Fourier transform of the diffracted light. (b) 3D CTF of the object. The object diffracts the incident wave and the resulting diffracted wave is captured by the imaging system. The vector K represents a characteristic wave vector of the incident wave, the object as a grating, and the diffracted wave.
47 3.2.2.2. Principle of 3D wide-field pump-probe structured illumination microscope
(a)( x (b)
S
...... m
(dl) (e)
(d2) (e2)
(d2) (e2) Lmm
(3)
Fig. 3.7. Pump beam configuration and probe beam extended transfer function for pump-probe structured illumination microscopy (ppSIM). (a) The five wave vectors corresponding to the each pump beam direction. All five wave vectors have the same magnitude k = 24r/A . (b, c) The
resulting spatial frequency components of the illumination intensity for the ppSIM with (b) a single grating period and (c) grating period scanning. (d-e) The transfer function for (d) the conventional wide-field microscopy, (e) the single grating period ppSIM, and (f) grating period scanning ppSIM in (1) 3D, (2) mn plane, and (3) ms plane. The color in 3D transfer function represents the position in s axis, not a weighting.
In this chapter, I will discuss the pump-probe microscopy of which light-matter interaction by the pump beam excitation is linearly proportional to the intensity of the pump beam. For example, the exact relationship between the probe beam signal and the pump beam intensity is highly non-linear for stimulated
Raman scattering (SRS), but it can be linearized when the pump beam intensity is relatively low [34].
Because of this intensity dependency, the pump beam illumination is incoherent whereas the scattering of the probe beam is coherent. This is similar to the previous work about SIM coherent anti-Stokes Raman scattering (CARS) microscopy in two dimensions [22]. I develop a theoretical framework to reconstruct a
48 3D super-resolved image in the wide-field structured illumination pump-probe microscopy based on coherent imaging theory [22, 35] considering incoherent pump beam excitation.
Assuming unit magnification without loss of generality, the image field in a coherent imaging system with a uniform probe beam can be described as
u(x,y,z) =[o(x,y,z)e(x,y,z)] Oh(x,y,z) (3.13) where (x, y, z) is a spatial coordinate, u(x,y, z) denotes the image field at position (x, y, z), o(x,y, z) is the pump-probe active chemical species concentration distribution in the object, e(x, y, z) is the intensity distribution of excitation pump beam, h(x,y,z) is the coherent point spread function (PSF) defined by the system's limiting circular aperture, and the symbol 0 stands for the convolution operator.
After Fourier transforming the image field, the image spatial frequency distribution can be described as
U(m,n,s) = [O(m,n,s)OE(m,n,s)]H(m,n,s) (3.14) where (M, n, s) is a spatial frequency coordinate corresponding to the spatial coordinate (x, y, z) and H is called as a coherent transfer function (CTF). I denote Fourier-transformed entity by changing from lowercase to uppercase font in this thesis. The resolution of a microscope is limited by the "support" or
"observable region" of the CTF, the region of three-dimensional reciprocal space (spatial Fourier space) where the CTF has nonzero values in Fig. 3.7(d-f). Our pump-probe SIM has a larger effective CTF support
(Fig. 3.7 e, f: 1~3) than the normal observable region with uniform illumination (Fig. 3.7 d: 1-3) by shifting higher-resolution information into the CTF support using structured illumination (Fig. 3.7 a-c), also known as spatial frequency mixing [13].
For our SIM pump-probe microscopy, the sample is illuminated by five pump beams of which one beam propagates parallel to the optical axis and four beams cross the sample at equal incidence angles 0 in Fig.
3.7(a). For simplicity, only the case of S-pol in xz plane is considered. All five waves have the same wavelength and polarization parallel to y axis. In S-pol geometry, the five plane waves can be described as
49 (x, z) = jy'exp[i(k sin Ox+k cosOz +0)] iz(x,z)=5^exp[i(-ksinOx+kcosOz +02)
' (y, z)= j cos exp[i(ksin0y +kcos z+#3 -2sin exp[i(ksin0y+kcosoz++3)] (3.15)
24(y,z) = ^cos0exp[i(-ksin y +kcosz + 04)] + 2sin0exp[i(-ksin y + kcosz + 04)]
e, (z) =exp[i(kz +05 )] where j, denotes the field vector of each plane wave, the amplitude of each incident plane wave is assumed to be unity for simplicity, and ^ and 2 are unit vectors in each y and z directions. The pump beam wave vector is k = 2r / A assuming air as a propagation medium and A is the wavelength of the pump beam.
The phase of each wave is denoted as 0,. Since the temporal dependence is irrelevant in terms of average intensity finally measured by the camera, it is neglected here. The structured pump beam intensity for S- pol interference pattern can be calculated as
e(x,y,z)=EZ,+Z2 + t+ 4+ 51 =[5 +2cos(2ksin0- x+ A012 ) +2cos0cos(ksin0.x-ksin0y+ A0 1 3 ) +2cos0cos(k sin -x +k sin 0 y + A014 ) +2cos(ksin0 -x+kcos0 -z -kz+ A0 1 5 )
+2cos0cos(-k sin0 x -k sin0 -y + A023 ) (3.16)
+2cos0cos(-k .x+k sin0 sin60 y + A024 ) +2cos(-k sin0 -x + kcosO - z -kz + A025 ) +2cos 20cos (2k sin0 - y+ A034 ) +2cos0cos(ksin0 -y+k cos0 -z -kz +A0 35 ) +2cos0cos(-k sinG- y+kcos0 -z -kz+ A0 45 ) where A#, =0 - 0.
By Fourier transforming this pump beam intensity, F e(x,yz) , we obtain the pump beam spectrum in the spatial-frequency domain as
50 E (m, n,s)=.F Ie (x, y, z)= 5,(m,n, s)
+exp(-iAq1 2)3(m+ 2km,,nIS)
+exp(iAq1 2)8(m - 2k,,,n,s)
+cosO0exp(-iA 0,)+ exp(iA 24 g)} (m+k,n n-kmns)
+cos 0exp(iAf )+exp(-iA#2)}g(m-k,,,,n3 +km,,,s)
+cosO(exp(-iA#1 4 )+ exp(iA# 3 )15(m+kmn,n +kmn,s) (317)
+cos 0exp(iA#1 4)+exp(-iA 23 )IS(m -k,mn n-kmn,,S) +{exp (-iA#1 5)3(m+ km, n,s + k, )+ exp (iAA5 )5(m- kmn, n,s - k, +{exp(-iA# 5 )8(m-k,,,n,s +k,)+ exp(iA#25)S(m+ k,, n,s -k )} +cos20{exp(-iA# 34)6(m,n +2k,,s)+exp(iA#b 4)S(m,n -2kmn,s)}
+cosOjexp(-iA# 35 )5(m,n +kn,s + k,)+ exp(iA#35)6(m,n - kmn,s - k)
+cos0(exp(-iA$ 45)t(m,n -kmn,s+k,)+ exp(iA#45)5(m,n +kmn,s -k,)I where k, ksinO sin 0 and k, = kcos0-k cos0-1 . Interference among the five pump beams produces a three-dimensional excitation intensity pattern that contains seventeen Fourier components at each difference frequency between the two illumination wave vectors in Fig. 3.7(a, b).
Substituting the previous result into Eq. (3.14), and mathematically expanding, we get the corresponding image field in the Fourier space as
51 U(m,n,s) = [50(m,n,s)
+exp(-iAO1 2)O(m+ 2k,, n, s)
+exp(iAA 2)O(m -2kn,,nS)
+cos0(exp(-iA# 3)+ exp(iA#4)O(m+k ,n-k, kS)
+cosO(exp(iAA13)+exp(-iA# 24)}O(m -k,,n + km,s)
+cos6{exp(-iAA4 + exp(iA23 )}O(m+k, ,n+ k,,, s)
+cos8{exp(iA1 4 )+exp(-iA# 23)1O(m -k,,m,,n-kmnIS) +exp(-iAOs)O(m+km,, n,s + k,) + exp (iAO15)O(M - km ,n, s - k, )
+exp(-iA2 5 )O(m -k,,, n,s +k,)
+exp(iAb 2 5)O(m+ km, n,s -k,)
+cos20exp(-iA#3 4)O(m,n + 2kmn,)S)
+cos20exp(iAb 34)O(m,n - 2kmn,,S)
+cos0exp(-iA 35)O(m,n + kmns + k)
+cos6exp(iA# 35 )O(m,n -km,s -k,)
+cos exp(-iA#45)O(m,n -kmns +, k (3.18)
+cos exp(iA# 45 )O(m,n +kmns -k)]H(m,n, s)
Eq. (3.18) can be written as a linear combination of 17 spatial frequency shifted object information passing through the imaging system's passband as,
16 U (m, n,s)=L wjO (M9 n,s) H (m, n, s) (3.19) i=O where wi refers to the multiplied terms in front of shifted object information Oi and system's transfer function H in Eq. (3.19) and it is a function of the phases of structurally illuminated pump beams except the first one. This equation shows that the measured data gains new information that is not observable with a conventional uniform illumination microscope: the spatial frequency translation of the original object 0 moves new information into the support of H.
Although an image data captured by the pump-probe SIM system contains more spatial frequency information compared to the conventional wide-field pump-probe microscope, high frequency spatial information is translated and mixed with other components in the raw data. To restore the data, these
52 infonration components must be separated and shifted back to the original position in the spatial Fourier space. For this image reconstruction, at least 17 raw image data U(M,n,s) with corresponding
independent phase set (D, =(Ab 12 ,A 3 ,...,Aqe) i =0,1,...,16 must be acquired because there are 17 spatial frequency shifted image components as unknowns. Then we construct a 17x17 matrix W that describes the spatial frequency mixing of each components in the spatial Fourier space,
U(m,n,s)l =W5 O)0(m,n,s)H(m,n,s) . Given a set of (c, =(AA 2 ,A b1 ,...,AO )i=0,1,...,16 satisfying the condition det W w 0, we are able to use its inverse matrix W to unmix multiplexed
SI components by carrying out an operation O (m,n,s)H (m,n,s) = W1 U(m,n,s) with measured 3D raw image data. After shifting back these unmixed enhanced-resolution components to the original position in spatial-frequency space (Fig. 3.7 e: 1~3), inverse Fourier transform gives us a final enhanced-resolution object information. Although mathematics alone does not dictate the choice of the phase set
choice will {'1= (AO 2 ,Ab 13,-.,A 4 5 )ji=0,1,...,16} except the condition det [W w)0 , a poor compromise final image signal-to-noise (S/N) level because each position of the sample will get different dose of light [36]. The ideal choice of structured illumination phases will be further discussed in the following chapter.
I explained how the wide-field pump-probe SlIM can provide enhanced-resolution object information through shifting object information using a structured illumination with the conventional CTF. The effective CTF of the pump-probe SIM is the convolution of the seventeen-dot illumination structure of Fig.
3.7(b) with the conventional CTF support in Fig. 3.7(d: 1~3), resulting in the region shown in Fig. 3.7(e:
1~3). This region fills in the missing information in 3D that gives us a maximum factor of three lateral resolution extension and at the same time provides limited axial resolution from partially filled 3D Fourier space. The procedure can be repeated with additional illumination patterns to more densely fill the Fourier
53 space. In this thesis, I suggest scanning grating period with a configurable grating device like a spatial light
modulator or a digital mirror device. Fig. 3.7(c) shows the illumination structure using sixteen grating
periods. Whereas the CTF for a single grating period coarsely fills the Fourier space and the space between
CTF surfaces is empty in Fig. 3.7(e: 1-3), this empty space is effectively filled in when multiple grating
periods are used in Fig. 3.7(f: 1-3).
3.2.2.3. Ideal choice of structured illumination phase set for 3D wide-field pump-probe structured
illumination microscope
I discuss the ideal choice of structured illumination phase set ( each position of the sample to get the equal dose of light so that the final image signal-to-noise (S/N) level is uniform over the field of view. Let's start with one dimensional two beam SIM in Fig. 3.8(a) to grab an idea for the ideal phase set. In S-pol geometry, the two plane waves can be described as J (x,z)= jexp[i(ksin~x+kcos~z+$)] Y (3.20) F2 (x,z)= ^exp[i(-ksin~x+kcosz+#2 )] (a) (b) Tm (c) Tm 2 x s s 3.-,3 ~Re + Re . A2 3 - 2 Fig. 3.8. (a) The two wave vectors corresponding to the each pump beam direction. Two wave vectors have the same magnitude. (b) Three complexI numbers equally distributed on the circle whose center is located in the origin. (c) Arbitrary number of complex numbers equally distributed on the circle whose center is located in the origin. The structured pump beam intensity for S-pol interference pattern can be calculated as e(x, y,z) =It, +E212 =I +2cos (2k sin 0 -x+A0 2 ) (3.21) 54 where A5 2 = # - . By Fourier transforming this pump beam intensity, we obtain the pump beam spectrum in the spatial-frequency domain as E(m, n, s) =(m, n) + exp(-iAO6j 2 ) (m+ 2k,, .,n,s) + exp(iA1 2 ) 8(m - 2k,,,, n,s). (3.22) We need 3 phase shifts of structured illumination, (D, =(AA 2),i = O,1,21 because there will be 3 copies of sample information from this structured illumination. The total dose of light that the sample will get with 3 phase shifts is L +2Lcos(2ksinO- x+(A1 2 )). (3.23) The ideal phase shift equally spans the range of 0 - 2r [36] as (D, =_(Ae ) i = 0, 1, 2 = 0, , . (3.24) 1 3 3 This can be easily understood using Euler's formula to convert each cos( ) term into a matching complex number just like the phasor notation and complex number addition in two dimensional space in Fig. 3.8(b). Three complex numbers are equally distributed on the circle whose center is located in the origin. The sum of these three complex numbers with any arbitrary angular offset from Fig. 3.8(b) configuration is 0 because of the symmetry and this result doesn't change with an arbitrary number of complex numbers in Fig. 3.8(c). We can describe this result for our purpose as N-1 cos C+i 7 0 (3.25) i=O N ) where C is any arbitrary angular offset constant, M is an integer except 0, and N is a natural number. Let's go back to our original five beam SIM configuration. The total dose of light is the summation of Eq. (3.16) over 17 different phase set as 16 16 16 5 + 2Lcos(2ksinO- x+ AA2 ) + 2cosO cos(ksin -x- ksinO- y + A13 ) i==O i=O iO . (3.26) +--+ 2cosO cos(-ksinO -y + kcosO -z -kz + A04 5 ) 55 Our goal is to find the phase set which makes above equation constant over the position of the sample and we know the following phase set satisfies that condition from Eq. (3.25): Di -(A1,A#31 )ii=0,1--,l.,164 MA5 6 ( . 7 = ( {~2Mr i 2M13 ,r ,71...,2M17,) 1 i i = 0 1...16, M.M j,(~0) E(Z - 101)(3.27) From the definition of A# =#, - #in Eq. (3.16), we can get a simpler form of the phase set j(# ,#2'-- -,j ) i =0, 1,...,1161 ={ , ,...,r=,i2M2M . (3.28) jC 17 , 17 1..I 17 ) = 11..16 where one of the phases #,,#2,..-,05 can be a fixed reference phase, so Mje Z instead of M, e (Z - {0}) Then we can find many phase sets satisfying det [ w 0 with systematic trial and error. I used the following phase set for our simulation in this paper: j(#,02'---, 0 ) i=0,1,..., = 0,i ,i - ,I I i = 0,1,...,16 (3.29) 3.3. Methods for numerical simulation I performed numerical simulations to validate the theoretical framework to obtain super-resolution 3D resolved pump-probe microscopy with structured illumination. To make the validation more meaningful in the practical context, simulation parameters were taken from a typical case of stimulated Raman scattering (SRS) experiments. We assumed a nonlinear sample providing a pump-probe contrast similar to SRS at the Raman shift of 753 cm-' (corresponding to the pyrrole breathing mode V15 of cytochrome c) that is resonantly excited by a 755 nm pump laser and a 800 nm Stokes laser [37]. For simplicity, we used the same numerical aperture for both pump beam (755 nm) excitation and probe beam (800 nm) detection in the transmission stimulated Raman gain geometry: we used low NA (0.68) to provide a general explanation 56 about our pump-probe structured illumination microscopy (ppSIM) with artificial samples and compared the performance between the low NA (0.68) and the high NA (1.0) for an actual biological sample data. 3.4. Results 3.4.1. Pump-Probe Structured Illumination Imaging of a planar target: Calibration Chart To verify the theory, we show a simulated pump-probe structured illumination reconstruction of 1951 USAF test chart images as our pump-probe active sample in Fig. 3.9. The 3D test target is built to represent a sample distribution over the volume of 40 ptm x 40 pm x 40 pm (in 512 x 512 x 256 image pixels) and total 10 different grating period (1.11 pm -1 1.10 pm) is used. First, we compare the imaging result of a whole USAF target image which has a fitting pixel size to the computational field of view in lateral dimension and a single pixel thickness [27]. The lateral resolution improvement in the pump-probe SIM image (Fig. 3.9 bl and cl) can be clearly seen overall when compared with the conventional wide-field image (Fig. 3.9 al). We also note that the image field in xz plane from the single pixel layer object is not localized near the sample location and generates extended interference pattern along axial dimension for the conventional wide-field microscope in Fig. 3.9(a2) and for the ppSIM with a single grating period in Fig. 3.9(b2), whereas the corresponding grating period scanning ppSIM field in xz dimension shows localized sharp peaks around the sample plane and more moderate interference pattern in Fig. 3.9(c2). These observation also can be confirmed with Fourier domain field amplitude in both lateral and axial dimensions: the support of the transfer function for grating period scanning pump-probe SIM (Fig. 3.9 c3, c4) is increased in both lateral and axial dimensions compared with that for conventional wide-field microscopy (Fig. 3.9 a3, a4). We also notice that the grating period scanning ppSIM (Fig. 3.9 c3, c4) fills the Fourier space more densely compared with the single grating period ppSIM (Fig. 3.9 b3, b4). The pump beam intensity dependence characteristic of our pump-probe SIM provides the effective lateral frequency support of 2NA,,. / ,pmp+ NApro /lprb, which is more than 3 times increased in the current simulation condition compared with coherent Abbe frequency limit ( NAP,,,,/Apro,, ) in Fig. 3.9(a3, b3). The 3D transfer 57 function of the conventional coherent wide-field microscope looks like a thin shell (Fig. 3.8 dl) and the bandwidth in axial dimension is very narrow in Fig. 3(a4). On the other hand, the bandwidth of our pump- probe SIM is extended to ~500 lpmm (lines per mm) in the axial dimension (Fig. 3.9 c4) corresponding to the localized imaged field in Fig. 3.9(c2). For a more quantitative analysis about resolution in space, we synthesize isolated three bar patterns which follows the MIL-STD-150A standard about USAF 1951 resolution target [38] and compare the image performance between the conventional wide-field microscope and the ppSIM with grating period scanning. We adapted the procedure in the article about coherent SIM in scattering mode [35] to measure their corresponding field amplitude modulations, defined as m = - . (3.30) Amax + Am.i Here, Am and A. are the maxima and minima, respectively, of the image field amplitude of test bars. Fig. 3(d) and (f) show the field amplitude intensity profiles taken from the images of the lateral and the axial bar patterns. We plot the simulated field amplitude modulations observed in the conventional wide- field microscopy and the grating period scanning pump-probe SIM with respect to lateral (Fig. 3.9 e) and axial (Fig. 3.9 g) bar frequencies. In both the lateral and axial cases, excellent agreement with the theoretical expectations is observed in Fig. 3.9(e) and 3.9(g). The field amplitude modulation of lateral case is close to one because of the dense spatial frequency support in the lateral direction, while that of axial case is smaller than one because of relatively coarse spatial frequency support in the axial direction. 58 0 3 E2 10 2 0 10' ~10 2 101 -3-3?- -2 -1 -10 0 1I 2 3 3 1-33 22 -101 0 11 2 33 .2 10 -3 -2 -1 0 1 2 3 -3 - 2 -1 0 1 2 3 (d)- 0 6 ~ , ----~oa, .c. i -,' (e) - .c...... oiert C0.8 PPSN __. 0.8 PPSIM 0.6 0.6 -_-_p ilM( Mr y 0A 0.4 0-2 . O,.2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0 60D 1000 1500 2D00 2500 3000 3500 4000 4500 W*eral postbon (pm) Ipmm ( ) ------10.4 ------0.8 PPS14 10.2~0 -- - 10.2 -8 -6 4 2 0 2 4 6 8 0 200 400 600 800 1000 1200 1400 axial posiBon (m) lpmm Fig. 3.9. Numerical simulation results of USAF 1951 test chart: a whole image data for (a) conventional wide-field microscope, (b) ppSIM with a single grating period, and (c) ppSIM with grating period scanning in (1) xy (2) xz, (3) mn, and (4) ms planes. The image spectra (logarithmic scale) are displayed for amplitude with their axes normalized with the lateral cut-off frequency of the conventional microscope. An isolated three bar pattern simulation results comparing the conventional microscope and ppSIM with grating period scanning in (d, e) lateral and (f, g) axial dimensions: (d, f) cross-sectional profiles in space and (e, g) field amplitude modulations according to the spatial frequency. For the conventional wide-field imaging, the three bar patterns inside the dashed red rectangle in (al) appear blurred and they are barely resolvable where the period of element 1 (the most coarse set) is close to the coherent Abbe diffraction limit kNA (1.177 pm in this study). For the ppSIM imaging, on the other hand, three bars inside the dashed rectangles in (bI, cl) are clearly distinguished with more sharply defined edges. 59 3.4.2. Pump-Probe Structured Illumination Imaging of a non-planar target: a 3D MIT Logo ( (a) 10 -1 m NU U -10 ( 10 Y(UM) 0 U 10 -20-20 -10 0 10 x (um) Fig. 3.10. 3D MIT logo image simulation result: (a) original 3D object and (b) the image from the conventional wide-field microscope and (c) the image from ppSIM with grating period scanning in (1) xy plane and (2) xz plane. xy plane passes through the middle of the letter 'i' (z-0 pm) and xz plane cuts only the legs of each letter (y=0 pm). To further demonstrate the 3D sectioning capability of grating period scanning pump-probe SIM, we image a model of the three letters of the MIT logo where individual letter is located in a different axial position (separated by 8.3 pm). The same simulation condition as in the previous section was used with the volume of 40 pm x 40 pm x 40 pm (in 512 x 512 x 256 image pixels) and total 10 different grating periods (1.11 pm ~11.10 pm). The width of the bar shape in x direction is 1.719 pm and each bar is separated by 1.25 pm in both x and y directions. Fig. 3.10(a) shows the overall configuration of the original sample object in 3D. Comparing Figs. 3.10(b) and (c), it is clear that grating period scanning ppSIM has depth-sectioning 3D imaging capability that the conventional wide-field microscope doesn't have. The xy planes in Fig. 3. 10(bl) and (c1) are located in the middle of the letter 'i' which shows distinct contrast over other letters for our ppSIM in Fig. 3. 10(c 1), whereas all three letters are there with similar contrast for the conventional microscope in Fig. 3.10(b 1). In addition, we can also observe that the edge of the letter 'i' for the ppSIM in Fig. 4(cd) is sharper than that for the conventional microscope in Fig. 3.10(bl) because of the lateral resolution enhancement. The images of bar patterns located in different z locations are totally elongated over the whole axial dimension for the conventional wide-field microscope in Fig. 3.1 0(b2). On the other hand, the corresponding images of each bar patterns are clearly localized to the original positions for our ppSIM in Fig. 3. 10(c2). 60 3.4.3. Pump-Probe Structured Illumination Imaging of Biomolecules in HeLa Cells - - 2 10 2 (o3) 10(2 210 1 108 1 1o 10'k 0 0 0 10- 101 -1 105 I 106 -1 10 0 -21 0-2 105 -2 10, -4-3-2-10 1 2 3 4 .4-3-2-10 1 2 3 4 4 -3-2-10 1 2 3 4 -4-3-2-10 1 2 3 4 Fig. 3.11. The HECI (a-c) and DNA (e-h) in HeLa cells. (a, e) Original 3D data, (b, f) conventional wide-field microscope with 0.68 NA objective, (c, g) grating period scanning ppSIM with 0.68 NA objective, and (d, h) grating scanning ppSIM with 1.0 NA objective in (1) xy plane, (2) xz plane, and (3) ms plane. The image spectra (logarithmic scale) are displayed for amplitude with their axes normalized with the lateral cut-off frequency of the conventional wide-field microscope with 0.68 NA objective. m=0 and n=0 axes lines are added to help analyzing the CTF support change. ppSIM with 1.0 NA shows the axial cut-off frequency of -2.21 (which gives -0.53 pm sectioning capability). As a biological imaging test, I simulated the imaging result of HEC1 protein and DNA (fluorescently labeled with different emission colors in the original experiment) in HeLa cells from the 3D data which is available open to public on the web [25, 26]. Interestingly, the original 3D data itself was acquired on a fluorescence structured illumination microscope (DeltaVision Spectris; Applied Precision) with a 100x 61 1.35 NA objective. The simulation used the original data dimension (346 x 348 x 64 pixels with 0.0629 x 0.0629 x 0.2 im3 voxel size) and total 16 different grating periods (1.11 pIm -3.66 jim for 0.68 NA and 0.76 pm ~1.66 pm for 1.0 NA). Other conditions are the same as the previous sections. The original HEC 1 protein data in Fig. 3.11 (al) and (a2) shows isolated dot-like sharp features with low background, while the original DNA data in Fig. 3.11 (el) and (e2) have relatively bigger island-like features with wide diffuse background. The conventional wide-field microscope loses sharp features of the original data and blurred severely, but several big 'chunks' locations seem to be at least correlated to the actual object position laterally in Fig. 3.11 (bl) and (fl). However, the axial object information is completely distorted in the xz cross-sectional view in Fig. 3.11 (b2) and (f2). For all the HEC1 protein and the DNA, both the low NA pump-probe SIM and the high NA pump-probe SIM successfully reconstruct original sample information in both lateral and axial dimensions in Fig. 3.11(c), 3.11 (d), 3.11(g), and 3.11(h). Comparing the images in Fig. 3.11, we can observe the 3D imaging capability of our grating period scanning pump-probe SIM is consistent regardless of the characteristics of the sample itself and NA of the objective lens. 3.5. Discussion I have shown numerical simulation results that validate the theoretical framework proposed to obtain 3D imaging capability through structured illumination microscopy (SIM) of pump-probe active samples. It is worthwhile now to compare our work with existing pump-probe SLM publications. Previous works about pump-probe SIM are based on intensity measurement for 2D coherent imaging [21-24]. Let us call the reconstruction method in those works as the intensity SIM framework and the one that I am suggesting in this thesis as the field SIM framework. The image intensity measured in a coherent system with a uniform probe beam is given by the nonlinear relation d(r) = Io(r)e(r)] 0 h(r)2 (3.31) 62 where r is the spatial position vector, d (r) denotes the image intensity distribution, o(r) is the pump- probe active chemical species concentration distribution in the object, e (r) is the intensity distribution of excitation pump beam, h(r) is the coherent point spread function (PSF) defined by the system's limiting circular aperture, and the symbol 0 stands for the convolution operator. After Fourier transforming, the image intensity spatial frequency distribution can be written as D(w) = autocorr(O()0 E(w)] H(w)) (3.32) where w is the spatial frequency vector and H is a coherent transfer function (CTF) corresponding to the system PSF. Following the intensity SIM framework in the literature, we first consider the following hypothetically extended transfer function NF HET(1)= H(a-,) (3.33) j=I where w. is the spatial frequency vector and NF is a number of copies of the original transfer function. We write a corresponding "enhanced-resolution" image to this extended transfer function and a uniform illumination of the pump beam as NF, DET () = autocorr O(w)J H() -wj) NF NF = J O(w)H(w-i)* O(w)H(w-wj) (3.34) i=I I MF =G, (w) 1=1 To reconstruct these enhanced-resolution components, the intensity SIM framework considers a structured illumination pattern through the original aperture, given by N e(r)= cos(w, .r +#O) (3.35) j=1 63 Fourier transforming Eq. (3.35) and substituting it into Eq. (3.32), we write the corresponding intensity image as NF Ds1 (w) = autocorr Zo(w -w, ) H(w) ((j=1 NF NF =E Iexp (iDj O(w - w)H (w)* O(c - w )H (w) (3.36) i=1 j=1 MF =exp (iD, ) F (w) The intensity SIM framework is based on the mathematical similarity found between the two Fourier images, one with hypothetically extended CTF, DET (w) and the other with structured illumination, Ds1 (w). The final reconstructed image is the intensity image with an extended CTF, HET (ro which is exactly the same as the one given by the reconstruction method that I suggest here (optical field based). Let us assume that the number of images necessary for the four-point lock-in process to calculate the AC/DC ratio in Fig. 3.1(b) are similar between both the intensity and field SIM frameworks and I will omit them when we compare the total number of images to acquire to reconstruct a single 3D resolved image for both frameworks. The synthetic CTF I suggested to get 3D resolution consists of 17N, - (N, - 1) =1 6 N, +1 unique copies of the original CTF (the DC component is repeated for each grating angle) for Ng number of different grating periods in Fig. 3.7(c) and 3.7(fl). The intensity SIM framework with this CTF (16N +1 unique Fourier components) requires NF =16N +1 pump beams to excite the sample at the same time. The number of images for the intensity SIM framework to reconstruct a single 3D resolved image (MF) is determined by the number of unique Fourier components from cross-correlation ofNF components in Eq. (3.36) (M, =11851 images for Ng =10 and M =31717 images for Ng =16). On the other hand, the field SIM framework that I suggested requires 5 pump beams (17 Fourier copies), 4 measurements for optical phase and amplitude, and N, grating period scanning which results in total 4 x 17N = 68Ng images (680 images for Ng =10 and 1088 images for Ng =16 ). In conclusion, our optical field based pump-probe 64 SIM framework requires 30-50 times less number of pump beams and 17~29 times less number of images to reconstruct a single 3D image with the same synthetic CTF than the intensity SIM framework. This difference comes from the fact that object information is linearly related to the optical field while its relation with the light intensity is nonlinear for the coherent imaging [31, 32]. In addition to the difference in the number of beams and the number of images to be acquired between two SIM frameworks, we also notice that there is a difference in easiness of artifact removal. In Fig. 3.7, shifted copies of the original transfer function overlap one another and may generate an artifact pattern in the final reconstructed image. A periodic line pattern (z direction) is observed in the reconstruction result of Fig. 3.11 (h) and I could readily remove it by averaging or putting a proper weighting on the overlapped Fourier regions. This artifact removal process may be possible for the intensity SIM framework with non- linear deconvolution, but it is not as straight forward as the field SIM framework. As a result, we can more easily provide a higher fidelity reconstructed image with less artifact where the regions within the spectral domain passband are equally weighted while previous work reconstructed an image where some Fourier frequencies are over-weighted. Fully comparing these two approaches would be an interesting topic, but it is beyond the scope of the current paper. There are many prior works on using a point-scanning configuration with a lock-in amplifier to get 3D resolution with pump-probe contrast [1, 15, 28, 34]. The fastest point-scanning pump-probe microscope generates 512 x 512 pixels at a rate of 25 frames/s [1]. For a 512 x 512 x 256 pixel 3D image, the frame rate of the point-scanning system is about 0.1 frames/s. Our system needs 4 (four-points lock-in) x 4 (phase and amplitude for optical field) x 17 (un-mixing) x 16 (different grating periods) = 4352 images for one acquisition. We can easily find a camera with >2000 frames/s for a 512 x 512 pixel field of view in the market (for example, CP80-3-M-540 from Optronis). As a result, the final estimated frame rate of our method is about 0.5 frames/s (about 5 times faster than the point-scanning configuration) assuming that the camera can process all available probe photons without saturation. 65 Now let us consider laser power utilization to be more realistic. If the imaging system is photon shot noise limited, the amount of photons that the system can handle determines the imaging speed with a given sensitivity. Point scanning pump-probe microscopes utilizes only a fraction of available laser power (<50mW for SRS, <2mW for TA) because of sample damage in biological specimen. However, point scanning pump-probe microscopes has inherent 3D imaging capability and a single scanning over the sample volume is enough to acquire the 3D object information [1, 2, 34]. Our grating period scanning ppSIM approach needs to take 17 different 3D data set for one grating period and repeat 10-16 times for different grating periods to reconstruct a single 3D resolved object data, but wide-field configuration makes it possible to fully utilize the available laser power with three times higher lateral resolution. In conclusion, our method has a potential to provide higher imaging speed than the point scanning pump-probe microscope with sufficient laser power. The grating period scanning approach advocated in this paper is important as it allows much better filling of Fourier space than the conventional wide-field microscopy and previous 2D ppSIM approaches. As a result, our approach provides not only 3D imaging capability, but also 2D image with higher fidelity without non-linear image processing. 3.6. Conclusion I have proposed a theoretical framework to implement three-dimensional (3D) wide-field super-resolution pump-probe microscopy utilizing structured illumination microscopy technique. In the wide-field pump- probe imaging configuration, I have added a phase-shifting holography to measure the optical field of the probe beam scattered by the sample and the pump beam. It is important to note that quantitative phase and amplitude imaging of the electric field is required for the reconstruction in these coherent imaging conditions. The structured illumination pump beam serves as a key element that encodes the missing information in both lateral and axial dimensions into the conventional imaging passband. A rigorous 3D Fourier domain framework has been established on how to extract and reconstruct the 3D contents into an 66 image with an enhanced resolution in the context of coherent imaging formation (probe beam) with an intensity dependent structured excitation (pump beam). To check the validity of the proposed method, I have simulated a pump-probe structured illumination microscope with three different sets of samples: a computer-synthesized resolution test target, the MIT logo, and actual biomolecules in HeLa cells. The reconstructed ppSIM image has been examined and compared with the diffraction-limited coherent wide- field image to evaluate the resolution and depth sectioning performances. The results have clearly demonstrated the potential of our method to enable pump-probe microscopy to achieve more than 3 times better lateral resolution and 3D imaging capability over the conventional wide-field pump-probe system. In addition, I found our framework shows consistent 3D imaging capability regardless of the characteristics of the sample itself and NA of the objective lens. Future work might include the experimental implementation of the actual microscope and the investigation to achieve the 3D imaging capability instead of grating period scanning, and the development of simpler way to read the optical field without using phase-shifting holography. The proposed scheme is expected to provide a new way to add super-resolution and depth sectioning capability to wide-field pump-probe optical microscopy and make it a more useful tool for biological and material science. References 1. Saar, B.G., et al., Video-Rate MolecularImaging in Vivo with Stimulated Raman Scattering. Science, 2010. 330(6009): p. 1368-1370. 2. Matthews, T.E., et al., Pump-ProbeImaging DifferentiatesMelanoma from Melanocytic Nevi. Science Translational Medicine, 2011. 3(71). 3. Born, M. and E. Wolf, Principlesof Optics. 1959: Cambridge University Press, Cambridge, UK. 4. Betzig, E., et al., Imaging intracellularfluorescent proteins at nanometer resolution. Science, 2006. 313(5793): p. 1642-1645. 5. Rust, M.J., M. Bates, and X.W. Zhuang, Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nature Methods, 2006. 3(10): p. 793-795. 6. Hess, S.T., T.P.K. Girirajan, and M.D. Mason, Ultra-high resolution imaging byfluorescence photoactivation localizationmicroscopy. Biophysical Journal, 2006. 91(11): p. 4258-4272. 7. Hell, S.W. and M. Kroug, Ground-state-depletionfluorscence microscopy: A concept for breaking the diffraction resolution limit. Applied Physics B, 1995. 60(5): p. 495-497. 8. Hell, S.W. and J. Wichmann, Breaking the Diffraction Resolution Limit by Stimulated-Emission - Stimulated-Emission-DepletionFluorescence Microscopy. Optics Letters, 1994. 19(11): p. 780-782. 67 9. Hofmann, M., et al., Breaking the diffraction barrierin fluorescence microscopy at low light intensities by using reversibly photoswitchableproteins. Proceedings of the National Academy of Sciences of the United States of America, 2005. 102(49): p. 17565-17569. 10. Bailey, B., et al., Enhancement ofAxial Resolution in FluorescenceMicroscopy by Standing-Wave Excitation. Nature, 1993. 366(6450): p. 44-48. 11. Chung, E., et al., Wide-field extended-resolutionfluorescence microscopy with standing surface-plasmon- resonance waves. Optics Letters, 2009. 34(15): p. 2366-2368. 12. Frohn, J.T., H.F. Knapp, and A. Stemmer, True optical resolution beyond the Rayleigh limit achieved by standing wave illumination. Proceedings of the National Academy of Sciences of the United States of America, 2000. 97(13): p. 7232-7236. 13. Gustafsson, M.G.L., et al., Three-dimensionalresolution doubling in wide-fieldfluorescence microscopy by structured illumination. Biophysical Journal, 2008. 94(12): p. 4957-4970. 14. Shao, L., et al., Super-resolution 3D microscopy of live whole cells using structuredillumination. Nature Methods, 2011. 8(12): p. 1044-+. 15. Wang, P., et al., Far-fieldimaging of non-fluorescent species with subdiffraction resolution. Nature Photonics, 2013. 7(6): p. 450-454. 16. Silva, W.R., C.T. Graefe, and R.R. Frontiera, Toward Label-FreeSuper-Resolution Microscopy. Acs Photonics, 2016. 3(1): p. 79-86. 17. Lu, J., et al., Super-Resolution Laser ScanningMicroscopy through SpatiotemporalModulation. Nano Letters, 2009. 9(11): p. 3883-3889. 18. Zheng, J.J., et al., Vibrationalphase imaging in wide-field CARSfor nonresonantbackground suppression. Optics Express, 2015. 23(8): p. 10756-10763. 19. Heinrich, C., S. Bernet, and M. Ritsch-Marte, Wide-field coherent anti-Stokes Raman scattering microscopy. Applied Physics Letters, 2004. 84(5): p. 816-818. 20. Toytman, I., et al., Wide-field coherent anti-Stokes Raman scatteringmicroscopy with non-phase-matching illumination. Optics Letters, 2007. 32(13): p. 1941-1943. 21. Massaro, E.S., et al., Imaging theory ofstructuredpump-probemicroscopy. Optics Express, 2016. 24(18): p. 20868-20880. 22. Park, J.H., et al., A methodfor super-resolvedCARS microscopy with structured illumination in two dimensions. Optics Express, 2014. 22(8): p. 9854-9870. 23. Watanabe, K., et al., Structured line illuminationRaman microscopy. Nature Communications, 2015. 6. 24. Massaro, E.S., A.H. Hill, and E.M. Grumstrup, Super-ResolutionStructured Pump-ProbeMicroscopy. Acs Photonics, 2016. 3(4): p. 501-506. 25. Porter, I.M., et al., Bodi, a novel kinetochoreprotein requiredfor chromosome biorientation. Journal of Cell Biology, 2007. 179(2): p. 187-197. 26. Library, C.I. Z-focal series of HeLa cells at metaphase. 2016 [cited 2016 15 Nov]; Available from: http://www.cellimagelibrary.org/images/1 3378. 27. Commons, W. Resolution test charts. 2016 [cited 2016 15 Nov]; Available from: https://commons.wikimedia.org/wiki/File:USAF-195 1.svg. 28. Fischer, M.C., et al., Invited Review Article: Pump-probe microscopy. Review of Scientific Instruments, 2016. 87(3). 29. So, P.T.C., T. French, and E. Gratton, A Frequency-DomainTime-Resolved Microscope Using a Fast-Scan Ccd Camera. Time-Resolved Laser Spectroscopy in Biochemistry Iv, Proceedings Of, 1994. 2137: p. 83-92. 30. Yamaguchi, I. and T. Zhang, Phase-shiftingdigital holography. Optics Letters, 1997. 22(16): p. 1268-1270. 31. Gu, M., Advanced OpticalImaging Theory. 2000: Springer, Heidelberg. 32. Goodman, J.W., Introduction to Fourier Optics. 2005: Roberts & Company, Greenwood Village, CO. 33. Kou, S.S. and C.J.R. Sheppard, Imaging in digital holographicmicroscopy. Optics Express, 2007. 15(2 1): p. 13640-13648. 34. Freudiger, C.W., et al., Label-Free Biomedical Imaging with High Sensitivity by Stimulated Raman Scattering Microscopy. Science, 2008. 322(5909): p. 1857-186 1. 35. Chowdhury, S., A.H. Dhalla, and J. Izatt, Structuredoblique illumination microscopyfor enhanced resolution imaging ofnon-fluorescent, coherently scatteringsamples. Biomedical Optics Express, 2012. 3(8): p. 1841-1854. 68 36. So, P.T.C., H.S. Kwon, and C.Y. Dong, Resolution enhancement in standing-wave total internal reflection microscopy: a point-spread-functionengineering approach. Journal of the Optical Society of America a- Optics Image Science and Vision, 2001. 18(11): p. 2833-2845. 37. Hamada, K., et al., Raman microscopyfor dynamic molecular imaging of living cells. Journal of Biomedical Optics, 2008. 13(4). 38. Center, D.T.I. MIL-STD-150A. [cited 2016 15 Nov]; Available from: www.dtic.mil/dtic/tr/fulltext/u2/a345623.pdf. 69 Chapter 4 Structured illumination wide-field photothermal digital phase microscopy 4.1. Introduction 4.1.1. Motivation 0.7- (b) 0.6- 0.5- 0.1 0.0- 20 30 40 50 60 70 XU Nanosphere diameter D (nm) Fig. 4.1. (a) Dark field image of anti-CD4 plasmonic nanoparticles (PNPs) (80 nm) in surfactant solution added to whole blood. Red blood cells stack, leaving open regions for the detection of leukocytes. Two CD4 positive white cells are seen. Green PNP's measure 80 nm in diameter. Yellow and red PNP's are larger and are the result of aggregation, which is possibly caused by antigen clumping [1]. (b) ratio of scattering to absorption cross-sections of spherical nanoparticles with various diameters [2]. Ovarian cancer (OC) is the second most common and the most lethal gynecologic malignancy in the U.S. Epithelial OC comprises the majority of malignant ovarian tumors in adult women. Over 70% of women with OC are diagnosed with advanced stage disease when 5-year relative survival is 30%. Five-year survival is 90% when disease is confined to the ovaries but overall survival is poor because only 25% of cases are found in this early stage [3]. 70 CAl 25 is s a mucin-type 0-linked glycoprotein of high molecular mass estimated at 2.5 to 5 M Daltons under natural conditions. It is expressed as a membrane-bound protein at the surface of cells that undergo metaplastic differentiation into a Millerian-type epithelium, or released in soluble form in bodily fluids. CAl 25 is the most extensively studied biomarker for possible use in the early detection of ovarian carcinoma (OC), and it has proved valuable in both detection and disease monitoring [3]. Existing fluorescence based assay suffers low detection threshold which hinders proper diagnosis of ovarian cancer using CA125 detection. Plasmonic nanoparticle (PNP) with dark field microscopy shows a few tens of molecule per cell detection threshold that is good enough to diagnose the ovarian cancer. However, current dark field imaging of plasmonic nanoparticles suffers cell scattering background as seen in Fig 4.1(a) and it is slow for small particle imaging (80 nm particle size is the practical limit with dark field microscopy). Smaller particle shows better stability against aggregation and weaker non-specific binding than larger particles. PNP with photothermal microscopy shows no cell scattering background and has higher imaging speed than dark field microscopy for small particles. Comparing the cross-sections of scattering and absorption, the change is dramatic when the size of the particle becomes smaller than 80 nm in Fig 4.1(b). 4.1.2. Photothermal microscopy Photothermal effect is a phenomenon associated with a production of thermal energy (heat) by the photoexcitation of material under electromagnetic radiation. This thermal energy increases in temperature around the photoexcited material and finally changes the refractive index of neighboring medium. Photothermal microscopy usually utilizes two different wavelength laser beams, a pump (heating) beam of which wavelength matches to the absorption peak of the photothermal active material (for example, plasma frequency of the metal nanoparticles) and a probe beam of which wavelength is chosen for minimum absorption by the photothermal active material. The probe beam is scattered by the refractive index change of the medium induced by the pump beam. This change of probe beam scattering intensity can be directly measured as a photothermal signal [4] or the refractive index change itself can be measured holographically 71 [5]. The temperature profile caused at a distance r of a modulated point source of heat with power P[l+ cos(ot)] in a homogeneous medium is given by [6] AT(r,t) = " [I+ exp(-r/r,,) x cos(ot - r/lrh (4.1) where K is the thermal conductivity of the medium, and r is the characteristic length for heat diffusion at frequency o, given as V2K/oC, (C, is the heat capacity of a unit volume of the medium). This point- source approximation temperature profile is a generally accepted temperature profile in a photothermal community [4, 6-8]. In the next chapter, I will derive an exact analytical solution for the same condition and compare the temperature profile from the point-source approximation and that from the exact solution. This temperature profile causes a refractive index profile An(rt) = AT(rt) x an/aT in the medium. The photothermal scattered probe field E, (t) interferes with a reference probe field Erf (original probe beam input), and the resulting intensity is detected as [4] Idet =IEf + EcI2 = ErjI +2Re[E* E"] +IE 2 . (4.2) The lock-in amplifier filters out the weak (1 in 104 ~ 1 in 106) interference signal 2Re [EfE] modulated at the same frequency as the pump (heating) beam. The contribution of the modulated scattered intensity IE, 12 is neglected for small particles (typically <60 nm) [4] and still negligible for even bigger particles (<600 nm) in the transmission configuration. Based on the above equation, the signal-to-background ratio (SBR) and the signal-to-noise ratio SNR for the shot-noise-limited photothermal detection are given by Ire 2Re[E* E ,2, 2Re.[EefEs, re (4.3) SBR = " ~ ~ 2 Irerf jErefEf I 2Re[E Ec SNR= S ~ e . (4.4) f E 2 72 The photothermal signal S(= 2Re[E,*ES,]) is proportional to the field scattered by an effective volume V where the refractive index is modulated and can be written as an optical power as [4] 1 an I cr S~ - n- 1 eabspp At. (4.5) ,rwo aT C,2 2 A where w is the probe beam focal radius (beam waist for the point-scanning configuration and diffraction limited spot size for wide-field configuration), C, is the heat capacity per unit volume of the photothermal medium, A and probe are the wavelength and power of the probe beam, and At is the integration time of the lock-in amplifier. Since the pump beam power is limited by the sample damage and the material properties are fixed with given sample, the probe beam power determines the photothermal signal level (sensitivity) with given imaging speed or the photothermal imaging speed with given sensitivity. (a) ' s'o'w" (b ) heaing laser M IF 70/30BS as APO prb ae------M*0 WoeMWU:b M1 D ID DI__ ID IF AOpl PD :.FM ELn Fig. 4.2. Typical photothermal microscopy setups for (a) point-scanning configuration [4] and (b) wide-field configuration [5]. The current state of the art photothermal system is a point-scanning system in Fig. 4.2(a) which has much higher sensitivity than the wide-field system in Fig. 4.2(b) with similar speed. It is because the dynamic range of the point detector (photo-diode, 120 dB, 1 in 106 sensitivity) is 3 orders magnitude higher than the area-type detector (camera, 60 dB, 1 in 103 sensitivity). There is a new high well depth camera with 100 times deeper electron well depth (80 dB, 1 in 104 sensitivity) than conventional cameras, but still not comparable to the photodiode in dynamic range. Photothermal imaging speed is limited by photon-shot noise and is determined by the final probe beam power detected by the system. The probe beam power of 73 point-scanning system is limited by the sample damage to about 20 mW which is a fraction of the available laser power (existing wide-field photothermal system takes orders of magnitude less power, a few uW, limited by the saturation of area-type detectors). In this chapter, I will show the development of high-speed wide-field photothermal microscopy that fully utilizes the available laser power by overcoming dynamic range limit of area type detectors. 4.2. Methods and Experimental Setup 4.2.1. Structured illumination photothermal digital phase microscopy (a) Pump beam E ND filter Esc Probe beam aef _Dsc - DMDD Camera (b) - -1-t-t-f- - (ci) n (c2) ,n (63) m m m M I M Fig. 4.3. Schematic diagram of structured illumination photothermal digital phase microscopy about (a) an simplified optical setup, (b) Signal-to-background ratio (SBR) enhancement, and (cl)-(c3) resolution enhancement in Fourier space: (c1) Original sample, (c2) Structured illumination pattern, and (c3) Modulated sample information by the structured illumination excitation. I suggest structured illumination photothermal digital phase microscopy to accomplish the goal I mentioned in the previous section. For photothermal microscopy, original probe field (reference) and photothermal scattered field interfere each other and the resulting intensity is detected. The original probe beam intensity 74 acts as a huge background resulting in the problem that a high dynamic range detector is required. For the structured illumination configuration, pump beam interferes inside the sample and generates grating pattern which is called as a transient grating or virtual grating in contrast to a physical grating in Fig. 4.3(a). While most of the probe beam energy follows the 0th order direction (source of background), photothermal field diffracts multiple orders. Selectively attenuating the 0th order beam by putting an ND filter decreases the background level as Fig. 4.3(b). One may consider removing this Oth order beam completely. However, in this case, the cross term 2 Re[E,E,] in Eq. (4.2) is identically zero and only the small quadratic scattered light term IEJ|2 remains (this small signal can be dominated by instrumental noise). Similar to optical coherence tomography, keeping and controlling the amplitude of the reference field allows us to have heterodyne gain and enable shot-noise level detection limit even in the presence of instrumental noise [9, 10]. Structured illumination provides additional advantage of enhanced resolution. Structured illumination modulates high frequency sample information into low frequency so that it can be allowed to pass through the imaging system in Fig. 4.3(c) [11-15]. 4.2.2. Experimental setup 10x objective (NA 0.4) f=18 150 DMD D - 200 150 Color 100 razor bl e filter D off 50 60x objective (NA 1.2) 125 f=3 300 spatial 200 - - filter 75.6 150 Camera 785 nm SLM . 520 nm Camera Fig. 4.4. Experimental setup for structured illumination photothermal digital phase microscopy. 75 The schematic of structured illumination wide-field photothermal digital phase microscope is shown in Figure 4.4. The wavelengths of laser beams were carefully selected to maximize the photothermal signal. The pump beam wavelength matches to the nanoparticle plasmon resonance wavelength or absorption peak of bio-molecules. The probe wavelength was chosen to minimize the absorption by the nanoparticle or target bio-molecules. The pump beam (heating beam) is provided by a diode laser at 520 nm (LDM-520- 1000-C, Lasertack, max output 1 W) with a diode driver (LDTEC-4500, Lasertack, max modulation 500 Analog / 250 TTL kHz). We modulate the pump beam for lock-in detection of the photothermal signal (small change of probe beam). A 2x beam expander is used in the inverse direction to adjust the diameter of the pump beam. A spatial light modulator (SLM) (PLUTO, HOLOEYE, reflective liquid crystal on silicon type) is used as a phase grating for the pump beam transient grating pattern inside the sample. The diffracted pump beam from the SLM passes through a spatial filter to allow only the 1st orders and block the 0th order for 2D structured illumination pattern (it will be open for 3D pattern) and unnecessary weak higher orders. A sharp edged razor blade is positioned in the image plane after the spatial filter to block the pump beam over the part of the field of view so that a fraction of the camera pixels record only the probe laser intensity fluctuation without the photothermal signal. The single-mode fiber coupled laser diode (FPL785S-250, Thorlabs, max output 250 mW) generates probe beam at 785nm with a laser diode controller (LDC21OC, Thorlabs) and a temperature controller (TED200C, Thorlabs). The pump and probe beams are combined with a dichroic mirror (DiG 1-R532-25x36, Semrock) and focused at the back aperture of the microscope objective (UPlanAPO lOx NA 0.4, Olympus). The probe beam diffracts into multiple orders by the transient grating inside the sample. The scattered probe beam is collected by a second objective lens (UPlanAPO 60x NA 1.2 water, Olympus). A long-pass filter (LP03-532RE-25, Semrock) blocks the pump beam after passing through the sample. After the long- pass filter, 0th order probe beam is selectively attenuated by a custom dot (d=1 mm) neutral density filter (OD=1.4, Reynard Corporation) which is located 1 inch in front of the Fourier plane (DMD plane). The detection pathway is a typical common-path holography or quantitative phase microscopy configuration 76 and a DMD (DLP6500FYE, Texas Instruments) is used as a configurable spatial filter. Finally, the probe beam interferogram is captured by an extreme high electron well depth CMOS camera (Q-2A750/CXP, Adimec Advanced Image Systems) with a frame grabber (Cyton-CXP4, BitFlow). Another auxiliary CMOS camera (Flea3 FL3-U3-13S2M, FLIR Integrated Imaging Solutions, formerly Point Grey Research) is used with a flip mirror for alignment (overwrapping pump and probe beams) and pump beam interference fringe contrast calibration because DMD diffracts the pump beam into a different angle from the probe beam and it is not captured with the main camera. 4.2.3. Structured illumination of pump beam In the previous section, the diffraction of the pump beam by SLM for structured illumination is briefly introduced, but more careful consideration is needed for the design of an experimental setup and the image reconstruction process about the pump beam transient grating pattern. For uniform resolution enhancement in both horizontal and vertical directions, both horizontal and vertical binary grating patterns are projected to the SLM sequentially with matching DMD patterns and each grating pattern is spatially shifted three times for structured illumination reconstruction that will be explained in section 4.3.2. The issue is that a reflective SLM is used with a horizontally incident pump beam in Fig 4.5(a). While a grating vector, a surface normal vector, and an incident pump beam wave vector are in the same plane for the vertical pattern on the SLM in Fig 4.5(bl), they are not for the horizontal pattern on the SLM in Fig 4.5(b2). As a result, the incidence angle for the reflection grating equation in Fig 4.5(c) is the same as the pump beam incidence angle for the vertical pattern on the SLM and an effective grating period on the sample must be calculated from the reflection grating equation. On the other hand, the incident angle for the reflection grating equation in Fig 4.5(c) is 0 for the horizontal pattern on the SLM (cutting the 3D volume with a plane including the grating vector and the surface normal vector, the pump beam looks like coming in perpendicular to the grating surface). An effective grating period on the sample is the same as the SLM pattern period (when considering the demagnification factor by following optics). While an effective grating period for a vertical 77 SLM pattern of 12 pixel period (8 [pm per pixel) for the experiment is calculated 67.8 pm, that for a horizontal SLM pattern of 9 pixel period is just the same as the SLM pattern of 9 x 8 Pm =72 pm (intentionally similar effective grating periods were chosen for both patterns). One more thing to be careful is that the photothermal signal is proportional to the pump beam intensity so that the structured illumination pattern (transient grating) period is the half of the calculated effective field grating period. (a) (bi) (c) - 200 razor bl e I. 50 spatial % - filter (b2) 150 m=. M=0 SLM 520 nm a [sin(6,) + sin(9.)] = 10 Fig. 4.5. Structured illumination pump beam by SLM as a phase grating. (a) Schematic diagram of an optical setup. (bI) Vertical pattern on the SLM. (b2) Horizontal pattern on the SLM. (c) Grating equation for a reflective grating: A is the wavelength of the light, a is the grating period, 0, is the incidence angle, 9,,, is the diffraction angle, and m is the diffraction order from the grating [16]. In the implementation of the structured illumination of pump beam, the parameters are always different from the theoretical values and they actually slightly varies for each measurement for different samples or for even different area-of-interests (AOIs) within the same sample. The computer simulation result about the effect of such deviations of parameters to point spread functions is shown in Fig. 4.6. While the errors of structured illumination direction and period are critical to the image quality (direction error 5 0.5* and period error 5 1% are needed respectively for reasonable point-spread function images for whole field of view of 40 gm x 40 pm area, simulation result not shown), errors of structured illumination pattern shift and contrast do not seriously degrade the point spread function (PSF). To get accurate values of the 78 structured illumination direction and period for image reconstruction, they are included as optimization parameters in the second step aberration correction process that will be explained in section 4.3.3. No error Direction (10 deg) Shift (+- 5% of 2pi rad) 016 0.14 0.14 0.12 0.12 01 01 E E 0 0.06 Z. 006 0,06 -10w 0.04 004 -15w 0 02 002 -20 -20 -15 -10 -5 0 5 10 15 -20 -15 -10 -5 0 5 10 1 -ZU -15 -1 -n U n IV 10 Parind (+1 flo Parinr I-i m rnntr-act frnOAI 0 16 0 10 15 0.14 0 14 10 0.12 0.12 0.1 E 0.04 0.06 ZL 0.00 0.00 -10 004 -10 0.04 -15 0.02 0.02 20 -15 -10 -5 0 5 10 15 -20 -15 -10 -5 0 5 10 15 -20 -15 -10 -5 0 5 10 15 Fig. 4.6. Effect of structured illumination parameter errors to point spread functions 4.2.4. Sample preparation Samples of gold colloids with diameters of 60 nm (82150-60, suspension in Milli-Q Water, Ted Pella, Inc.) were prepared by dilution in ultra-pure water at the ratios of 1 : 40. Approximately 10 pL of the suspension were deposited on the cover glass surface (# 1.5 thickness, 24 x 50 mm, VWR) and dried on the hot plate at 90 *C for 10 minutes. To minimize aberration from the glass thickness in my transmission configuration, a thin wide cover glass (0.17 mm thickness) was used instead of a conventional glass slide (1 mm thickness). As a thermal medium, 2% agarose solution was prepared with ultra-pure water. Agarose powder (low melting point agarose, 15517-022, Gibco BRL) and water mixture with a stir bar was put into a glass bottle and the bottle was heated in the water bath above 85 *C to dissolve it into solution. The temperature was kept at the same high temperature so that the agar solution was in the fluidic state during the sample 79 preparation. 10 ptL of the fluidic agar gel was pipetted on the gold nanoparticle deposited cover glass surface and immediately sandwiched by another cover glass (# 1.5 thickness, 18 x 18 mm, VWR) before the agarose solution became a solid gel. Two-part epoxy adhesive (31345, 30 minute handling time, Devcon) was used to seal the gap between the two cover glasses to prevent the water content of the agar gel from being dried out into the air. Samples of gold colloids with diameters of 150 nm (742058, stabilized suspension in citrate buffer, Sigma-Aldrich) were prepared by dilution in ultra-pure water at the ratios of 1 : 40. Approximately 10 pL of the suspension were deposited on the cover glass surface and dried on the hot plate at 90 'C for 10 minutes. I chose the 150 nm gold nanoparticle for an easier target than 60 nm particle in the early stage of the system development because photothermal signal shows a cubic dependence on the particle diameter. However, we found that local heating caused the melting of the agarose gel and showed severe Brownian motion during data acquisition. As epoxies are thermosetting materials, the final cured epoxy material does not melt or reflow when heated (unlike thermoplastic materials), but undergoes a slight softening (phase change) above 350 'F (177 *C) [17]. Because of this high heat resistance, UV-curing optical adhesive epoxy resin (NOA88, Norland Products, Inc.) was chosen as a thermal medium for the 150 nm gold. One drop of the UV-curing adhesive was put on the gold nanoparticle sample and sandwiched with another cover glass. Finally, the sample was exposed to the UV light with a high-output UV spot curing system (ELC-410, Thorlabs Inc., NJ) for 10 minutes to harden the epoxy. 80 4.3. Image acquisition and processing flow DMD pattern Holographic reconstruction (Left, right, down, up) -H (mixed 3 E-fields) Structured illumination of Aberration correction pump beam (0, 120, 240 deg) Raw interferogram SIM reconstruction (4x3 = 12 images) (separating 3 E-fields) 1/f noise correction CTF equalization Lock-in (AC/DC ratio) F- Final image Fig. 4.7. Image acquisition and processing flow for structured illumination photothermal digital phase microscopy. (a2) (al) Pump beam Sample I I Camera DM D Sky ,ky ky (bi) (b2) (b3) tkY (b4) t ky (b5) Sampl Sinfo Pump beam intensity kx kx Obj NA F(S} F{R) Fig. 4.8. (al) Schematic diagram of an optical setup for collection of light. (a2) Patterns on DMD. Fourier space diagram for (bl) original sample information, (b2) pump beam intensity (transient grating), (b3) sample information modulated by the pump beam intensity, (b4) DMD plane, and (b5) acquired image filed by the camera. 81 The structured illumination photothermal digital phase microscopy involves multiple steps of image acquisition and intensive image processing. Therefore, I will briefly overview the whole image acquisition and processing flow shown in Fig. 4.7 and explain the details in the following sections. For each DMD pattern, three images with pump beam grating pattern shift are needed because there are three image components overwrapped from the structured illumination in Fig. 4.8. Photothermal interaction of the probe beam is coherent scattering process that is directly related to the electromagnetic field of the probe beam, not the intensity. Therefore, DMD works as a configurable spatial filter to create a reference field for the interferometric measurement of the electromagnetic field (E-field) of the probe beam in Fig. 4.8(al). Since the transfer function is asymmetric with a single DMD pattern, I repeat the previous process four times with DMD patterns at four different orientations to get a spatially symmetric transfer function. The temporal modulation frequency is not fast enough to circumvent the 1/f noise and additional 1/f noise correction process is implemented before the lock-in amplification of the photothermal signal. Noise rejected interferogram is reconstructed to recover the E-field with standard holographic reconstruction technique by Fourier filtering. The structured illumination photothermal microscopy system is sensitive to the aberration due to its synthetic image reconstruction process - adding multiple raw images to generate a single final image. When multiple raw images are not spatially synchronized each other, the final image becomes totally burred out. Aberration corrected E-field contains three components from the structured illumination of the pump beam and they have different spatial frequency information. These three components are separated and put back into the proper position in the Fourier space during the structured illumination microscopy (SIM) reconstruction process. I get SIM reconstructed E-fields from four different DMD patterns and synthesize a final E-field for a spatially symmetric transfer function. However, some spatial frequency regions are added multiple times from each DMD pattern and these regions must be properly normalized to get a reasonable imaging result. 82 4.3.1. 1/f noise correction and lock-in amplification 50-60 Hz noise, acoustic and other interferences filIter rad io, mobile 1/f-noise white noise f frequency f2 Fig. 4.9. Qualitative noise spectrum of a typical photothermal experiment. The pump (heating) beam modulation frequency should be chosen in a region with small background, avoiding any discrete peaks coming from technical sources. In the example, f2 will yield better results than fl for the same filter bandwidth, since it is located in a clean white noise region above the 1/f noise at low frequencies. [18] The photothermal probe beam scattering does not generate new color and therefore we need to impose a modulation on the pump beam so that the photothermal signal appears as a modulation in the probe beam. This probe beam modulation (transferred from the pump beam) can be detected with a lock-in amplifier with high sensitivity. The lock-in amplifier acts as a narrow band-pass filter around the modulation frequency to reject the noise. Fig. 4.9 shows the qualitative spectrum of different noise sources. While photon-shot noise has a flat spectrum for all practical frequencies and contributes to the "white noise", 1/f noise has literally 1/f frequency dependence ("flicker noise" or "pink noise"). For the point-scanning system, the modulation frequency is chosen in an uncongested band of the radio-frequency spectrum of the probe beam (typically >1 MHz, away from the low-frequency 1/f noise). However, the acquisition speed at a single point for area-type detectors like CCD or CMOS cameras (a few tens of thousands frames per second 83 at best) is orders of magnitude slower than the point-detectors and wide-field photothermal imaging system using an area-type detector suffers from strong 1/f noise. (a) (b) Camera i noise (frame set 1) 680 660- 640 -- C 120-- 0 600 Probe beam only -T (No pump beam) 580- 560- 540- 0 20 40 60 80 100 120 140 160 180 200 no of tames Fig. 4.10. Probe beam intensity fluctuation compensation (a) schematic diagram of camera pixels which observes probe beam only region of the sample to track laser 1/f noise fluctuation (b) 1/f noise fluctuation of the probe beam laser intensity over time. To suppress this strong 1/f noise for the wide-field photothermal system with an area-type detector, I recorded the probe beam laser fluctuation utilizing a fraction of the camera pixels which only observes the probe beam not affected by the pump beam in Fig 4.10(a). The probe laser fluctuation intensities are averaged over the area to selectively obtain laser noise or "common mode noise" by suppressing the other noises like photon-shot noise and read noise of each pixel. Finally, I got a time series of the probe laser fluctuations in Fig 4.10(b) and normalized the intensity values of the photothermal interferogram with this time series to suppress the probe laser fluctuation. 84 (a) (b) DC = VO+V1+V2+V3 4 _(VO-V2 +(V1-V3 ) AC = 2 VO V1 V2 V3 VO V1 V2 V3 M = Time DC Fig. 4.11. (a) Camera samples and integrates the signal synchronously to the pump beam modulation at four points per sinusoidal waveform. (b) Four-step modulation signal reconstruction. The lock-in amplification in the wide-field mode is implemented in two steps. First, every other four frames of images are averaged into four images (for example of total 1000 images taken, VO=(Il1+15+...+1997)/250, V1=(12+I6+...+I998)/250, V2=(13+I7+...+I999)/250, V3= V2=(I4+I8+...+11000)/250). Averaging multiple images into a single image acts as an effective bandpass filter because other frequency components than the modulation frequency are literally averaged out. Second, the original probe beam intensity (DC), photothermal modulation intensity (AC), and AC/DC ratios are calculated from four images (photothermal signal directly proportional to the concentration of the photothermal active material is AC/DC ratio). To calculate the probe beam modulation (AC), I applied the four-point reconstruction method which is commonly used for frequency domain time-resolved microscopies [19] and phase-shifting interferometry [20]. The photothermal intensity to be recorded by a CMOS camera is expressed as V(t) = DC + AC cos(cot + #) (4.6) where DC represents the original probe beam without frequency modulation and AC is the amplitude of the photothermal modulation with frequency co and initial phase # . Eq. (4.6) shows that there are three unknowns in the recorded intensity time series: DC, AC, and # . With three unknowns, at least three measurements with different values of wt are needed to recover unknowns. While the three-step algorithm gives an exact result, it is sensitive to errors in the value of t and measurement noise. Four-step algorithms 85 provide superior performance over the three-step algorithm to reduce the noise sensitivity of the measurement [20]. In the four-step algorithm, the intensity measurements for each step are VO =DC + ACcos(#) V1=DC + AC cos(# +-) 2 (4.7) V2=DC+ACcos(#+lf) V3=DC+ACcos(#+3) 2 Solving for the DC and AC values gives DC = VO+V1+V2+V3 4 (4.8) A___ [(VO - V2)2 +(V1-V3)2] 2 4.3.2. Holographic reconstruction, SIM reconstruction, and CTF equalization ky A,ky Aky (a) (b) ky{||2 (C) (d) F{1R 1 } kx kx kx kx F(SR} F{S*R} F{SR*} F{S} F{R} Obj NA F{ IS|2 (e) ky ky ky ky ky _kx kx kx kx kx YObj NA Fig. 4.12. Image reconstruction in Fourier space. (a) Image field and (b) intensity captured by the camera. (c) Holographic reconstruction for light field information: spatial frequency sample information is masked by the numerical aperture size and the pump beam intensity grating period (cut-off in the middle between the transient grating and zero spatial frequencies). (d) Sample information (light field) after structured illumination reconstruction. (e) Synthesizing a symmetric transfer function by adding structured illumination reconstructed image field with four different DMD patterns to get a reasonably isotropic PSF. 86 The acquired interference pattern from the DMD pattern in Fig. 4.12(a) can be described as, 2 2 2 IR + S =|R +|S + R*S + RS*. (4.9) where R and S corresponds to the spatially uniform reference E-field and the photothermal signal E-field respectively. The Fourier transform of this interference intensity in Fig. 4.12(b) shows clear separation of a real image (F {R*S} ) from a virtual image ( F{RS*} ) and autocorrelation of the reference field (F R1I ), but some overwrap with an autocorrelation of the original spectrum of the sample (F {1S12 ). However, amplitude of the photothermal signal (ISl) is several orders of magnitude lower than that of the reference field (IRI) and overwrap with an autocorrelation of the original spectrum of the sample (F {IS121 ) can be safely neglected. The Fourier transformed interferogram is subsequently cropped by a mask of which size is determined by the numerical aperture of the imaging system and the pump beam interference fringe period. The masked information (recovered E-field) contains three different components from the structured illumination pump beam as S(m,n) = 2E 2 L(m,n)+O m+2sin6 -exp(-i#)+m - 2sinOnjexp(iA#)jh(m,n). 2 A 2 (4.10) where m and n are spatial frequency coordinates corresponding to x and y directions respectively, X is the Fourier transform of X, S is the E-field by holographic reconstruction, 0 is the sample information, EO is the amplitude of the pump beam, 0 is the incidence angle of the pump beam, A is the pump beam wavelength, A# is the phase of the pump beam interference pattern (transient grating), and h is the coherent transfer function of the probe beam collection system. This equation is a two beam interference version of the ppSIM framework in the previous chapter where detailed derivation is described. With three measurements with different values of A0, three components can be linearly separated. Since each of these component is spatially modulated by the structured illumination pump beam (translated in the Fourier space 87 by the amount of pump beam interference spatial frequency), they are translated back to the original position in the Fourier space as Fig. 4.12(d). The translation in the Fourier space is implemented in the spatial space by multiplying by the complex phase gradient exp i 4 (which represents the frequency-space 2sin9 2sin9 shift by ) because the shift vectors do not in general fall on integral pixels of the discrete 2 2 frequency space [13]. The pump beam interference period and direction accuracy is critical to the final imaging quality, which will be discussed in the next section more thoroughly. Single DMD pattern structured illumination imaging extends lateral resolution only in one direction and generates an asymmetric transfer function in Fig. 4.12(d), so the procedure is repeated with three other DMD patterns for more reasonable imaging result (uniform resolution lateral resolution enhancement and symmetric transfer function) in Fig. 4.12(e). However, synthesizing a final image with multiple DMD patterns shows a side effect of exaggerated lower spatial frequency information and a pattern artifact because of the frequency support overwrapping among DMD patterns. This frequency support overwrapping can be pre-calculated from the experiment design and the synthetic image from multiple DMD patterns is equalized in the Fourier domain. 4.3.3. Aberration correction PupilM - (xy) Idjeal -. ," ft.. , " W(x, y) = W* (x,y) W, I Aberrated 's Fig. 4.13. Spherical (sp) and aberrated (ab) wavefronts [2 1] 88 For an ideal (diffraction-limited) imaging system, a point-source object yields at the exit pupil a perfect spherical wave, converging toward the ideal geometrical image point. A system with aberrations (found in most practical imaging systems) has a wavefront phase surface that deviates from the ideal spherical wave and their effect reduces image quality. Figure 4.13 shows the exit pupil (XP) with an ideal spherical (sp) wavefront and aberrated (ab) wavefront in profile. The wavefront error is described by W (x, y), an optical path difference (OPD) function that represents the difference between the spherical and aberrated wavefront surfaces in the pupil plane. Wavefront OPD is commonly described by Zernike polynomials in Table 4.1 like the following because they represent an orthonormal basis for a circular area with weights (or coefficients) [22]: W(r, 0) aZ4 (r,0) (4.11) where (r,9) is the polar coordinate, Z, is the Zernike polynomial with a single index i, and a, is the constant weight (spatially invariant aberration) or weight functions (spatially variant aberration). Then the aberration pupil function is defined as [21] P(u,v;x,y)= circ V +' jexpr-ikW uv;L, j (4.12) where the circle function describes the diffraction-limited exit pupil, w, is the exit pupil radius, (u, v) is the spatial coordinates of the image plane and (x,y) is the spatial coordinates of the pupil plane, k is the probe beam wave vector, and W (x, y) is the wavefront OPD. The light field distribution at the pupil plane (i.e., the back focal plane of the objective lens) is directly related to the Fourier transform of the light field at the object plane by [21] H(u,v~mn) = P(u,v;-Az,m,-Azxpn) (4.13) 89 where H is the transfer function of the optical system, (u, v) is the spatial coordinates of the image plane and (m, n) is the corresponding spatial frequency coordinates to (u,v) , A is the probe beam wavelength, and z, is the distance from the exit pupil to the axial image point. In conclusion, aberrations can be mathematically expressed as an additional phase term described by Zernike polynomials in the Fourier plane compared to the aberration-free imaging system. More detailed explanation and theory about aberration can be found in the literature [21-24]. Table. 4.1. The first 17 Zernike polynomials [22] Noll Name index (i) Z" (r,0) 1 1 piston 2 2rcos0 x tilt 3 2r sin0 y tilt 4 ,/i(2r2 -1) defocus 5 iI6r2 sin (20) y primary astigmatism 6 --[6r2 cos(20) x primary astigmatism 7 J8(3r3 -2r)sin0 y primary coma 8 ,8(3r3 - 2r)cos 0 x primary coma 9 -fr3 sin (30) y trefoil 10 -8r3 cos(30) x trefoil 11 5(6r4 - 6r2 +1) primary spherical 12 ,[1-0(4r4 -3r 2 )cos(20) x secondary astigmatism 13 fI 0 (4r -3r 2 )sin(20) y secondary astigmatism 14 /i7r4 cos(40) x tetrafoil 15 .i 6r4 sin (40) y tetrafoil 16 1-(IOr' -12r 3 +3r)cos0 x secondary coma 17 12(1Or' -12r 3 +3r)sin 0 y secondary coma 90 The aberration correction for structured illumination photothermal microscopy is achieved by iteratively finding Zernike polynomial weightings in the pupil plane (Fourier plane) that minimize aberration. More specifically, two main steps are implemented for this purpose. The first step is minimizing peak position difference and maximizing average peak signal for different DMD patterns in uniform illumination mode in Fig. 14(a). Summing structured illumination image with three pump beam grating shifts cancels out spatially modulated high spatial frequency terms and provides uniform illumination image in Eq. (4.10). In the first step, the first 13 Zernike polynomials in Table 4.1 are iteratively optimized and theoretical structured illumination pattern parameters (period, direction, and contrast). After the first aberration correction, while peak positions in different DMD pattern matches (data not shown) and the peak signal strength increases for the uniform illumination mode, the structured illumination image is still blurry in Fig 4.14(c2). The second step is minimizing peak position difference between uniform and structured illumination modes and maximizing the peak signal in the structured illumination mode in Fig 4.14(b). In the second step, structured illumination pattern parameters (periods and directions in both horizontal and vertical directions with perfect contrast assumed) and the first 17 Zernike polynomials in Table 4.1 are iteratively optimized together starting from the first step corrected image field. After the second step correction, both uniform and structured illumination modes show sharper contrast PSFs in Fig. 4.14(c3). It is also noted that the peak amplitude of the uniform illumination mode after the second step correction is about 40% higher than the first step corrected value. The first step correction process provides a roughly correct answer as a good enough starting point for the second step correction iteration so that it prevents the iteration algorithm from being trapped into a wrong local optimum values. When the second step correction process was directly applied to the raw image field without the first step correction, totally messed up results were often observed. For the reference, the field of view of the setup was small (about 40 prm x 40 pm) and I assumed spatially invariant aberrations. 91 (a) Before aberration correction (b) x 1-3 Left N 2.5 e Uniform 10 10 Structured 20 20 S2 Minim ize 30 30 40 40 1.5 10 20 30 40 10 20 30 40 fnn "n 1 axir ize 10 10 0.5 20 20 30 30 0 1000 2000 40 40 10 20 30 40 10 20 30 40 x [nm] 3 x10 2.5 Uniform (Ci) +v'Structured (c2) (c3) 2 21.5 0.5 i ,#. 0 5~1 0 1000 2000 0 1000 2000 0 1000 2000 x [nm] x [nm] x [nm] Fig. 4.14. Aberration correction for structured illumination digital phase microscopy: (a) Step 1 correction method based on uniform illumination mode. (b) Step 2 correction method based on both uniform and structured illumination modes. (cI) Raw image before correction. (c2) Step 1 correction result. (c3) Step 2 correction result. Target sample is 150 nm gold nanoparticles in UV-curing adhesive (epoxy resin). 92 3 x 10- 10~3 (a) 2.5 (b) 25 5 5 2 2 10 10 15 1.5 15 1.5 EE 20 20 25 25 30 0.5 0.5 35 3W 0 0 10 20 30 20 30 x [AM] x [pm] (C) 2 -3 e+Before 1.5 .5* +After 0.5 0 0 1000 2000 x [nm] Fig. 4.15. Aberration corrected imaging with structured illumination photothermal digital phase microscopy: (a) before correction, (b) after correction, and (c) comparison of PSF profiles between before and after aberration correction. For iterative optimization of parameters for aberration correction and structured illumination, generalized pattern search (GPS) algorithm is used following the literature [24] because it is difficult to find a gradient of the cost function with respect to Zernike polynomial weightings and GPS algorithm does not require a gradient. This GPS algorithm directly calculates the cost (objective) function in the search mesh centered on the current position (mesh dimension is equal to the number of optimization parameters) in Fig 4.16(a) [25, 26]. If there is a point which gives smaller cost function than the current position, it moves to that position and increases the size of the search mesh (or keeps the same size) in Fig 4.16(b), (c), (e), and (g). If no such position, it decreases the size of search mesh and tries to compare cost function in Fig 4.16(d), (f), and (h). 93 (a) I (1W, 0.5 h/05 0.5 0 - 00 _ -05 -0I 1 -0.50s 0. 1 - -0.50 . (e) 0.5 o.s05 0.5 0/ 0 -- s: -oo as -AS 0 ~-11 -0.5 0 0.5by 1 1-5 0 0 sp d Fig. 4.16. Ster by step procedure of pattern search (PS) algorithm [25, 26]. I implemented the GPS algorithm using MATLAB (MathWorks) which provides a pattern search solver and related built-in functions in 'Global Optimization Toolbox'. More specifically, 'optimoptions' function was used to create an optimization options object with 'MaxIterations', 'MeshTolerance', 'InitialMeshSize', 'MaxMeshSize', 'MeshExpansionFactor', 'UseCompletePoll', and 'UseParallel' options. The optimization process ends if the number of iterations reaches to 'MaxIterations' value or the size of search mesh is less than or equal to 'MeshTolerance' value. For aberration correction, the optimization parameters of Zernike polynomial weightings introduce an additional phase (e10) in the pupil plane (Fourier plane) and 'MaxMeshSize' of 3 (rad) is used to prevent phase-wrapping issue. 'MeshExpansionFactor' controls the mesh size by multiplication after each successful poll, which is related to the speed of the optimization and the stability (the value was optimized by trial and error). 'UseCompletePoll' determines if the evaluation of the cost function is stopped when smaller cost function value is found or the evaluation is performed for P all points in the search mesh. 'UseParallel' helps to speed up the iteration by enabling parallel computing over multiple cores of CPU. 'patternsearch' function was used to run the pattern search solver with an objective function (cost function) handle, initial values of optimization parameters, no linear inequality constraints, no linear equality constraints, no bounds, no nonlinear inequalities, and the option object created by the 'optimoptions' function. For further information about other options about pattern search or 94 other optimization methods available in 'Global Optimization Toolbox', you can refer the MATLAB help file or online help website. 4.4. Results and discussion 4.4.1. Signal-to-background ratio (SBR) enhancement (a) Pump beam sN D filter Es Probe beam EC DMD Camera (b) x 10-3 Mc x 10-3 5 8 5 8 10 10 6 6 15 15 a 20 .120 25 25 30 2 30 2 35 35 0 0 10 20 30 10 20 30 x[pm] x [/m] Fig. 4.17. (a) Schematic diagram of an optical setup for SBR enhancement. (b) Before SBR enhancement. (c) After SBR enhancement. 60 nm gold nanoparticles in 2% agarose gel were used for a photothermal sample. For the structured illumination photothermal microscopy, the probe beam is diffracted into multiple orders by pump beam transient grating in Fig. 4.17(a). Selectively attenuating the 0th order beam by a factor of a 2 (intensity) changes the detected intensity as Idet =IaEref + = IE,, +alEf1 + 2a Re[E,*E]+|EI (4.14) 95 where E,f is the original probe beam input field and E,, is the photothermal scattered probe beam field. With the selective attenuation, the signal-to-background ratio (SBR) and the signal-to-noise ratio SNR for the shot-noise-limited photothermal detection are given by I 2aRe[E*EfEsc 2Re[E* Ec SBR = -5-= =- I (4.15) Ibg a E12 a E 2 I2ReE' Ec 2 Re[ Ef E SNR= c 2 [ef. =Re (4.16) Nbg a IEref r JEfIr Comparing with Eq. (4.3) and (4.4) about the SBR and SNR without the attenuation, SBR is enhanced by the factor of 1/a and SNR is the same. It means that selectively attenuating the 0th order beam enables full utilization of available laser power keeping the same SNR without saturation of camera (higher speed with given sensitivity or higher sensitivity with given speed). In the experiment, I had poor signal-to-background ratio in the left image without attenuating the 0th order probe beam in Fig. 4.17(b). After attenuating the 0th order beam and increasing the probe beam input inversely proportional to the attenuation (increased dynamic range of the system), background intensity and photothermal signal decreased differently according Eq. (4.14). With 10 times intensity attenuation, I got about 3 times SBR enhancement in Fig. 4.17(c), which exactly matches to the theory. 4.4.2. Resolution enhancement (Point spread function measurement) A comparison of uniform illumination and structured illumination photothermal images is shown in Fig. 4.18. The electric field amplitude profiles in both horizontal and vertical directions show that the structured illumination photothermal PSF has an about 40% improvement over uniform illumination in terms of the FWHM in Fig. 4.18(c), (d), and (e). This observation is less than the theoretical prediction of 60% improvement with given NA of the collection objective and pump beam transient grating period. While the theoretical resolution assumes an ideal point source of which size is infinitesimally small, the size of the 96 gold nanoparticle is 150 nm (diameter). Considering the theoretical FWHM of about 350 nm and the actual size of the gold nanoparticle, the experimental resolution improvement makes sense. The resolution enhancement is not impressive because a low NA (0.6) air objective was used to generate the pump beam transient grating for ease of experiment (long working distance and no clean-up of sticky immersion oil). 3 (a) x10- )104 5 14 5 5 12 10 4 10 10 15 15 3 8 20 20 6 25 2 25 30 30 4 1 2 35 35 10 20 30 10 20 30 xflim] x[Pm] (C) ROI1 (dl) -4- ROI Z2 a ROI 3 Unif"" 0.8 0.8 -tucurs 0.8 FWHM FWHM FWHM 0.6 439nm 0.6 441m 0.6 418nm 349nm 307nm 312nm 0.4 0.4 0.4 0.2 0.2 0.2 0D 0 0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000 x [nmj x[nm] x Inm (c2) 1 (d2) (e2)' ROIl1 -- Unfr ftcue 8 ROI 2 ROI 3 0.8 0. 0.8 FWHM 6 0.6 0. FWHM R 0.6 FWHM 418nm 479nm 419nm 0. 4 0.4 307nm 368nm a0.4 365nm 0.2 0.. 2 0.2 :4 0 0 500 1000 1500 2000 0 500 1000 1500 2000 y Inmj y inm] y [nmi Fig. 4.18. Extended-resolution imaging with structured illumination photothermal digital phase microscopy in horizontal and vertical directions: (a) uniform illumination image, (b) structured illumination image, and (c)-(d) comparison of PSF profiles of two different imaging modes at a selected region of interest (ROI) in horizontal (suffix 1) and vertical (suffix 2) directions. 97 4.5. Conclusion I have proposed structured illumination wide-field photothermal digital phase microscopy as a better tool to diagnose ovarian cancer using CA125 detection. In the wide-field photothermal holographic imaging (digital phase microscopy) configuration, I have added structured illumination to solve the dynamic range issue of the area type detector and get increased resolution. It is important to note that quantitative phase and amplitude imaging of the electric field is required for the reconstruction of structured illumination photothermal imaging (coherent scattering process). An experimental setup has been designed and built to demonstrate the feasibility of structured illumination wide-field photothermal digital phase microscopy. Intensive image processing algorithms also have been developed to detect the small signal with noisy environment, reconstruct the E-field from the interferogram, correct the aberration, and reconstruct the modulated and mixed E-fields from the structured illumination. To check the validity of the proposed method, I have tested a 60 nm gold nanoparticles in 2% agarose gel sample and a 150 nm gold nanoparticle in epoxy resin sample: shot-noise limited 3 times SBR enhancement and 40% lateral resolution were acquired. The results have clearly demonstrated the potential of my method to enable high speed wide-field photothermal microscopy with improved the lateral resolution. As a next step, the current low NA objective will be replaced into a high NA objective to achieve more than 3 times resolution enhancement. The sensitivity limit of the system also will be tested with smaller nanoparticles. The 3D depth sectioning capability with incidence angle scanning (or transient grating period scanning) proposed in the previous chapter can be demonstrated for wide-field photothermal microscopy. The system may be integrated into a ovarian cancer diagnosis method as a high throughput imaging tool and be applied to melanosome observation for melanoma research without metal nanoparticles (melamine pigment itself is a highly light absorbing molecule) [27]. Photothermal microscopy belongs to pump-probe microscopy and the framework I developed can be directly applied to other wide-field pump-probe microscopies: transient absorption for melanoma research (differentiation between melanoma and healthy skin cell) [28] and stimulated Raman scattering for lipid observation [29]. 98 References 1. Corporation, P.R., Darkfield image of anti-CD4PNPs in surfactant solution added to whole blood. 2018. 2. Jain, P.K., et al., Calculated absorption and scatteringproperties of gold nanoparticles of different size, shape, and composition: applicationsin biologicalimaging and biomedicine. J Phys Chem B, 2006. 110(14): p. 7238-48. 3. Scholler, N. and N. Urban, CA 125 in ovarian cancer. Biomark Med, 2007. 1(4): p. 513-23. 4. Gaiduk, A., et al., Detection limits in photothermalmicroscopy. Chemical Science, 2010. 1(3): p. 343-350. 5. Absil, E., et al., Photothermalheterodyne holography of gold nanoparticles. Optics Express, 2010. 18(2): p. 780-786. 6. Boyer, D., et al., Photothermalimaging of nanometer-sizedmetal particlesamong scatterers. Science, 2002. 297(5584): p. 1160-1163. 7. Berciaud, S., et al., Photothermal heterodyne imaging of individual metallic nanoparticles: Theory versus experiment. Physical Review B, 2006. 73(4). 8. Cognet, L., et al., Single metallic nanoparticle imagingfor protein detection in cells. Proceedings of the National Academy of Sciences of the United States of America, 2003. 100(20): p. 11350-11355. 9. Bouma, B.E. and G.J. Tearney, Handbook of optical coherence tomography. 2002, New York: Marcel Dekker. x, 741 p. 10. de Boer, J.F., R. Leitge, and M. Wojtkowski, Twenty-five years of optical coherence tomography: the paradigm shift in sensitivity and speed provided by Fourier domain OCT [Invited]. Biomedical Optics Express, 2017. 8(7): p. 3248-3280. 11. Bailey, B., et al., Enhancement of Axial Resolution in Fluorescence Microscopy by Standing-Wave Excitation. Nature, 1993. 366(6450): p. 44-48. 12. Shao, L., et al., Super-resolution 3D microscopy of live whole cells using structured illumination. Nature Methods, 2011. 8(12): p. 1044-+. 13. Gustafsson, M.G.L., et al., Three-dimensionalresolution doubling in wide-fieldfluorescence microscopy by structured illumination. Biophysical Journal, 2008. 94(12): p. 4957-4970. 14. Frohn, J.T., H.F. Knapp, and A. Stemmer, True optical resolution beyond the Rayleigh limit achieved by standing wave illumination. Proceedings of the National Academy of Sciences of the United States of America, 2000. 97(13): p. 7232-7236. 15. Chung, E., et al., Wide-field extended-resolutionfluorescence microscopy with standing surface-plasmon- resonance waves. Optics Letters, 2009. 34(15): p. 2366-2368. 16. Thorlabs. Gratings Tutorial. Available from: https://www.thorlabs.com/tutorials.cfin?tablD=OCA9A8BD- 2332-48F8-B01A-7F8BF0C03D4E. 17. Morena, J.J. and Society of Plastics Engineers., Advanced composite mold making. 1988, New York: Van Nostrand Reinhold Co. xi, 431 p. 18. AZoMaterials. The Main Principles of Lock-in Detection. 2016 [cited 2018 24 Nov]; Available from: https://www.azom.com/article.aspx?ArticlelD=13327. 19. So, P.T.C., T. French, and E. Gratton, A Frequency-DomainTime-Resolved Microscope Using a Fast-Scan Ccd Camera. Time-Resolved Laser Spectroscopy in Biochemistry Iv, Proceedings Of, 1994. 2137: p. 83-92. 20. Schwiegerling, J., Opticalspecification, fabrication, and testing. 2014. xi, 203 pages. 21. Voelz, D.G., Computationalfourieroptics : a MA TLAB tutorial. 2011, Bellingham, Wash.: SPIE Press. xv, 232 p. 22. Schmidt, J.D., Numerical simulation of optical wave propagation with examples in MATLAB. 2010, Bellingham, Wash.: SPIE. xi, 196 p. 23. Singer, W. and M. Totzeck, Handbook of OpticalSystems, Vol 2: Physical Image Formation. Handbook of Optical Systems, Vol 2: Physical Image Formation, 2005: p. 1-690. 24. Zheng, G.A., et al., Characterizationof spatially varying aberrationsfor wide field-of-view microscopy. Optics Express, 2013. 21(13): p. 15131-15143. 25. Javed, M.Y., et al., A novel MPPTdesign using generalizedpattern searchfor partialshading. Energy and Buildings, 2016. 133: p. 59-69. 26. Torczon, V., On the Convergence ofPatternSearch Algorithms. SIAM Journal on Optimization, 1997. 7(1): p. 1-25. 99 27. Kerimo, J. and C.A. DiMarzio, Photothermalimaging ofmelanin. Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing Xx, 2013. 8589. 28. Matthews, T.E., et al., Pump-Probe Imaging Differentiates Melanoma from Melanocytic Nevi. Science Translational Medicine, 2011. 3(71). 29. Saar, B.G., et al., Video-Rate Molecular Imaging in Vivo with Stimulated Raman Scattering. Science, 2010. 330(6009): p. 1368-1370. 100 Chapter 5 Analytical derivation of temperature field by a spherical particle that absorbs temporally modulated radiative energy 5.1. Introduction The photothermal detection of metal nanoparticles as an optical label for biological molecules has led to a demand for the solution of the heat conduction problem in which a homogeneous metal sphere generates heat in an infinite medium [1-4]. In order to calculate the photothermal signal, the spatial and temporal temperature distributions around the metal particles are required under excitation light with sinusoid temporal modulation. Previous work has used a point source approximation for the nanoparticle [5] to get a solution analytically [1, 2, 4] or numerically solved the heat equation without the point source approximation [3]. A previous paper by Goldenberg and Tranter [6] has given an analytical solution of a similar problem with a constant (no temporal modulation) heat generation inside a finite size sphere. The photothermal probe beam scattering does not generate new color and the signal is detected as a small fractional change in probe beam intensity (from 10-3 down to 10-6). Therefore, the pump (heating) beam is temporally modulated so that the photothermal signal appears as a modulation in the probe beam. In this chapter, I will give an exact analytical solution to the temporally modulated heat flow problem based on Goldenberg and Tranter's approach and compares the temperature field to the point source approximation result. 5.2. Analytical derivation of temperature field Consider a homogeneous sphere of radius a embedded in the infinite homogeneous medium and heat is produced in the sphere for time t > 0 at the rate A (1 + cos wt) per unit time per unit volume in Fig. 5.1. 101 Both the sphere and medium are initially at V temperature. If suffixes 1 refer to the sphere, 2 to the medium, and V, K, and k denote the temperature, thermal conductivity and diffusivity respectively, we require the solution of the boundary value problem: 1 aV = I 2aV A 2 - +-(I+ cos wt), 0 r < a, k, at r 2 r ( ar )K, t>O (5.1) 1 aV2 1 a V 2 .2 2 k2 at r 'ar ar V, = V2 = VI when t=0 . (5.2) V VV= V2 a av 2 when r = (5.3) K, a = K2 r 2 ar V, finite as r -+0 and V2 finite as r - oo . (5.4) 1 2 a Fig. 5.1. Schematic diagram for a heat flow problem: a homogeneous sphere (heat is generated inside by absorbing temporally modulated light) embedded in the infinite homogeneous medium. Suffix 1 refers to the sphere and 2 to the medium for thermal properties. Introducing the Laplace transform of V, defined by I = e-'V(t)dt = L {V(t)} and applying the standard Laplace transform theorems and pairs to above equations and boundary conditions, we can get the following subsidiary equations and boundary conditions: 102 -rqV rV, rA( 1+ a d(r 2)rq F2= 2r V k2 V = 2 dv dv when r =a. (5.6) dr dr V, finite as r ->0 and V2 finite as r-oo (5.7) where q,2 = p/k, q2 = p/k2 - The solution of (5.5), (5.6), and (5.7) is V Ak SK a sinhqr AkI + KL2 a sinhq, r] (5.8) 2 2 p K~p [ K rD(p) +KI(P2 + 2 + KI rD (p) [sinhqia- qla cosh - V Ak, a qa]exp[-q2 (r -a)] 2 2 p K p r (1 + q 2a)D(p) (5.9) [sinh q,a - qla coshqa]exp[-q2 (r - a)] + Ak, a K (p2 + w2) r (I+ q 2a)D(p) where D(p)= sinhqia- qa coshqla K2 sinhqja (5.10) 1+q2a K To get the inversions of above equations, they can be split into three parts as , = .+2 +V 3 and V7 = 2 1 + V 22 + 23 where V -V p P Ak, K, asinhqrl an ~Ak, a [sinh qa -qacoshqa]exp[-q2 (r -a)] -____ Ak Ak a [sinh q,a - q,a cosh qa]exp [-q2 (r - a)] I +[K2 a sinhq, r K (p 2 + CO2) r (1+q a)D(p) " 1 K (p2 4 ) I KI rD(p) I 2 (5.11) 103 From the table of Laplace transform [5], it follows that VII=L-1 =V. (5.12) A previous paper by Goldenberg and Tranter [6] gives -a2 1 KI 11 2 2ab. exp(-y2t 7) (sin y -y cos y)sin(ry/a) dy (5.13) +b2y2 sin2 12 K, 3 K 2 6 a2 r7r 0 y2 [(csin y y cos y)2 Kr K (k" (r l and where y =-, c=1- , b=- , -=I- 1 and i K1' K, k2 ) KaJ ~ F(p) - (c - bj)sinh f -Fcosh j . The inversion forF3 is simplified by the following theorem [7] L { #(At)} = (D(p/A) (5.14) where L {#(t)} = F(p) and A is any real positive number. From the Eq. (5.11) and Eq. (5.14) with notations in Eq. (5.13), the expression for V, is Ak aK l+(-kp/k2)]sinh[(rj)1a] + a . (5.15) V = A13 () where -f3() -1 2 2 2 [ K, I, p +(7 1w) rK, (2 +iw)2)F(p) To evaluate fA (yt) in Eq. (5.15), we apply the Mellin inversion theorem for the Laplace transformation [5]. This gives 1 1 s+I F 1 K + (kp/k2 )]sinh[(rfJ)/a] -f 3 (yt)= J e 2 + dp (5.16) 7, 2r - p2 +('w)2 rK, (p2 +(yw)2)F(p) where s is to be so large that all the singularities of the integrand lie to the left of the line (s - ioo, s +ioo) . Since the integrand in above equation has a multivalued function I which maps, for example, the value p =1 to 1, we use the contour of Fig. 5.2 with a cut along the negative real axis and Eq. (5.16) corresponds to the integral over the line AB, 104 A 3 (yt)= ABg(P)dp (5.17) /V sinh [(rfp)/a] where I e Pt 1 aK2 [1+ (klp/k 2 )] 23 (p2 +(yO)2 rK (p2 +(y, )2)F(p) For physically possible values of b and c, no zeros of F(p) occur within or on the contour of Fig. 5.2 [6]. 7 B p=iygO * A F E -D l 2- p=-iYI(Oi ---- 'A Fig. 5.2. Contour of integration. By Cauchy's residue theorem, the integral over this closed contour is 2ri times the sum of the residues at the poles p = iYc within it: 2(p)dp=& (pj)gRes27rij g 21 (5.18) =24NM, cos yrpt - cos-I N, + M, 2[ nM, aK2_ I+ f(iv wk / k2)]sinh[(r irc)1 where N, = 1 { an 2 iy, c[ rK, F(iyrw) -i c) a] { K + (-iy,k,)]s 7 1 M,2iypm rK, F(-iylw) 105 The integrals over the arcs BF, CA tend to zero as the radius of theses arcs tends to infinity [6]. The integral over the small circle ED about the origin also goes to zero as the radius of this circle goes to zero because the integrand g. 3 (p) converges to a constant value. Hence, the line integral in Eq. (5.16) can be replaced by a real infinite integral, derived from the integrals along DC and FE, together with the contribution from the residues. On DC and FE we write p = y2 exp(-i;r) and p = y2 exp(i7r) respectively and find that the integral derived from those along DC and FE is 2 g(p)dp - f exp(-y t) 2 (sin y - y cos y) sin(ry/a) dy . (5.19) 2 7' yDC+FE o)4 [(csiny2ycosy) +b 2y2 sin2y Substituting Eq. (5.17) and Eq. (5.19) into Eq. (5.18), we get the expression about f 3 (yt) as = NM s N +M, -T2ab (-2 2ex (sin y - ycos y)sin(ry/a) d , 2 2 2 1 32NtM, 2) rr y +(yI C)2 [(csin y -y cos y)2 +b y sin y] From above expression about f 3 (y~t) and Eq. (5.15), the final expression for V3 is a 2A 2,fNM-i os wt-cos_1 N,+M, 2ab exp(- 2ty) 42 (siny-ycosy)sin(ry/a) 2 ) y+(yw) [(csin yycosy)2+b2y2K1 sin2 2J1M1 r A similar procedure is applied to equation for the temperature outside the spherical particle. The final expressions for the temperature inside the sphere V, , the temperature in the center of the sphere (V )O = lim V, , and the temperature outside of the sphere V2 are 106 exp (-y2tl ) )a~ =V Ka 1 K, + 1 1r2 _2abf. (sin y -- y cosyv'isin(ry K, 3 K2 6 a2 rI e y 2 [(c sin y _ y cos y)2 +b2y2 sin2Y] dy +2 y2 (sin y - y cos y) sin(ryja) N M, cos ct -Cos-] N, + M, 2ab e 2tly, 2 2 2 2 y' +(Yi ) [(c sin y - y cos y)2 +b y sin (5.20) where N,- 1 1+ aK2 [I+ (iricok, /k2)] sinh [(r 7y) a and 2iyrp rK, F(iy1 p) -yio)/a] MI- 1+ aK2 [1+ (-ky iwo/k2)] sinh [(r 2iy,or rK, F(-yri) 2 )( 2 AF K, 1 2b exp(-y t /y,) (sin y - y cos y) K, 3 K2 6 7r y [(csiny -y cosy)2 +b2 y 2 sin2 7]dy} +{ 2 N0 M1 0 cos Wt-cos_' N10 +Mjo 2b fexp_ y2t/yK) y' (siny-ycosy) dy} K1 4 2 2 2 2 2 2 N, M10 ) 0 y +(y Co) [(c sin y - y cos y) +b Y sin y] (5.21) where N, -_ K '1+I ( 2) and M 1 +2 YEii+ (-ky/ 2iyva F K F(iyo)) 2iyw Ko F(-yKi,) 2 V 0 +aAIoa= A 1K 2 exp(-y t/y,) (sin y -y cos y)[by sin y cos-y -(csin y -ycos y)sino-y] rK, 3 K2 T y3 [(c sin y -ycosy)2 +b, 2 Y sin2Y] + aAj 2 N2M2 cos ycot -cos- N2 +M2 rK 2 2 N 2 M 2 (sin y - y cos y) sin ycoso-y - 2 4 [by (c sin y - y cos y) sino-y] -2: exp(- y t/y,) Y 2 T y4 + (Y ~)2 [(csiny -y cos y)2 +b 2y2 siny] y (5.22) where N2 = I f [siFlyn " - y Fwcosh iroexp(-- and 2irlw F(iyrw) M,- i I {[sinh -iYIo F--iywcosh) iy exp(- )-io 2iyl co F(-iylo) 107 5.3. Temperature distribution in the experimental condition (a) 4 (b) 50 15 6, 3.5 40 3 10 4. 2.5 301 -2 5 Gold Water 2 1 201 1.5 n 20 25 30 35 40 radius (nm) 200 10 Water 10 0.5 Gold A- - 100 5 n ra dius (nm) 0 0 time (ms) 0 50 100 150 200 radius (nm) Fig. 5.3. Temperature increase for gold nanoparticle in water (the center of the gold nanoparticle is put in the origin) (a) Transient temperature profile from the exact solution. (b) Comparison of temperature profile at peak time (t = 2.5 ms) between the exact solution (solid blue) and the point- source approximation (dashed red). Subset graph is the same temperature profile near gold-water interface. The temperature increase for the gold nanoparticle (diameter = 60 nm) in water with 8 kW/cm 2 laser beam intensity [8] and modulation frequency of 400 Hz has been computed using equations (5.20), (5.21), and (5.22) by numerical integration. Thermal constants for selected materials at room temperature and normal pressure are from CRC Handbook of Chemistry and Physics [9] (Kgld =317W/mK, 2 k,,d = 1.27 x 104 m /s, K,,ter = 606.52 x 10~ W/mK, ktr =1.45 x 10-' m 2/s). The heat generation rate inside the gold nanoparticle is simply calculated from its typical molar extinction coefficient [10] ( S= 3.07 x 10 M-'cm-' ), the particle volume, and the laser beam intensity. The resulting transient temperature curve from the exact solution is shown in Fig. 5.3(a). The temperature starts from at t = 0, keeps increasing, and finally follows the temporal modulation of the laser beam. The temperature profiles at the fixed time (when the temperature is peak in the center of the gold nanoparticle) are compared between the exact solution and the point-source approximation in Fig. 5.3(b). We can notice that the 108 point-source approximation over-estimates the temperature increase about twice near the gold-water interface than the exact solution. 5.4. Conclusion I have derived the exact analytical solution to the heat conduction problem in which a sphere absorbs temporally modulated laser beam and it transfers the heat into an infinite medium of different thermal properties. I confirmed that the point-source approximation over-estimates the temperature increase near the metal-water interface than the exact solution in the experimental condition. As a future work, the exact temperature distribution would be integrated into an analytical expression of the photothermal signal using the theory of light scattering from a fluctuating medium [1]. In conclusion, the exact temperature field solution may help to calculate the photothermal signal more precisely and to estimate the laser beam input threshold before the sample is damaged. References 1. Berciaud, S., et al., Photothermalheterodyne imaging of individual metallic nanoparticles: Theory versus experiment. Physical Review B, 2006. 73(4). 2. Boyer, D., et al., Photothermalimaging of nanometer-sizedmetal particlesamong scatterers. Science, 2002. 297(5584): p. 1160-1163. 3. Blum, 0. and N.T. Shaked, Prediction ofphotothermalphase signaturesfrom arbitraryplasmonic nanoparticlesand experimental verification. Light-Science & Applications, 2015. 4. 4. Cognet, L., et al., Single metallic nanoparticle imagingfor protein detection in cells. Proceedings of the National Academy of Sciences of the United States of America, 2003. 100(20): p. 11350-11355. 5. Carslaw, H.S. and J.C. Jaeger, Conduction ofheat in solids. 2nd ed. 1986, Oxford Oxfordshire New York: Clarendon Press ; Oxford University Press. viii, 510 p. 6. Goldenberg, H. and C.J. Tranter, Heat Flow in an Infinite Medium Heated by a Sphere. British Journal of Applied Physics, 1952. 3(Sep): p. 296-298. 7. Carslaw, H.S. and J.C. Jaeger, Operationalmethods in applied mathematics. 1948: Oxford University Press. 8. Skala, M.C., et al., Photothermal Optical Coherence Tomography of EpidermalGrowth FactorReceptor in Live Cells Using ImmunotargetedGold Nanospheres. Nano Letters, 2008. 8(10): p. 3461-3467. 9. Rumble, J.R. and J. Rumble, CRC Handbook of Chemistry and Physics, 98th Edition. 2017: CRC Press LLC. 10. Sigma-Aldrich. Gold Nanoparticles:Properties and Applications. 2018 [cited 2018 Mar 9]; Available from: https://www.sigmaaldrich.com/technical-documents/articles/materials-science/nanomaterials/gold- nanoparticles.html?utm source=redirect&utm medium-promotional&utm campaign=materials-science- nanomaterials-gold-nanoparticles.html. 109 Chapter 6 Conclusion and Outlook This thesis has presented novel applications of structured illumination microscopy to surface plasmon fluorescence and pump-probe scattering imaging. First, structured illumination was introduced to surface plasmon resonance fluorescence (SPRF) imaging for a high signal-to-noise and high resolution wide-field imaging technique over conventional total internal reflection fluorescence (TIRF) imaging methods. Thin gold film (40 nm) with a silica layer (SiOx, 5 nm thickness) on the coverslip was prepared by vapor deposition and diffraction limited size fluorescent beads (nominal diameter 40 nm) were spread on the gold coated coverslip and the bare coverslip for control. With the same laser input power, signal intensity for SPR in P-polarization was more than 4 times higher signal than TIR in P-polarization (TIR in S-polarization signal is even lower than that of TIR in P-polarization). Structured illumination reconstruction with deconvolution for SPRF imaging succeeded to increase the lateral resolution more than twice and convert the convert the doughnut-shape SPRF PSF into a conventional Airy disk shape PSF. Secondly, a theoretical framework for three-dimensional wide-field pump-probe structured illumination microscopy has been developed to increase the lateral resolution and enable depth sectioning (3D resolution). Assuming holographically measured optical field of the probe beam scattered by a structured pump beam, a rigorous mathematical 3D Fourier domain framework has been established on how to extract and reconstruct the 3D contents with an enhanced resolution. The proposed method was validated by computer simulation of three different sets of samples: a computer-synthesized resolution test target, the MIT logo, and actual biomolecules in HeLa cells. The results have clearly demonstrated the potential of our method to enable pump-probe microscopy to achieve more than 3 times better lateral resolution and 3D imaging capability over the conventional wide-field pump-probe system. 110 Further, I have proposed structured illumination wide-field photothermal digital phase microscopy as a better tool to diagnose ovarian cancer using CAI 25 detection. In the wide-field photothermal holographic imaging (digital phase microscopy) configuration, I have added structured illumination to solve the dynamic range issue of area type detectors and demonstrated enhanced resolution. An experimental setup has been developed to validate the proposed method: a SLM is used as a phase grating for the pump beam transient grating pattern inside the sample, a custom dot ND filter selectively attenuates the 0th order beam, and DMD is used as a configurable spatial filter for holography. I also developed intensive image processing algorithms including noise suppression, image reconstruction, and aberration correction to get a proper PSF images over the whole field of view. Proposed structured illumination wide-field photothermal digital phase microscopy setup successfully probed the suggested performance: shot-noise limited 3 times SBR enhancement and 40% lateral resolution improvement. High NA objective and angle scanning method have a potential to enable 3 times higher lateral resolution than diffraction limit and to obtain 3D depth sectioning respectively in the future. Finally, I have derived the exact analytical solution to the heat conduction problem in which a sphere absorbs temporally modulated laser beam and it transfers the heat into an infinite medium of different thermal properties. For the photothermal signal calculation, prior works used a point source approximation for the nanoparticle to get temperature distribution around the particle. The work by Goldenberg and Tranter provided an analytical solution of a similar problem but with only constant heat generation (i.e. no temporal modulation as needed in photothermal imaging) inside a finite size sphere. Following Goldenberg and Tranter's approach, I derived an exact analytical solution to the temporally modulated heat flow problem. Under the experimentally feasible condition with thermal properties from literatures, it was confirmed that the point-source approximation over-estimates the temperature increase near the metal-water interface than the exact solution in the experimental condition. This result may help to estimate the photothermal signal more precisely in the future. 111 In spite of the significant development of super-resolution imaging for fluorescence, the advances in achieving super-resolution in other contrast generation modalities are relatively limited. Furthermore, it is even harder to find non-fluorescence wide-field super-resolution technologies that fully utilize the available laser power. This thesis just showed the proof of concept possibility of high-speed super-resolution wide- field microscopy based on structured illumination microscopy in non-fluorescence imaging. The proposed method has a great potential to be applied to various pump-probe modalities and application areas (for example, transient absorption for melanoma research and carbon nano-material research, and stimulated Raman scattering for protein and lipid observation). I hope this thesis would be a helpful starting point in further advancing these fields for future researchers. 112