US008509879B2
(12) United States Patent (10) Patent No.: US 8,509,879 B2 Durkin et al. (45) Date of Patent: Aug. 13, 2013
(54) APPARATUS AND METHOD FOR (56) References Cited WIDEFIELD FUNCTIONAL IMAGING (WIFI) USING INTEGRATED STRUCTURED U.S. PATENT DOCUMENTS LLUMINATION AND LASER SPECKLE 6,208.415 B1* 3/2001 De Boer et al...... 356,450 MAGING 2003/0002028 A1 1/2003 Rice et al...... 356/39 2006, O155195 A1* 7, 2006 Maier et al...... 600,476 2006/0184043 A1* 8/2006 Tromberg et al...... 600,476 (75) Inventors: Anthony J. Durkin, Irvine, CA (US); 2006/0268241 A1* 11/2006 Watson et al...... 353/94 David Cuccia, Costa Mesa, CA (US); OTHER PUBLICATIONS Bruce J. Tromberg. Irvine, CA (US); Dunn, A. K. Simultaneous imaging of total cerebral hemoglobin Amaan Mazhar, Irvine, CA (US); concentration, oxygenation, and blood flow during functional acti Bernard Choi, Irvine, CA (US) vation, Optics Letters, vol. 28, 1 Jan 2003, pp. 28-30.* Briers, J.D., Laser Doppler, speckle, and related techniques for blood (73) Assignee: The Regents of the University of perfusion mapping and imaging. Physiological Measurement, 22 California, Oakland, CA (US) (2001), R35-R66.* MacKinnon, N., et al. Spectrally Programmable light engine for in (*) Notice: Subject to any disclaimer, the term of this vitro or in vivo molecular imaging and spectroscopy, Applied Optics, patent is extended or adjusted under 35 Apr. 10, 2005, vol. 44, No. 1 1. pp. 2033-2040.* U.S.C. 154(b) by 1294 days. Barlow, A.L., etal, Quantization of Widefield Fluorescence Imaging Using Structured Illumination and Image Analysis Software, Micros copy Research and Technique, 70: 76-84 (2007).* (21) Appl. No.: 12/016,918 Ventalon, C., et al., “Dynamic speckle illumination microscopy with Filed: Jan. 18, 2008 translated versus randomized speckle patterns. Opt. Express 14, (22) 71.98-7209 (2006).* (65) Prior Publication Data * cited by examiner US 2009/O118622A1 May 7, 2009 Primary Examiner — Unsu Jung Assistant Examiner — Daniel Huntley (74) Attorney, Agent, or Firm — Daniel L. Dawes; Marcus Related U.S. Application Data C. Dawes (60) Provisional application No. 60/985,872, filed on Nov. (57) ABSTRACT 6, 2007. An apparatus for wide-field functional imaging (WiFI) of (51) Int. C. tissue includes a spatially modulated reflectance/fluores A6 IB 6/00 (2006.01) cence imaging (SI) device capable of quantitative Subsurface (52) U.S. C. imaging across spatial scales, and a laser speckle imaging USPC ...... 600/473; 600/476; 600/479: 600/407 (LSI) device capable of quantitative subsurface imaging (58) Field of Classification Search across spatial scales using integrated with the (SI) device. The USPC ...... 600/473, 407,476, 479; 382/128; SI device and LSI device are capable of independently pro 250/459. 1; 35.9/368 viding quantitative measurement of tissue functional status. See application file for complete search history. 9 Claims, 20 Drawing Sheets
24
U.S. Patent Aug. 13, 2013 Sheet 1 of 20 US 8,509,879 B2
|-—
• CN
FIG. 1A U.S. Patent Aug. 13, 2013 Sheet 2 of 20 US 8,509,879 B2
DEMOD. & CAL. (EQS. 20, 23)
DATA FIT (EQ. 10)
FIG. 1B U.S. Patent Aug. 13, 2013 Sheet 3 of 20 US 8,509,879 B2
ORIGINAL SAMPLE FRINGE WRAPPED PATTERN
GE_LOÍTHISNOOEYH LH5OIEH
FIG. 1C U.S. Patent Aug. 13, 2013 Sheet 4 of 20 US 8,509,879 B2
ILLUMINATION: q = q (z) COS(kx+d) 1444444444444444444,A/A/A/A" X AA ?\. Afis 2 4-N-7.--N 7-N-7--Nys/A. Po(z) D C)
e 1-NS- -- a-ara Tea as - A - a -- - - A -- an A. D H as-----> -----> -----As
FLUENCE RATE: (p = p(z) COS(kx + d)
FIG. 1D U.S. Patent Aug. 13, 2013 Sheet 5 of 20 US 8,509,879 B2
ABSORPTION VARIATION EXPERIMENT 0.002 mm guag 0.12 mm
ob2 oboos oos o o2 o4 SPATIAL FREQUENCY, fx(mm)
SCA TERING WARIATION EXPERIMENT 0.32 mm.< us''< 1.8 mm a = 0.0046 mm
- P as 0.5 mm
0. 0.02 0.04 0.06 0.08 0.1 0.12 0.14 SPATIAL FREQUENCY, f(mm) FIG. 2
U.S. Patent Aug. 13, 2013 Sheet 7 of 20 US 8,509,879 B2
REFLECTANCE IMAGE + ROI (650 nm) ;
AMPSAT 650 nm (I/nm) us' AMPSAT 650 nm (I/nm)
0.05
0.04 0.03
R. Sm. 0.02 MEAN=0.034382, SID=0.0048508 6000 ? 4000 D C O 2000
O O 0.02 0.03 0.04 O,O5 O,06 0.07 0.02 0.03 0.04 0.05 0.06 0.07 u (mm) us' (mm) FIG. 4B U.S. Patent Aug. 13, 2013 Sheet 8 of 20 US 8,509,879 B2
9.6
9.4
9.2
8.8 FIG. 5A 86
-5 0 5 O 15 20 25 30 TIME (min)
0.64
0.63
0.62
0.61 5 to 0.6 S.
0.59
FIG. 0.58
-5 O 5 10 15 20 25 30 TIME (min) U.S. Patent Aug. 13, 2013 Sheet 9 of 20 US 8,509,879 B2
AOHb (uM) AHHb (uM) ATHb (uM) Asto (%)
10 15 20 25 30 TIME (min) FIG. 5C U.S. Patent US 8,509,879 B2
?I-(u?u)BWIL U.S. Patent Aug. 13, 2013 Sheet 11 of 20 US 8,509,879 B2
MAP, 650 nm 5mm FIG. 6A
ABSORPTION VS. WAVELENGTH 0.055 O.05 0.045 0.04 0.035 0.03 FIG. 6B 0.025 0.02 0.015
0.0050.01 S 600 650 700 750 800 85O 900 950 1000 1050 WAVELENGTH (nm)
REDUCED SCATTERING VS. WAVELENGTH 1.3 1.2 1.1 1. 0.9 FIG. 6C 0.8 0.7 0.6 0.5 600 650 700 750 800 85O 900 950 1000 1050 WAVELENGTH (nm) U.S. Patent Aug. 13, 2013 Sheet 12 of 20 US 8,509,879 B2
FIG. 7A
5 to 15 2025.303s 4045 m
FLUORESCENCE (G) f = 0.21mm x10'
FIG. 7B
5 O 15 20 25 30 35 40 45 n
FLUORESCENCE (G) f = 0.45mm
FIG. 7C
5 O 15 20 25 30 35 40 45 U.S. Patent Aug. 13, 2013 Sheet 13 of 20 US 8,509,879 B2
1. UM CY5.5
lar.01, his ~ 1. G) 660nm O. UM CY5.5
FIG. 7D
INCREASING SPATIAL FREQUENCY
O 2 3 4. 5 6 DISTANCE (mm) FIG. 7E. U.S. Patent Aug. 13, 2013 Sheet 14 of 20 US 8,509,879 B2
FLUORESCENCE IMAGE ABSORPTIONG 660nm
f s 4 O.O2 3 5 8. S < S. 2 O. O. O. >- C O ne' 0.01
FIG. 6D "" ; ;QUADRANT ; ; U.S. Patent Aug. 13, 2013 Sheet 15 of 20 US 8,509,879 B2
VENULES TUMOR ARTERIOLES 30 mm/s
is 8000 9, 7000 C 2 D O 2 N1 U a.
f O 2 4 6 8 10 2 14 16 1820 O 2 3 4. 5 6 ACTUAL FLOW RATE mm/s) ACTUAL FLOW RATE mm/s)
? 500000 ?ts, 80000 400000 CY 2 2, 60000 as 300000 O 5 85 40000 3 200000 6 a. ein 20000 100000 T = 10 us i T = 100 us - l f O O O 50 100 150 200 250 300 O 50 100 150 200 250 300 ACTUAL FLOW RATE Imm/s) ACTUAL FLOW RATE Imm/s)
FIG. 9 U.S. Patent Aug. 13, 2013 Sheet 16 of 20 US 8,509,879 B2
o Co o O o O Cd Co f C O N N o ser
i. s U.S. Patent Aug. 13, 2013 Sheet 17 of 20 US 8,509,879 B2
ANALYSIS WORKSTATION FIG. 11A
DUAL WAVELENGTHLED
(785nm,820mm) IMAGING STAGE
FIG. 11B
L2 Ay LP 1 11 || Y IN NJ /1 || 1 U.S. Patent Aug. 13, 2013 Sheet 18 of 20 US 8,509,879 B2
FIG. 12A ANALYSIS
WORKSTATION
DUAL WAVELENGTH LASER (785nm,82Onm)
CUSTOM LIGHT LIGHT GUIDE ENGINE (APOGEN) FILTER WHEEL
BROADBAND SOURCE FILTER (TUNGSTEN HALOGEN) WHEEL
IMAGING STAGE 8 ENCOSURE
U.S. Patent Aug. 13, 2013 Sheet 19 of 20 US 8,509,879 B2
SNAPSHOT FIG. 13A ANALYSIS WORKSTATION
DUAL WAVELENGTH LASER (785nm,820mm)
BROADBAND SOURCE CUSTOM LIGHT ran-ras A re- IGHT GUIDE (TUNGSTEN HALOGEN) ENGINE (APOGEN) CTF CIS w M DUAL WAVELENGTH LED (785nm,820mm) ARTICULATING ARM
FIG. 13B U.S. Patent Aug. 13, 2013 Sheet 20 of 20 US 8,509,879 B2
sassisssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssarsssssssssssssssssssssssssarrarararass
s
&
US 8,509,879 B2 1. 2 APPARATUS AND METHOD FOR BRIEF SUMMARY OF THE INVENTION WIDEFIELD FUNCTIONAL IMAGING (WIFI) USING INTEGRATED STRUCTURED We disclose a wide-field functional imaging (WiFI) with LLUMINATION AND LASER SPECKLE the objective of developing an integrated imaging platform MAGING capable of quantitative Subsurface metabolic imaging across spatial scales. WiFI simultaneously measures tissue blood RELATED APPLICATIONS flow, biochemical composition (i.e. oxy- and deoxy-hemo globin, water and lipid content), and molecular fluorescence The present application is related to U.S. Provisional in turbid tissues. It possesses sufficient spatio-temporal reso Patent Application Ser. No. 60/985,872, filedon Nov. 6, 2007, 10 lution to study both fast (i.e., ms timescale) and localized (i.e., which is incorporated herein by reference and to which pri tens of um to mm) events at depths of several millimeters in ority is claimed pursuant to 35 USC 119. thick tissues. This platform enables quantitative insight into disease progression and therapeutic response in areas such as GOVERNMENT RIGHTS wound healing, neuroscience and cancer. 15 We disclose the design specifications and fabrication ele This invention was made with government Support under ments necessary to construct a series of instruments based on grant RR001 192 awarded by the National Institutes of the integration of two wide-field imaging modalities; Spa Health. The government has certain rights in the invention. tially modulated imaging (MI) or structured reflectance/fluo rescence illumination (SI) and laser speckle imaging (LSI). BACKGROUND OF THE INVENTION WiFI is based on concepts from SI and LSI: technologies that independently provide quantitative insight into tissue 1. Field of the Invention functional status. Our preliminary results demonstrate the The invention relates to the field of medical imaging using ability of each modality to quantitatively characterize bio light. logical tissue. The complementary nature of the two imaging 2. Description of the Prior Art 25 modalities, in terms of extracted tissue functional character Due to its relatively low cost and ease of implementation, istics and similarities in required hardware Support, drives the optical imaging is an attractive technology to study intrinsic design specifications for WiFI instrumentation. We combine signals associated with endogenous chromophores as well as these modalities in order to develop integrated WiFI instru targeted exogenous probes. With the explosive growth in mentation capable of absolute depth resolved quantification available molecular reporter strategies for studying funda 30 of tissue absorption, Scattering, fluorescence, and blood flow. mental biophysical processes, there has been a paradigm shift WiFI provides researchers with a quantitative tool to study in research efforts from ex vivo destructive evaluation to in disease progression and therapeutic response with 1) a high Vivo analysis, allowing for characterization of dynamic bio degree of fidelity and spatial localization, and 2) Sufficient logical processes and for each animal to serve as its own spatiotemporal resolution and probe Volume to study events control. Despite these tremendous advances in molecular 35 on length scales that have broad biologic and clinical rel imaging, absolute quantification of the magnitude and origin evance (i.e., mm-cm). With WiFI the ambiguity that exists in of cellular and molecular events remains a significant chal planar imaging modalities (between molecular reporter depth lenge. and signal strength) will be overcome, resulting in absolute In the neuroscience community, optical imaging of intrin measurements of signal and more accurate comparisons of sic signals has long been used to study the organization and 40 multiple experimental conditions. The knowledge of both functional architecture of different cortical regions in animals local metabolic activity and molecular reporter dynamics will and humans. Local changes in intrinsic signals have been result in an improved understanding of cell-vascular coupling attributed to an increase or decrease in local neurovascular phenomena. With absolute quantification of local oxygen activity, but separation of these signal dynamics into basis saturation and blood flow, researchers will be able to draw components such as oxy/deoxygenation of blood, changes in 45 comparisons among data collected in serial measurement blood Volume, and optical scattering, has not been performed, sessions on a single patient and among patients measured at resulting in an incomplete picture of the underlying mecha different sites worldwide. Furthermore, with absolute quan nisms. Technologic advances such as combined reflectance? tification of tissue parameters, we envision the possibility of fluorescence imaging, multi-parameter full-field imaging, WiFI-based epidemiologic studies to facilitate development and laminar optical tomography enable improved separation 50 of physiologically meaningful quantitative metrics of tissue of the signals to study important parameters such as local function (i.e., “normal vs. "abnormal blood flow). tissue metabolic dynamics; however, these technologies cur We intend to focus our WiFI instrument fabrication efforts rently can provide only relative changes in hemodynamic to address specific preclinical and clinical needs. The illus parameters and do so without consideration of optical scat trated embodiment includes a real-time optical neuroimaging tering effects on extracted tissue parameters. 55 instrument. A multimodal neuroimaging instrument is pro In U.S. Pat. No. 6,958,815 we presented a disclosure vided designed to perform fast and quantitative optical meta involving wide field, broadband, spatially modulated illumi bolic imaging of the brain. This system combines reflectance? nation of turbid media. This approach has potential for simul fluorescence SI and LSI imaging techniques for the first time. taneous Surface and Subsurface mapping of media structure, In order to optimize speed, WiFI instrument 1 is a small function and composition. This method can be applied with 60 field-of-view (1 cmx1 cm), dual wavelength (LED or laser), no contact to the medium over a large area, and could be used dual-frequency system targeted at real-time (20fps) measure in a variety of applications that require wide-field image ment, analysis, and visualization of dynamic neural signals characterization. The approach described in U.S. Pat. No. such as stroke and epilepsy. The system is based on a LCOS 6,958,815 and a fluorescence imaging capability described in spatial light modulatorin order to permit laser projection with U.S. patent application Ser. Nos. 11/927,396 and 1 1/336,065, 65 a motion-free system (preventing speckle dephasing in the each incorporated herein by reference is further refined in the instrument itself). In order to achieve maximal acquisition present disclosure. rates, both projection and detection arms is spectrally multi US 8,509,879 B2 3 4 plexed with a dichroic combiner and splitter, respectively. motion artifacts or discomfort to the patient. The projection Dual-CCD detection will be provided by Dalsa Pantera 1M60 Subsystem is comprised of a separate DMD light engine from camera-link devices. Both cameras and the LCOS develop Apogen geared toward lightweight construction (no filter er's board are synchronized at 60 frames per second via wheels, magnesium exoskeleton, and fiber light guide tung hardware triggering, projecting 3 phase patterns at a rate of 20 Sten source delivery), and is designed for integration with the HZ. Acquired data will be 1) frequency-demodulated, and 2) CTIS/Dalsa imaging arm. The Apogen light engine and Dalsa calibrated, then 3) processed into absorption and reduced Pantera 11 M04 camera are synchronized via the same under scattering optical property maps using an established rapid lying platform developed for WiFI Instruments 1 and 2. We lookup table approach, then 4) processed into chromophore constructed this system initially with a liquid crystal tunable maps with linear spectral analysis of the multispectral absorp 10 filter (LCTF), later replaced with the CTIS upon delivery of tion data. Parallelized code operating on an eight-core work the device. A graphics processor unit (GPU) provides accel station performs these four processing components simulta eration of the CTIS tomographic reconstruction code. While neously, with computational power to spare for user-GUI acquisition with the proposed system is less than 1 s, the CTIS interaction and visualization. The acquisition, control, pro reconstruction step is currently limited to more than 1 min/ cessing and visualization code for this instrument is based on 15 image for a total of more than 3 min computational time. the MI Inc. CH/C++ acquisition framework. The “measure While this delay in feedback is acceptable for longitudinal ment loop' for this instrument incorporates tight synchroni studies of chronic disease progression and therapeutic Zation of hardware and analysis components to achieve the 20 response, it is incompatible with applications geared at fps frame rate, which requires the development of hardware informing a physician while monitoring an acute therapy, specific drivers as well as analysis code in C#. Such as port-wine stain treatment, and resection of cancerous A Small animal tomographic imaging instrument (WiFI tissues in brain, melanoma, and breast cancer Surgeries. We instrument 2) is fabricated for the primary purpose of tumor utilize the programmable, massively data-parallel nature of angiogenesis studies. A light-tight enclosure allows 3D mea GPUs to solve the CTIS expectation-maximization (EM) Surement, analysis, and visualization of endogenous and problem (a naturally-parallel algorithm). This is imple exogenous fluorescence, absorption and scattering contrast. 25 mented using the CUDA programming model by nVidia, an The system is based on a custom-built, digital micromirror abstracted set offloating-point libraries aimed at general pur device (DMD)-based light engine (DVImage developer's kit, pose GPU computation (GPGPU). Preliminary reconstruc Apogen Inc.) for near-infrared (NIR) structured light illumi tions of CTIS data have yielded long reconstruction times (1 nation. For detection, this system incorporates a back-illumi minute per image). A work station with GPUs will reduce the nated, electron-multiplying, linear-gain CCD (QuantEM, 30 reconstruction time of the CTIS to allow near real-time (1 fps) Photometrics Inc.). Dual filter wheels are placed in front of quantitative hyper-spectral imaging. source and detector to allow a flexible combination of multi Thus, the illustrated embodiments include an apparatus for spectral reflectance and fluorescence measurements. Special wide-field functional imaging (WiFI) of tissue comprising: a care is taken with filter wheel alignment and stray light rejec spatially structured reflectance/fluorescence illumination tion in order to maximize fluorescence contrast. The Apogen 35 (SI) device capable of quantitative subsurface imaging across light engine, QuantEMCCD camera, and both filter wheels spatial scales; and a laser speckle imaging (LSI) device are synchronized via the same underlying platform, devel capable of quantitative Subsurface imaging across spatial oped for WiFI Instrument 1. A custom Computed Tomo scales using integrated with the (SI) device. graphic Imaging Spectrometer (CTIS) is incorporated in the The SI device and LSI device are capable of independently instrument in order to facilitate hyperspectral tomography 40 providing quantitative measurement of tissue functional sta and simultaneous visualization of multiple fluorophores. This tuS. instrument serves as a testbed platform for the development/ The SI device and LSI device when integrated together are visualization of tomographic algorithms and depth contrast capable of absolute depth resolved quantification of tissue information in the small animal ATK. The added challenge absorption, Scattering, fluorescence, and blood flow. for this instrument is 3D volumetric and cross-sectional visu 45 The SI device and LSI device when integrated together are alization and measurement tools. This is implemented using capable of quantitative measurement of disease progression the existing MI Inc. rendering engine based on Microsoft’s and therapeutic response with 1) resolution and spatial local Managed DirectX platform. ization, and 2) Sufficient spatiotemporal resolution and probe A fast, clinic-friendly imaging instrument (WiFI Instru Volume to quantitatively characterize biological events in in ment 3) is fabricated for therapy guidance and wound healing 50 Vivo tissue on mm-cm length scales. monitoring. Instrument 3 is a clinic-friendly 'snapshot’” The SI device and LSI device when integrated together are hyperspectral (500-1000 nm) system, capable of broadband capable of quantitative measurement of both local metabolic spatial-frequency-domain imaging on a sub-second times activity and molecular reporter dynamics. cale. The device enables mapping of the spatial distributions The SI device and LSI device when integrated together are of hemoglobin, lipid, water, and tissue scattering in layered 55 capable of quantitative measurement of absolute quantifica tissue systems. This lightweight system is mounted on an tion of local oxygen Saturation and blood flow. articulating arm to allow arbitrary positioning for a variety of The SI device and LSI device when integrated together are clinical applications, including flap and diabetic wound capable of quantitative measurement of absolute quantifica monitoring, melanoma studies, and port-wine stain imaging. tion of tissue parameters. The key component of this system is a custom holographic 60 The SI device and LSI device when integrated together are computed tomographic imaging spectrometer (CTIS). In capable of quantitative measurement of real-time optical neu combination with the 2Kx4K Dalsa 11 M04 camera, the 7-or roimaging. der filterprovides ~5 nm spectral resolution of absorption and The SI device and LSI device when integrated together are scattering across the entire spectral range from 500-1000 nm, capable of quantitative measurement of real time quantitative all with only three phase projection images. This allows Snap 65 optical metabolic imaging of the brain. shot clinical measurements and multiple chromophore map The integrated SI and LSI devices comprise a system char extraction before, during, and after therapies with minimal acterized by a small field-of-view of the order of 1 cmx1 cm, US 8,509,879 B2 5 6 a dual wavelength or dual-frequency probe, and means for tering across a spectral range from 500-1000 nm with only generating data maps at real-time rate of at least 20 fps. three phase projection images to allow real time clinical mea The apparatus is further characterized as an integrated Surements and multiple chromophore map extraction before, system capable of analysis and visualization of dynamic neu during, and after therapies with minimal motion artifacts or ral signals including as stroke and epilepsy. discomfort to a patient. The apparatus comprises a laser, and a LCOS spatial light The apparatus comprises a projection Subsystem including modulator to permit laser projection as a motion-free system a separate DMD light engine integrated with the articulating by preventing speckle dephasing in the integrated device a. itself. a. The illustrated embodiments of the invention further The apparatus comprises projection and detection arms 10 include methods for operating or performing the measure which are both spectrally multiplexed with a dichroic com ments of each and any one of the above apparatus. biner and splitter, respectively. While the apparatus and method has or will be described The apparatus comprises a light Source and a dual-CCD for the sake of grammatical fluidity with functional explana detector and a developer's board synchronized to each other tions, it is to be expressly understood that the claims, unless at least at 60 frames per second via hardware triggering, 15 expressly formulated under 35 USC 112, are not to be con projecting 3 phase patterns of light from the light Source at a Strued as necessarily limited in any way by the construction of rate of at least 20 Hz. “means' or “steps” limitations, but are to be accorded the full The apparatus comprises a computer or data circuit for Scope of the meaning and equivalents of the definition pro acquiring data, frequency-demodulating the data, calibrating vided by the claims under the judicial doctrine of equivalents, the demodulated data, processing the demodulated and cali and in the case where the claims are expressly formulated brated data into absorption and reduced scattering optical under 35 USC 112 are to be accorded full statutory equiva property maps and processing the optical property maps into lents under 35 USC 112. The invention can be better visual chromophore maps with linear spectral analysis of the mul ized by turning now to the following drawings wherein like tispectral absorption data. elements are referenced by like numerals. The computer or data circuit operates with parallelized 25 code to perform simultaneous processing and to provide user BRIEF DESCRIPTION OF THE DRAWINGS GUI interaction and visualization. The integrated SI and LSI devices comprise a system char FIG. 1a is a simplified block diagram of an instrument acterized as a Small animal tomographic imaging instrument platform for performing the methodology of the invention. having a light-tight enclosure allowing 3D measurement, 30 FIG. 1b demonstrates a quantum yield computation. The analysis, and visualization of endogenous and exogenous phantom illustrated in these photograph measured by SI con fluorescence, absorption and scattering contrast. sists of 4 quadrants of increasing fluorophore concentrations. The integrated SI and LSI devices are capable of making In the top row a fluorescence image is a concentration depen tumor angiogenesis measurements. dent measure of a fluorophore. In the second row is an absorp The apparatus comprises a digital micromirror device 35 tion map at 660 nm show the same trait, and the bottom row (DMD)-based light engine for near-infrared (NIR) structured is a quantum yield map which provides a concentration inde light illumination. pendent measure of fluorophore. The apparatus comprises a back-illuminated, electron FIG. 1c is a 3D graph of the measured spatially-varying multiplying, linear-gain CCD as a detector. phase, yielding topological data about the 3D tissue surface. The apparatus comprises a Source, a detector, and dual 40 FIG. 1d is a diagram which depicts the diffuse propagation filter wheels in front of the source and detector to allow a of a normally-incident, periodically-varying plane wave flexible combination of multispectral reflectance and fluores source with spatial frequency f. and spatial phase C, giving cence measurements, where filter wheel alignment is main rise to a diffuse fluence rate with the same frequency and tained and stray light rejected to maximize fluorescence con phase. trast, where the source, detector, and dual filter wheels are 45 FIG. 2 is a graph of absorption (left) and reduced scattering mutually synchronized with each other. (right) optical properties measured using SI from sixteen The apparatus comprises a computed tomographic imag turbid phantoms. ing spectrometer (CTIS) incorporated with the integrated SI FIG. 3 is a graph of the diffuse reflectance (left), multi and LSI device to facilitate hyperspectral tomography and wavelength absorption (center) and reduced scattering (right) simultaneous visualization of multiple fluorophores. 50 properties of a typical in-vivo flap obtained 48 hrs post sur The integrated SI and LSI devices comprise a system char gery in a rat model. acterized as a real time, clinical, imaging instrument capable FIG. 4a is a digital map of the planar reflectance and of therapy guidance and wound healing monitoring. 3.8x5.9 ROI window of cortex at 650 nm. FIG. 4b is quanti The integrated SI and LSI devices are characterized as a tative absorption (left) and reduced scattering (right) maps hyperspectral (500-1000 nm) system, capable of broadband 55 and image histograms. Spatial modulation data were acquired spatial-frequency-domain imaging on a sub-second times at two spatial frequencies of 0 and 0.13 mm over a 5x7 cale. (VxH) mm field of view. For baseline measurements, data The integrated SI and LSI devices are capable of mapping were acquired at 10 nm intervals over the entire range of the spatial distributions of hemoglobin, lipid, water, and between 650 and 980 nm, using a 10 nm bandwidth liquid tissue scattering in layered tissue systems. 60 crystal tunable filter camera. The apparatus further comprises an articulating arm to FIGS. 5a-5d are directed to a cortical spreading depression allow arbitrary positioning for a variety of clinical applica experiment, induced by 1MKCl administration to the cortical tions, including flap and diabetic wound monitoring, mela surface. FIG. 5a is the absolute absorption and FIG. 5b is the noma studies, and port-wine stain imaging. reduced scattering coefficients within the region of interest The integrated SI and LSI devices comprise a holographic 65 versus time, for 780 and 830 nm. FIG. 5c shows cerebral computed tomographic imaging spectrometer (CTIS) with a OHb, HHb, THb and stO within the region of interest versus camera capable of spectral resolution of absorption and scat time. FIG. 5d are chromophore maps at characteristic time US 8,509,879 B2 7 8 points, demonstrating spatio-temporal components of hemo claims. It is expressly understood that the invention as defined dynamic changes oxygenation and blood Volume. FIG. 5e is by the claims may be broader than the illustrated embodi a map of reduced scattering coefficient dynamics (AL) at 800 ments described below. nm, exhibiting biphasic wave propagating at 2.1 mm/min. FIG. 6a is a reconstructed absorption coefficient map DETAILED DESCRIPTION OF THE PREFERRED shortly after injection of one mL of Stock nigrosin, with a L EMBODIMENTS of about 1 mm at 650 nm. FIG. 6b is a graph of average absorption, and FIG. 6c is a graph of reduced scattering Quantitative characterization of tissue structure and func spectra of background (pre-injection) solution of nigrosin, tion across spatial scales is one of the most challenging prob Intralipid and water with properties of u=0.01 mm and 10 lems in medical imaging. Field of view, depth of interroga u."—1.1 mm at 650 nm. tion, and resolution are critical features that dramatically FIGS. 7a-7e are the first demonstration of modulated fluo impact image quality and information content. Optical meth rescence imaging for depth discrimination of fluorophores. ods can potentially provide a single platform for imaging FIGS. 7a-7c are photographs of demodulated fluorescence biological tissues with resolution and depth sensitivity from images at a low, middle, and high spatial frequency, which 15 microns to centimeters, limited by fundamental light-tissue demonstrate background Suppression and increased sensitiv interactions. ity to surface structures at higher frequencies. FIG. 7d is a The broad advantage of the wide-field functional imaging diagram of the phantom which was used. FIG. 7e is a graph of (WIFI) core is that it is an integrated imaging platform the intensity profiles of the far right fluorescent bead in FIG. capable of quantitative Subsurface metabolic imaging across 7d with increasing spatial frequencies and also shows a spatial scales. WiFI simultaneously measures tissue blood decrease in FWHM Suggesting resolution improvements. flow, biochemical composition (i.e. oxy- and deoxy-hemo FIGS. 8a-8d demonstrate a quantum yield computation. globin, water and lipid content), and molecular fluorescence The phantom illustrated in these photograph measured by SI in turbid tissues. It possesses sufficient spatio-temporal reso consists of 4 quadrants of increasing fluorophore concentra lution to study both fast (i.e., ms timescale) and localized (i.e., tions. FIG. 8a that a fluorescence image is a concentration 25 tens of um to mm) events at depths of several millimeters in dependent measure of a fluorophore. FIG. 8b is an absorption thick tissues. This platform provides quantitative insight into map at 660 nm show the same trait, and FIG. 8c is a quantum disease progression and therapeutic response in areas such as yield map which provides a concentration independent mea wound healing, neuroscience and cancer. sure of fluorophore. FIG. 8d is a graph where the average The illustrated embodiment includes a series of instru quantum yield and average absorption for all pixels in the 30 ments based on the integration of two wide-field imaging respective quadrants are plotted. Quantum yield is graphed on modalities that have been recently developed: (1) spatially the left axis and shown by the upper plots. The absorption modulated reflectance/fluorescence imaging (SI) and (2) coefficient is graphed on the right axis and are the plots laser speckle imaging (LSI). This broad objective is accom connected by a linear graph line. The starred quadrant 1 plished by using the following four specific aspects: clearly has a significantly different quantum yield than quad 35 a. models of light propagation and WiFI contrast/resolution rants 2-4. developed using heterogeneous tissue-like phantoms FIG.9 is four graphs of background-corrected speckle flow and appropriate numerical simulations. Phantom studies index values are affected by speckle image exposure time. characterize the contrast, resolution and quantization of The data show that the blood vessel types (tumor flow, arte SI and LSI signals. Specific issues include origins and rioles/venules, arteries/veins) that can be assessed in a linear 40 co-registration of reflectance, fluorescence, and speckle; fashion depends critically on image exposure time. spatial sensitivity maps; and tomographic WiFI capa FIG. 10 is a series of micrographs of vascular remodeling bilities. and blood flow dynamics were evident during the 21-day b. WiFI instrument embodiments targeting: a) real-time (20 monitoring period, with the Day 0 and Day 21 structural frames per second) optical neuroimaging; b) whole images having similar appearances. 45 body, Small animal tomographic imaging; and c) clinic FIG.11a is a schematic of WiFI instrument 1 which is a friendly Snapshot (1 fps) spectroscopic imaging. Each small field-of-view (1 cmx1 cm), dual wavelength (LED or system is fabricated to address the specific spatial, tem laser), dual frequency system. FIG. 11b is a diagram of a poral, and functional contrast requirements for each LCOS spatial light modulator to permit laser projection with unique application. a motion-free system. 50 c. The preclinical use of the WiFI platform to study essen FIG.12a is a block diagram of a small animal tomographic tial quantitative hemodynamic, metabolic, and cellular imaging instrument (WiFI instrument 2) which is fabricated processes in vivo. As an illustration Disease progression for the primary purpose of tumor angiogenesis studies. FIG. through acute and chronic models of ischemic stroke, 12b is a photograph of a custom-built, digital micromirror epilepsy (Instrument 1) and tumorangiogenesis (Instru device (DMD)-based light engine for near-infrared (NIR) 55 ment 2) while quantifying therapeutic response to neu structured light illumination used in the system of FIG. 12a. roprotective agents and chemotherapies. FIG. 13a is schematic of WiFI Instrument 3, which is a d. The clinical use of the WiFI platform as a noninvasive clinic-friendly imaging instrument. FIG.13b is photograph of diagnostic and therapy monitoring tool. With Instrument an embodiment of the clinical device of FIG. 13a with the 3, pilot studies related to both guiding therapy (i.e., port light engine mounted on articulated arm and with real-time 60 wine stain, neuroSurgery, skin cancer) and monitoring visualization. wounds (i.e., port wine stain, flap monitoring, diabetic FIG. 14 is a table showing three versions of the WiFI ulcers) can be performed. instrumentation platform. The WiFI instruments of the illustrated embodiments pro The invention and its various embodiments can now be Videa translational research tool with abroad medical impact. better understood by turning to the following detailed 65 Imaging cameras are ubiquitous in medicine, from Surgical description of the preferred embodiments which are pre microscopy and endoscopy to image inspection and docu sented as illustrated examples of the invention defined in the mentation. WiFI takes advantage of the unique light-tissue US 8,509,879 B2 9 10 interactions that are known to occur with spatial, spectral, and tially modulated light. The projector's color filter wheel 18 temporal modulation and can be decoded by SI and LSI was removed, producing abroadband “white light' illumina analytical methods. Thus, WiFI is expected to replace con tion of the sample. Interference filters can be placed for nar ventional camera-based imaging and allow viewing of row detection of a specified wavelength. The diffusely entirely new functional tissue attributes beneath the surface, reflected light is captured by a 16-bit frame-transfer CCD where disease typically begins. With the continued emer camera 22 (Roper Cascade 512F at 512x512 resolution). gence ofNIR fluorescent probes, WiFI integration of intrinsic Cross-linearized polarizers 20 are also introduced at the and extrinsic contrast elements to achieve functional tomog source 16 and detector 22 to eliminate specular reflectance. raphy in pre-clinical animal models is relevant and timely. Camera 22 is coupled to computer and display 24 which The eventual FDA approval of molecular-targeted fluorescent 10 controls platform 10 to provide Scanned maps and to process probes will introduce novel fluorescence methods for track ing biological processes in humans. As in the case of positron the data according to the disclosed methodology to produce emission tomography (PET), these exogenous imaging the maps of the figures using conventional Software and the agents will require well-matched technologies, such as those disclosed conventional algorithms. used in the WiFI technology core. 15 FIG.1b photographically displays the typical data process Consider the significance of wide-field functional imaging ing flow chart for spatially modulated illumination in the case (WiFI). With the first demonstration in 1991 of functional of an in vivo measurements of a human forearm shown in magnetic resonance imaging to study cerebral hemodynam FIG. 1b, leftmost column, f.i. Intensity data at each fre ics the use of imaging methods to study biological processes quency f. f.s and fit, (3 phase images per frequency) are has experienced explosive growth. In particular, noninvasive, demodulated in the top row of FIG. 1b, calibrated in the in vivo imaging of Small animal preparations is a rapidly second row of FIG.1b, and fit to yield the spatial maps of the growing field of biomedical research. absolute absorption coefficient LL and reduced scattering Consider first modulated imaging (MI) or more generally coefficient L. Data are processed separately for each pixel, structured illumination (SI), which can be considered as a generating spatial maps of optical properties as seen in the conventional but recently developed technology. Recently, 25 bottom row of FIG. 1b. Note the differential contrast in dif biomedical Scientists have developed Stand alone technolo fuse reflectance (R) versus spatial frequency (f) is the basis gies including structured illumination (SI) and laser speckle for quantitative separation absorption and scattering. The imaging (LSI), that when integrated into a single instrument, processing of the data in SI uses conventional algorithms. See can resolve these shortcomings. SI has the unique capability D. J. Cuccia, F. Bevilacqua, A. J. Durkin and B. J. Tromberg, of spatially resolving optical absorption and scattering 30 parameters, allowing wide-field quantitative mapping of tis "Modulated Imaging: Quantitative Analysis and Tomography sue optical properties. While compatible with temporally of Turbid Media in the Spatial-frequency Domain.” Opt Lett modulated photon migration methods, SI alternatively uses 30 (11), 1351 b-1356 (2005). spatially modulated illumination for imaging of tissue con Four evenly spaced frequencies between 0 and 0.15 mm stituents. Periodic illumination patterns of various spatial 35 were collected at a wavelength of 61b.0 nm. The differential frequencies are projected over a large (e.g. many cm) area of contrast observed as illumination frequency increases is the a sample. The reflected image differs from the illumination basis for the quantitative separation of absorption and scat pattern due to the optical property characteristics of the tering. As shown in the final absorption map of FIG. 1b, vein sample. Typically, sine-wave illumination patterns are used in structure can be clearly visualized due to absorption contrast. modulated imaging, or at least analytically modulated illumi 40 Also, a vertical feature of lower scattering is evident in the nation patterns. Structure illumination includes the notion of middle of the scattering map on the bottom right image of modulated imaging, but is further generalized to contemplate FIG. 1b, which is coincident with a large superficial tendon. any kind of structured pattern of illumination, whether ana A detailed description of the SI method including spatial lytical or not. The demodulation of these spatially-modulated frequency domain measurement, calibration, and analysis has waves characterizes the sample modulation transfer function 45 been previously reported and are treated here as conventional (MTF), which embodies the optical property information. though state-of-the-art methods. See D. J. Cuccia, et. al. Light from a halogen lamp is expanded onto a spatial light “Quantitative Mapping of Turbid Media Optical Properties modulator (SLM). The current system uses a digital micro Using Modulated Imaging.” J. Biomed. Opt. (In Press); and mirror device (DMD) from Texas Instruments, which is a D. J. Cuccia, “Modulated Imaging: A Spatial Frequency 1024x768 binary mirror array which generates arbitrary 50 Domain Imaging Method for Wide-field Spectroscopy and grayscale patterns. Such patterns are directed to the tissue Tomography of Turbid Media.” University of California, Irv surface. The diffusely reflected light is then recorded by a ine. (2006). Here, we outline the key concepts important for digital CCD camera. A filter wheel or tunable filter is used to this disclosure. As disclosed in copending application Ser. interrogate a discrete number of wavelengths. Crossed linear No. 1 1/927,396, filed on Oct. 29, 2007, and incorporated polarizers can be introduced into the Source and detection 55 herein by reference, tissue is illuminated with a spatial pattern light paths to remove specular reflectance. of the form: The SLM, CCD and filter wheel are synchronized with a computer, enabling fast acquisition of a series of patterns at various spatial frequencies. A TiO-based silicone reflectance S = 1 + Mocos(2it fix + a) (1) standard is used to calibrate the source intensity and to correct 60 for spatial nonuniformities in both the illumination and imag ing Systems. where S.M. f. and C. are the illumination source intensity, FIG. 1a illustrates the platform generally denoted by ref modulation depth, spatial frequency, and spatial phase, erence 10 used for spatial modulation of NIR light. A simple respectively. The diffusely reflected intensity, I, is a sum of digital projector 12 (NEC HT1000), based on a digital micro 65 AC and DC components, where the measured AC component mirror DLP light engine (Texas Instruments), and a UHP of the reflected intensity, I can be modeled as IM (X, mercury lamp 16 are used to generate the structured or spa f)cos(27tf+C). Here, M.(X, f) represents the amplitude of US 8,509,879 B2 11 12 the reflected photon density “standing wave' at frequency f. a spectrally-filtered source in combination with source-rejec Note that M can be a function of position, X. tion in the detection arm. In the illustrated embodiment, we To obtain M(X, f), we employ a conventional simple have built a fluorescence light engine with filter wheels in time domain amplitude demodulation method, see A. B. Carl both source and detection arms allowing us to quantify reflec son, Communication Systems, McGraw-Hill, New York tance-based absorption and reduced scattering optical prop (1988); and M. AA Neil, R. Juskaitis and T. Wilson, “Method erties at both excitation and emission wavelengths. In the of obtaining optical sectioning by using structured light in a presence of fluorophore absorption and fluorescent emission, conventional microscope. Opt Lett 22 (24), 1905-1907 Equation 2 becomes: (1997), illuminating a sinusoid pattern three times at the same spatial frequency, with phase offsets a-0, 2/3 L and % It 10 radians. M(X, f) can then be calculated algebraically at where X and m Suffixes denote optical properties at the each spatial location, X, by Mac(x.f.)-(I-1)+(I-Is) + excitation and emission wavelengths, respectively; and the (Is-I)'', where I, I, and Is represent the Limage values Source, q, is a product of the fluorescence quantum yield, m. at each location with shifted spatial phases. The spatially the excitation fluence rate, p, and fluorophore absorption varying DC amplitude, M(x), can be calculated at any 15 coefficient, Lla at the wavelength of excitation. The homoge frequency of illumination using M, (X, f) I+I+II/3. neous solution to this coupled equation resulting from a pla Finally, measurement of a reference turbid phantom of nar illumination is given by Wu et al Analytical model for known optical properties allows model-based calibration for extracting intrinsic fluorescence in a turbid medium. Appl the source intensity S, and therefore conversion of M, and Optics 32 (3585 (1993). M, to calibrated diffuse reflectance, R, and R, respec The general approach to fluorescence tomographic imag tively. In a similar algebraic fashion, the spatially-varying ing is performed by modeling the measured fluorescence as a phase can be measured, yielding topological data about the linear Superposition of contributions from Small perturbing 3D tissue surface as shown in the graph of FIG. 1c). objects. In the Born approximation, a small fluorescent (or FIG. 1d depicts the diffuse propagation of a normally absorbing) perturbation in the optical field, cp ef is given by: incident, periodically-varying plane wave source with spatial 25 frequency f. and spatial phase C, giving rise to a diffuse Pei(Phon+Pert fluence rate with the same frequency and phase. The behavior where p is given by Equation 3. In the first Born of these waves can be described by a 1-D second-order Helm approximation, the Green's function of a thin, absorbing pla holtz equation for the fluence rate as a function of depth, Z: nar perturbation is given in the spatial frequency domain by: 30
i2 f2 (2) la d2 po(z) - ui Po (z) = -34trigo(z) per (ktper kype 3) = 2Du.pert kep-tail: - 3 peril) + Halfpenze - 1 35 exp(-Harperl3+ sal): e whereu?-lu-(27t?)'', p. is the fluence rate, q is the Heif perize + 1 34tr Source, ,-(1-4) is the transport coefficient, L 3Lu'i, u, is the absorption coefficient, l'u (1-g) is the reduced scattering coefficient, and g is the cosine of the aver where kee, and kee, of lege, refer to the spatial fre age scattering angle. The Solution for the resulting diffuse quency content of the perturbation, D is the diffusion coeffi fluence rate of FIG. 1d is 40 cient, Z, is the object depth and Z, is the extrapolation distance. The scope of the invention contemplates advanced forward fluorescence radiative transport solvers and a tomographic po(3) = inversion algorithm in order to advance the modeling. Briefly, 45 the inverse problem can be stated in the spatial frequency domain in terms of a series of 1D, planar inverse problems. where P is the source intensity and C is determined by the Nonlinear image reconstruction of increasing order can be choice of a boundary condition. Using the partial current performed by an iterative series of linear reconstructions. boundary condition from convention R. C. Haskell et. al. Consider now laser speckle imaging (LSI). Noninvasive "Boundary conditions for the diffusion equation in radiative 50 blood flow imaging can provide critical information on the transfer. J. Opt Soc Am A Opt Image Sci Vis 11 (10), state of biological tissue and the efficacy of approaches to treat disease. Laser Doppler flow metry and laser Doppler 27272741 (1994), the diffuse reflectance, R, is given by: imaging have previously been applied in numerous preclini cal and clinical studies on the brain, retina, skin, and joints. A 55 primary limitation of these methods is the need for mechani 3Au f Hir (4) cal scanning of the probe laser beam, resulting in long (on the order of minutes) image collection times. A method for high spatial and temporal resolution imaging of blood flow where A is a proportionality constant from boundary con dynamics is required to provide objective evaluation of exter ditions at the air-tissue boundary, and LL, is a function of both 60 nal stimuli. Such as pharmacological intervention, electrical optical properties and spatial frequency of illumination. stimulation, or laser irradiation. Consider now modulated fluorescence imaging. In a fluo In 1981, it was proposed to use a laser speckle imaging rescent medium, the photon fluence rate generated from a (LSI) approach as an alternative to laser Doppler imaging. sinusoidal source (Eq. 3) will produce a resulting sinusoidal This method employs quantitative, spatially resolved analysis fluorescent emission. Therefore, measurement of spatial fre 65 of the speckle pattern that is observed within images of laser quency domain fluorescence amplitude is performed in the irradiated objects. The speckle phenomenon is due to EM same fashion as that for reflectance, with the modification of wave interference effects that result essentially in both spatial US 8,509,879 B2 13 14 and temporal modulation of the imaged reflectance pattern. fluorescence, and blood flow. The technology provides On the basis of this study, it was concluded that variations in researchers with a quantitative tool to study disease progres speckle contrast can be used to provide directly a wide field sion and therapeutic response with 1) a high degree offidelity Velocity distribution map. With laser Doppler imaging, tem and localization, and 2) Sufficient spatiotemporal resolution poral intensity fluctuations of each speckle (or a collection of 5 and probe Volume to study events on length scales that have speckles) is monitored at high sampling frequencies (on the broad relevance (i.e. mm-cm orders of size). With the illus order of MHz). An increase in fluctuation frequency is asso trated WiFI instruments, the ambiguity that exists in planar ciated with faster blood flow. In contrast, LSI relies on acqui imaging modalities (between molecular reporter depth and sition and analysis of a single image captured at an exposure signal strength) are overcome, resulting in absolute measure time that is considerably longer than a characteristic correla 10 ments of signal and more accurate comparisons of multiple tion time associated with the fluctuation frequency. A faster experimental conditions. The knowledge of both local meta blood flow appears more blurred in the captured image than bolic activity and molecular reporter dynamics results in an regions of slower or no flow. The degree of blurring is quan improved understanding of cell-vascular coupling phenom tified as the local speckle contrast value (see Equation 8 ena. With absolute quantification of local oxygen Saturation below), with Zero contrast representing no speckle and hence 15 and blood flow, researchers will be able to draw comparisons high blood flow, and unity contrast representing a fully devel among data collected in serial measurement sessions on a oped speckle pattern and hence no flow. single patient and among patients measured at different sites Based on laser speckle statistics, the following relationship worldwide. between the speckle contrast (K) and the normalized autocor Furthermore, with absolute quantification of tissue param relation function of the remitted light was previously derived: eters, we envision the possibility of WiFI-based epidemio logic studies to facilitate development of physiologically (8) meaningful quantitative metrics of tissue function (i.e., "nor where O is the variance, Summary of Phantom Validation Studies Multi-layer phantoms Perturbation Phantoms General Phantom Design tal, us' d Key Phantome Features Vary optical properties in each Fluorescent absorption layer perturbations at depth Vary layer thickness (d) Tube perturbations with flow Bead perturbations Perturbation separation Primary Models Multi-layer SDA Fluorescence Radiative Developed Multi-layer MC Transport (FRT) and Validated Multi-layer ö-P Tomography LSIMI flow validation Collaborations VP: VTS VP: Small Animal ATK (FRT) WP: Small Animal ATK Dr. John Frangioni (FRT) Dr. Stuart Nelson Dr. James Tunnell (tomography) Dr. David Boas (LSI/MI) Applications Quantitative Planar Imaging Small animal tomography Brain (skull/brain) CMRO validation Tumor (skin tumor) Skin (melaniniskin) magnification of 0.8.x is required. For LSI, the appropriate Along with the in vitro phantom validation studies (Table f-stop setting would be 8, which is readily achievable with 1), there are vivo validation studies. We perform both sys standard macro lenses. As the magnification decreases (i.e., 65 temic and localized perturbations to the exposed microvas expanding the field of view to a 10 cmx10 cm field of view, as cular network of rodent window chambers. We have extensive in Instruments 2 and 3), the upper limit (M->0) on the f-stop experience with the dorsal window chamber in mice, ham US 8,509,879 B2 25 26 sters, and rats. With this model, we have unique access to both optical neuroimaging instrument (WiFI Instrument I). The the epidermal and subdermal sides of skin, with thicknesses multimodal neuroimaging instrument performs fast and ranging between about 300 um (mouse) to about 2 mm (rat). quantitative optical metabolic imaging of the brain. This sys Thus, we can characterize quantitatively both Superficial tis tem combines reflectance/fluorescence SI and LSI imaging Sue characteristics (i.e., direct imaging of the Subdermal techniques for the first time. In order to optimize speed, WiFI microvascular network) to assess the true tissue metabolic instrument 1 is a small field-of-view (1 cmx1 cm), dual wave state, and Subsurface tissue characteristics (i.e., imaging from length (LED or laser), dual frequency system targeted at the epidermal side) to evaluate WiFI accuracy in a more real-time (20fps) measurement, analysis, and visualization of realistic tissue geometry. To evaluate the ability of WiFI to dynamic neural signals such as stroke and epilepsy (FIG. 11 image large-scale metabolic changes, we use established 10 a). The system is based on a LCOS spatial light modulator methods: 1) vasoactive agents (acetylcholine for vasodila (FIG. 11 b) in order to permit laser projection with a motion tion, norepinephrine for vasoconstriction) administered via free system (preventing speckle dephasing in the instrument tail-vein or jugular-vein catheters, and 2) thermal interven itself). In order to achieve maximal acquisition rates, both tions (low-irradiance argon laser heating of the microvascu projection and detection arms are spectrally multiplexed with lature, circulating cold water at a set temperature). With the 15 a dichroic combiner and splitter, respectively. Dual-CCD therapeutic laser systems available at BU, we investigate the detection is provided by Dalsa Pantera 1M60 camera-link focal changes in tissue metabolism induced with pulsed laser devices. Both cameras and the LCOS developer's board is irradiation. In general, we expect to observe an increase (de synchronized at 60 frames per second via hardware trigger crease) in tissue metabolism with a decrease (increase) in ing, projecting 3 phase patterns at a rate of 20 Hz. Acquired blood flow. Furthermore, we expect to observe co-localiza data are 1) frequency-demodulated, and 2) calibrated, then 3) tion of blood flow and regions of high total hemoglobin processed into absorption and reduced scattering optical content, which is a key validation step demonstrating Suc property maps using an established rapid lookup table cessful fusion of SI and LSI. approach, then 4) processed into chromophore maps with Weanticipate that the primary bottleneck in practical appli linear spectral analysis of the multispectral absorption data. cation of WiFI instrumentation, especially in the clinic, will 25 Parallelized code operating on an eight-core workstation per be MI data acquisition time. LSI acquisition times are insig forms these four processing components simultaneously with nificant (about 100 ms) compared to SI acquisition times. In computational power to spare for user-GUI interaction and SI, we generally acquire up to thirteen spatial frequencies and visualization. The acquisition, control, processing and visu thirty wavelengths (about 6 minutes acquisition time) for our alization code for this instrument is based on the MI Inc. current MI Studies and reduce our dataset during postprocess 30 CH/C++ acquisition framework. The most demanding chal ing steps in order to extract the desired chromophores and to lenge is to implement the “measurement loop' for this instru obtain depth selectivity. The current imaging platform is ment (i.e. tight synchronization of hardware and analysis capable of acquiring reflectance maps at two spatial frequen components to achieve the proposed 20fps framerate) which cies and three phases at a rate of 1.5 S per wavelength. This is requires the development of hardware specific drivers as well the first work that quantitatively assesses the minimum num 35 as analysis code in C#. ber of required spatial frequencies and interrogated wave Consider now the Small animal tomographic imaging lengths for accurate chromophore extraction and depth selec instrument (WiFI Instrument 2). A small animal tomographic tivity in a clinical situation. This work sets the benchmark in imaging instrument (WiFI instrument 2) is fabricated for the minimizing acquisition schemes. primary purpose of tumorangiogenesis studies. A light-tight Processing time is also an obstacle to achieve our goal for 40 enclosure allows 3D measurement, analysis, and visualiza real-time visualization. We expect that hardware and software tion of endogenous and exogenous fluorescence, absorption advances focused on parallel processing schemes and tools, and scattering contrast (FIG. 12a). The system is based on a Such as graphics processor units (GPU), will shorten analysis custom-built, digital micromirror device (DMD)-based light time to achieve near real-time (about 20 frames pers) imaging engine (DVImage developer's kit, Apogen Inc.) for near rates. All the WiFI instruments include the hardware 45 infrared (NIR) structured light illumination (FIG. 12b). For resources to permit fast data analysis as needed and reduce the detection, this system incorporates a back illuminated, elec bottleneck for visualization to acquisition schemes. tron-multiplying, linear-gain CCD (QuantEM, Photometrics We expect to have an optimized, validated set of WiFI Inc.). Dual filter wheels will be placed in front of source and image processing algorithms to perform quantitative depth detector to allow a flexible combination of multispectral resolved characterization of absorption, scattering, fluores 50 reflectance and fluorescence measurements. Special care is cence, and blood flow along with a firm understanding of the taken with filter wheel alignment and stray light rejection in resolution and contrast limitations of our WiFI instrumenta order to maximize fluorescence contrast. The Apogen light tion. engine, QuantEMCCD camera, and both filter wheels are The WiFI instrument embodiments have 1) real-time (20 synchronized via the same underlying platform, developed frames per second) optical neuroimaging; 2) whole-body, 55 for WiFI Instrument 1. Small animal tomographic imaging; and 3) clinic-friendly A custom computed tomographic imaging spectrometer spectroscopic imaging. The integration of SI, LSI, and fluo (CTIS) is incorporated in the instrument in order to facilitate rescence imaging for instrumentation development based on hyperspectral tomography and simultaneous visualization of our research needs has the objective to design and fabricate multiple fluorophores. This instrument serves as a testbed three multimodal WiFI instruments designed to address three 60 platform for the development/visualization of tomographic specific application categories: 1) real-time optical neuroim algorithms and depth contrast information in the Small animal aging; 2) whole-body, Small animal tomographic imaging; ATK. The added challenge for this instrument is the 3D volu and 3) clinic-friendly spectroscopic imaging. This is an illus metric and cross-sectional visualization and measurement trative set of embodiments and is not intended to limit the tools. This is implemented using the existing MI Inc. render Scope of the invention. 65 ing engine based on Microsoft's Managed DirectX platform. We focus our instrument fabrication efforts to address spe FIGS. 12a and 12b are schematics of WiFI Instrument 2. cific preclinical and clinical needs. Consider the real-time FIG.12a is a block diagram of first generation Small animal US 8,509,879 B2 27 28 imaging and tomography instrument. FIG.12b is a rendering cellular processes in vivo Introduction. The objective is to of Apogen light-engine prototype demonstrating delivery of investigate disease progression through acute and chronic light via a light guide (I) onto a OMO chip (ii), which is then models of ischemic stroke, epilepsy (instrument 1) and tumor projected onto a target and then detected by a CCO camera angiogenesis (Instrument 2) while quantifying therapeutic (iii). response to neuroprotective agents and chemotherapies. Our Consider now the clinic-friendly spectroscopic instrument approach involves application of the instruments towards ani (WiFI Instrument 3). A fast, clinic-friendly imaging instru mal models. It is our expectation that these studies will estab ment (WiFI Instrument 3) is fabricated for therapy guidance lish WiFI instrumentation as an absolute quantitative tool of and wound healing monitoring. Instrument 3 is a clinic metabolic and molecular reporter activity in animal studies. friendly “snapshot hyperspectral (500-1000 nm) system, 10 The neuroimaging instrument (WiFI Instrument 1) can be capable of broadband spatial-frequency-domain imaging on a applied in Studies of epilepsy and ischemic stroke. In the Sub-second timescale. The device enables mapping of the epilepsy research, the objective is to study cellular Swelling spatial distributions of hemoglobin, lipid, water, and tissue associated with seizure events using WiFI generated maps of scattering in layered tissue systems. This lightweight system reduced scattering coefficient. Such an approach is expected is mounted on an articulating arm to allow arbitrary position 15 to have substantial advantages over the gold-standard method ing for a variety of clinical applications, including flap and of EEG recordings, which are invasive and limited in number diabetic wound monitoring, melanoma Studies, and port-wine of spatial samples. A fluorescence-based method is essen stain imaging. The key component of this system is a custom tially a point measurement at a discrete time point; thus, holographic computed tomographic imaging spectrometer interrogation of optical property dynamics with high spatial (CTIS), built by Jet Propulsion labs. In combination with the resolution is impractical to assess. We coregister optical requested 2Kx4K Dalsa 11 M04 camera, the 7-order filter will images with both the EEG and fluorescence-based recordings provide -5 nm spectral resolution of absorption and Scatter to assess the potential of WiFI to furnish data that are predic ing across the entire spectral range from 500-1000 nm, all tive of epilepsy. In the collaborative ischemic stroke research, with only three phase projection images. This allows Snap we attempt to understand both acute and chronic optical shot clinical measurements and multiple chromophore map 25 changes in neurovascular coupling after the onset of ischemic extraction before, during, and after therapies with minimal stroke. The Small animal tomographic imaging system (WiFI motion artifacts or discomfort to the patient. The projection Instrument 2) is utilized for studies of tumorangiogenesis and Subsystem is comprised of a separate DMD light engine from chemotherapy monitoring in Small animal models. In these Apogen geared toward lightweight construction (no filter studies, we determine the viability of using exogenous fluo wheels, magnesium exoskeleton, and fiber light guide tung 30 rescence contrast agents to track tumor growth and treatment. Sten source delivery), and designed for integration with the We also have the potential to learn more about the efficacy of CTIS/Dalsa imaging arm. The Apogen light engine and Dalsa certain chemotherapeutic agents as well as learn more about Pantera 11 M04 camera are synchronized via the same under contrast agent dynamics. lying platform developed for WiFI Instruments 1 and 2. This system is initially constructed with a liquid crystal tunable 35 TABLE 3 filter (LCTF), which can be replaced with the CTIS. A graph ics processor unit (GPU) is provided for acceleration of the Preclinical validation studies CTIS tomographic reconstruction code. While acquisition with the proposed system is <1 s, the CTIS reconstruction Instrument Clinical Problem Objective step is currently limited to >1 min/image for a total of 3 min 40 Real-time Optical Epilepsy Co-register brain computational time. While this delay in feedback is accept Neuroimaging electrical signals with (Instrument 1) optical scattering able for longitudinal studies of chronic disease progression dynamics and therapeutic response, it is incompatible with applications Ischemic Stroke Study long-term geared at informing a physician while monitoring an acute dynamics of cortical therapy, such as port-wine stain treatment, and resection of 45 function and metabolic activity in cancerous tissues in brain, melanoma, and breast cancer Sur response to ischemic geries. We utilize the programmable, massively data-parallel stroke nature of GPUs to solve the CTIS expectation-maximization Small Animal Tumor Angiogenesis Study dynamics of (EM) problem (a naturally-parallel algorithm). This is imple Tomographic optical biomarkers Imaging Instrument related to tumor mented using the CUDA programming model by nVidia, an 50 (Instrument 2) growth abstracted set offloating-point libraries aimed at general pur Chemotherapy Visualize changes in pose GPU computation (GPGPU). Preliminary reconstruc Monitoring optical contrast during tions of CTIS data have yielded long reconstruction times (1 chemotherapy minute per image). A work station with GPUs will reduce the treatinent reconstruction time of the CTIS to allow near real time (1 fps) 55 quantitative hyper-spectral imaging. We expect that valuable insight regarding the role of optical FIGS. 13a and 13b are schematics of WiFI Instrument 3. wide-field imaging in pre-clinical animal models for disease FIG.13a is a block diagram of clinic-friendly imaging instru progression and therapeutic monitoring will be realized. We ment. FIG.13b is an embodiment of a clinical device with the are in the position to impact the fields of: 1) neuroimaging, by light engine mounted on articulated arm and with real-time 60 studying neurovascular and metabolic physiology and devel visualization. opment of neuroprotective therapies for diseases such as epi Thus, there are three distinct versions of the WiFI instru lepsy and stroke, as well as 2) tumor biology, by studying the mentation platform (shown as FIG. 14, Table 2) provided as metabolic and angiogenic properties of cancer while moni examples. Each has features that focus on specific pre-clinical toring and developing chemotherapeutic strategies for treat and clinical applications. 65 ment. Consider the preclinical potential of the WiFI platform to The clinical potential of the WiFI platform as a noninvasive study essential quantitative hemodynamic, metabolic, and diagnostic and therapy monitoring tool is shown in a series of US 8,509,879 B2 29 30 in vivo clinical studies. The objectives are to determine the selective nano-photothermolysis of nanorods in Small animal efficacy of WiFI instrumentation as a quantitative therapeutic models are monitored with this system. monitoring and characterization tool in clinical scenarios. Finally, in addition to the preclinical neurobiology studies, Our approach involves implementation of our clinic-friendly we are already incorporating the SI component of WiFI into a instrument (WiFI Instrument 3) to conduct studies involving clinical neuroSurgical microscope. Our goal is to evaluate the both therapy guidance (port wine stain, neuroSurgery, skin capability of WiFI to perform quantitative intraoperative cancer), and characterization of wound healing (port wine brain mapping and provide information on Subsurface com stain, flap monitoring, diabetic ulcers) in pre-clinical and position. Applications include identifying eloquent neural clinical models. It is our expectation that these studies will tissue, localizing epileptic foci, and delineating tumor mar collectively justify the need for an integrated WiFi instrument 10 as a noninvasive tool for near real-time and quantitative feed gins. In addition, because WiFI is a non-contact method with back in clinical therapeutic treatment and diagnostics. quantitative near-real-time visualization, it can be coregis A summary of the clinical studies is presented below tered with intraoperative MRI to provide a complementary (Table 4). view of neuro-anatomic structure and function. 15 Insight gained regarding WiFI's potential as a medical TABLE 4 diagnostic imaging instrument is expected. Because of its rich metabolic information content, noncontact geometry, and Clinical Validation Studies wide interrogation field, we anticipate WiFI can be used in a broad range of clinical settings as a powerful method for Clinical Problem Objective therapeutic guidance. Skin Flap & Wound Quantify chronic Many alterations and modifications may be made by those Healing wound healing response having ordinary skill in the art without departing from the Port wine stain Surgical guidance for spirit and scope of the invention. Therefore, it must be under laser therapy of port stood that the illustrated embodiment has been set forth only wine stain birthmarks 25 for the purposes of example and that it should not be taken as Melanoma Early detection of melanoma limiting the invention as defined by the following invention Tumor resection Intra-operative tumor and its various embodiments. delineation using Therefore, it must be understood that the illustrated Exogenous Fluorophores embodiment has been set forth only for the purposes of 30 Cancer Treatment Imaging and example and that it should not be taken as limiting the inven treatment of cancer tion as defined by the following claims. For example, not using nanorods withstanding the fact that the elements of a claim are set forth NeuroSurgical Intra-operative brain below in a certain combination, it must be expressly under Guidance tumor delineation stood that the invention includes other combinations offewer, 35 more or different elements, which are disclosed in above even A skin flap wound healing project has already been estab when not initially claimed in Such combinations. A teaching lished in collaboration with MI Inc. and is facilitated by WiFI that two elements are combined in a claimed combination is Instrument 3. The central aim of this project is to employ further to be understood as also allowing for a claimed com WiFI to spatially resolve functional tissue characteristics in bination in which the two elements are not combined with an animal wound model. Our hypothesis is that WiFI instru 40 each other, but may be used alone or combined in other mentation can effectively work to provide quantitative assess combinations. The excision of any disclosed element of the ment of metabolic activity within ischemic chronic wounds of invention is explicitly contemplated as within the scope of the Superficial tissues. Although this is a pre-clinical model, this invention. study serves as validation of our instrument to perform Snap The words used in this specification to describe the inven shot spectroscopy in vivo. The knowledge regarding wound 45 tion and its various embodiments are to be understood not healing from this study is expected to lead into a clinical study only in the sense of their commonly defined meanings, but to of patients with diabetic ulcers. In addition, work in the field include by special definition in this specification structure, of port wine stains will drive the study of WiFI as a therapeu material or acts beyond the scope of the commonly defined tic guidance tool. The Surgery Laser Clinic affiliated with meanings. Thus if an element can be understood in the context BLI serves as a test-bed for clinical monitoring of port wine 50 of this specification as including more than one meaning, then patients before, during, and after therapeutic treatment. Our its use in a claim must be understood as being generic to all hypothesis is that this multi-modality SI instrument can serve possible meanings Supported by the specification and by the as an imaging platform for quantitative characterization of word itself. benign and malignant melanocytic skin lesions. A clinic The definitions of the words or elements of the following ready SI-based spectral imaging system for quantitative mea 55 claims are, therefore, defined in this specification to include Surements of cutaneous melanocytic lesions is provided. not only the combination of elements which are literally set Once this system is optimized, we acquire and analyze MI forth, but all equivalent structure, material or acts for per data from patients with benign pigmented lesions and those forming Substantially the same function in Substantially the with cutaneous melanoma who are scheduled to undergo same way to obtain Substantially the same result. In this sense treatment at the CHAO Family Comprehensive Cancer Cen 60 it is therefore contemplated that an equivalent substitution of ter Melanoma Clinic at UC-Irvine. Finally we compare vas two or more elements may be made for any one of the ele cular parameters obtained from WiFI data with those derived ments in the claims below or that a single element may be from histology. substituted for two or more elements in a claim. Although A combined fluorescence/SI instrument to provide a wide elements may be described above as acting in certain combi field image-guided Surgical tool for tumor margin delineation 65 nations and even initially claimed as such, it is to be expressly is being provided. Validation studies in Small animals are understood that one or more elements from a claimed com conducted to validate this instrument. Active targeting and bination can in Some cases be excised from the combination US 8,509,879 B2 31 32 and that the claimed combination may be directed to a sub rally coherent illumination imaging to simultaneously derive combination or variation of a Subcombination. static optical properties and dynamic motion using a spatially Insubstantial changes from the claimed Subject matter as structured coherent source comprises: viewed by a person with ordinary skill in the art, now known using the apparatus for performing both structured illumi or later devised, are expressly contemplated as being equiva nation imaging with laser speckle imaging to simulta lently within the scope of the claims. Therefore, obvious neously derive optical properties and speckle contrast at substitutions now or later known to one with ordinary skill in different spatial frequencies of the turbid medium by: the art are defined to be within the scope of the defined storing in a memory the three structured illumination elements. images at each spatial frequency; The claims are thus to be understood to include what is 10 specifically illustrated and described above, what is concep using the computer to calculate a mean intensity map, tionally equivalent, what can be obviously substituted and