© 2017 Nature America, Inc., part of Springer Nature. All rights reserved. [email protected] Institute of Technology, Cambridge, USA. Massachusetts, Renal Division, Brigham and Women’s Hospital, Harvard Medical School, Boston, USA. Massachusetts, USA. Massachusetts, Engineering, Institute Massachusetts of Technology, Cambridge, USA. Massachusetts, of Biochemistry of the Romanian Academy, Bucharest, Romania. Deaconess Medical Center, Harvard Medical School, Boston, USA. Massachusetts, 1 polymer, the to labels fluorescent couple to linker a of synthesis required of ExM version diffraction- original the Although microscopes. conventional limited on resolution nm) (~70 nanoscale with imaged be can they that so swelled, and homogenized mechanically to Samples are or network. labels the polymer fluorescent then ecules biomol key binds that tissue) a throughout evenly synthesized ylate polyacr sodium of mesh a (e.g., polymer swellable dense a in them embedding by tissues expand isotropically we ExM, strategy, this In samples by tissue physically, them than rather optically, magnifying context. research a in even samples, clinical to applied rarely are and practice clinical super- Accordingly, in utility routine found tissues. not have nanoscopy and human imaging resolution large-scale to apply to dif ficult are and curve learning steep a present can hardware, complex (EM) microscopy electron as well as methods microscopy super-resolution pioneering using science in basic to explored be begun has organization This and tissues. cells throughout precision nanoscale with configured and dimension in nanoscale however, are themselves, Biomolecules states. disease and predisease of variety wide a of pathogenesis the investigate or nose microscopy composition to molecular using diag diffraction-limited Physicians and researchers have long examined cellular structures and ExPath may enable the routine use of nanoscale imaging in pathology and clinical research. between discrimination computational early breast neoplastic lesions for which pathologists often disagree in classification. of kidney disease, minimal-change a process that previously required electron microscopy, and we demonstrate high-fidelity microscopes and standard antibody and fluorescent DNA ExPath enables imaging ~70-nm-resolution of diverse biomolecules in intact tissues using conventional diffraction-limited then applies an ExM protocol with protein anchoring and mechanical steps homogenization optimized for clinical samples. fresh frozen. The method, which we call expansion pathology (ExPath), converts clinical samples into an state, ExM-compatible of human tissue specimens that have been fixed with formalin, embedded in paraffin, stained with hematoxylin and eosin, and/or not been applied to clinical tissue samples. Here we report a clinically optimized form of ExM that supports nanoscale imaging Expansion microscopy (ExM), a method for improving the resolution of light microscopy by physically expanding a specimen, has Andrew H Beck Eun-Young Oh Yongxin Zhao expansion microscopy pathology-optimized Nanoscale imaging of clinical specimens using nature biotechnology nature Received 29 June 2016; accepted 28 April 2017; published online 17 July 2017; MIT MIT Media Lab, Institute Massachusetts of Technology, Cambridge, USA. Massachusetts, Recently, we developed a strategy for imaging large-scale cell and and cell large-scale imaging for strategy a developed we Recently, , , pathological aspects) or E.S.B. ( 1 8 , 2 11 Department Department of Pathology, Brigham and Women 2 , , Marcello DiStasio , , Octavian , Bucur Octavian 3 , 5 & Edward S Boyden advance online publication online advance 5– 7 ; but such methods require require methods such but ; 2 2 – [email protected] , , Vanda Torous 5 , 11 , , Humayun Irshad 11 1 These These authors corresponded equally to this work. should Correspondence be addressed to A.H.B. ( , 6 , 7 , 10 5 Broad Broad Institute of MIT and Harvard, Cambridge, USA. Massachusetts, ′ s s Hospital, Harvard Medical School, Boston, USA. Massachusetts, , , technical aspects).
in in situ 2 , , Benjamin Glass 1– 8 4 - - - - .
hybridization reagents. hybridization We use ExPath for optical diagnosis 3 d Ludwig Ludwig Center at Harvard Medical School, Boston, USA. Massachusetts, 2 oi:10.1038/nbt.389 hardware. We also expect the method will be useful for providing providing for insights useful into the pathogenesis of various human be diseases. will method the expect also We hardware. power available to pathologists without requiring investment in novel protein and cell-morphology levels. ExPath will enhance the diagnostic broad utility in enabling probing of nanoscale features at the genomic, diagnose subtypes of these lesions. We anticipate that ExPath will have ExPath facilitates computational pathology differentiation of hard-to- which pathologists often disagree in classification for lesions neoplastic breast early of atypia nuclear analyze to ExPath tional diffraction-limited light microscopy. As another example,can now we be directlyused and accurately diagnosed with ExPath and conven (MCD) disease change study,In a small-scale we show (such minimal that as kidney diseases variety of tissue samples from different human organs and diseaseslides. states.We explore ExPath’s ability to enable nanoscaleeosin (H&E)-stainedimaging andon fresh-frozena wide human tissue specimens on glass including formalin-fixed paraffin-embedded (FFPE), hematoxylin and can process most types of clinical samples currently used in pathology, polymer swellable the to directly to antibodies) fluorophore-bearing as molecule (such proteins tie anchoring available commercially a uses that ExM of version a (proExM), ExM retention protein developed recently we , 7 3 McGovern McGovern Institute, Institute Massachusetts of Technology, Cambridge, , Here, we report a clinically optimized form of proExM, ExPath, which 5 , , Fei Chen 2 Department Department of Pathology and Cancer Research Institute, Beth Israel 2 , , Isaac E Stillman 10 1 2 Department Department of Brain and Cognitive Sciences, Massachusetts , 9 6 . , 7 , , Astrid Weins 1 0 ) that previously required EM for diagnosis diagnosis for EM required previously that ) 2 , , Stuart J Schnitt 8 , 9 , , Andreea L Stancu 6 9 Department Department of Biological Department Department of Medicine, 1 1 , , and we show that ticles e l c i rt A 2 , 4 Andy. Institute Institute 2 , 5 - ,
© 2017 Nature America, Inc., part of Springer Nature. All rights reserved. factor, 4.1); ( 4.1); factor, in as stained ( HER2. against FISH DNA magenta, centrosome; without sample cancer breast a human of in sample the ( right. at square small the by framed area the of view a magnified is left) (upper Inset (ADH). hyperplasia ( 0.2)). (s.d. 4.0 factor, expansion average patients; different four mean; of error standard area, shaded channel; KRT19 of mean line, solid magenta channel; DAPI of mean line, ( microscope. ( (KRT19). keratin-19 magenta, vimentin; green, DAPI; Blue, tissue. ( 0.3)). (s.d. 4.3 factor, expansion average patients; mean; of error standard area, shaded channel; vimentin of mean line, solid green channel; DAPI of mean line, solid (blue of ( (VDAC). channel anion voltage-dependent magenta, vimentin; green, 1 Fig. 1 Figure and stained H&E states—FFPE, tissue starting three Weconsidered ( processing ExM for optimized to a state samples clinical to convert of steps a series We devised first microscopy expansion pathology-optimized and samples Clinical RESULTS s e l c i rt A 23 postexpansion,
• • • • Clinical samples:
b j f b a Fresh frozen Unmounted, H&E Mounted, H&E Paraffin embedded . . ( ). ( ). d , e
b ) Root mean square (RMS) length measurement error as a function of measurement length for pre-expansion versus postexpansion images images postexpansion versus pre-expansion for length measurement of a function as error measurement length (RMS) square mean ) Root Design and validation of expansion pathology (ExPath) chemical processing. ( processing. chemical (ExPath) pathology expansion of validation and Design ) Pre-expansion image of a 1.5-mm core of normal human breast tissue acquired with a wide-field epifluorescent microscope. Blue, DAPI; DAPI; Blue, microscope. epifluorescent a wide-field with acquired tissue breast human normal of core a 1.5-mm of image ) Pre-expansion l f h . Scale bars (yellow scale bars indicate postexpansion images): ( images): postexpansion indicate bars scale (yellow bars . Scale ) 10 ) 10 j , stained with antibodies against Hsp60 (magenta) and vimentin (green) and with DAPI (blue). ( (blue). DAPI with and (green) vimentin and (magenta) Hsp60 against antibodies with stained i ) RMS length measurement error as a function of measurement length for ExPath versus SIM images of human breast tissue (blue solid solid (blue tissue breast human of images SIM versus ExPath for length measurement of a function as error measurement length ) RMS conversion µ µ Format m; inset, 4.6 4.6 inset, m; m; ( m; g ) 10 ) 10 µ m (physical size postexpansion, 43 43 postexpansion, size m (physical µ g c m; expansion factor, 4.6); ( 4.6); factor, expansion m; Fig. Fig. k 1 HER2 and and 2. Anchoring(AcX) 1. Staining amplification. Blue, anti-HER2 (not visible); gray, DAPI; green, DNA FISH against chromosome 17 chromosome against FISH DNA green, gray, DAPI; visible); (not anti-HER2 Blue, amplification. Supplementary Fig. 1 Fig. Supplementary m ) ExPath wide-field fluorescence image of a human breast cancer sample with with sample cancer breast a human of image fluorescence wide-field ) ExPath f ) Super-resolution structured illumination microscopy (SR-SIM) image of normal human breast breast human normal of image (SR-SIM) microscopy illumination structured ) Super-resolution l ) and ( ) and j ) Hematoxylin and eosin (H&E)-stained human breast sample with atypical ductal ductal atypical with sample breast human (H&E)-stained eosin and ) Hematoxylin µ g d h m, expansion factor, 4.3); ( 4.3); factor, expansion m, ) ExPath image of the sample in in sample the of image ) ExPath m c RMS error RMS error ) Postexpansion (i.e., ExPath) wide-field fluorescence image of the sample sample the of image fluorescence wide-field ExPath) (i.e., ) Postexpansion ), physical size postexpansion 20 20 postexpansion size physical ), of measurement (µm) of measurement (µm) ). ). polymerizatio In situ 0.0 0.1 0.2 0.3 0.4
10 15 20 2 h,37°C 0 5 l 8 6 4 2 0 0 We evaluated whether xylene treatment to remove paraffin, followed followed to remove treatment paraffin, Wexylene whether evaluated or of sets permutations the for steps required processing. FFPE tissue sub be would categories other the in tissues for converting required retrieval) antigen and rehydration treatment, xylene (e.g., steps the that Wewe since slide. hypothesized samples, FFPE investigated glass first a on and sliced thin be to tissue the assumed we frozen; fresh b ) 200 ) 200 Measurement length Measurement length n 500 µ advance online publication online advance a m; ( m; ) Schematic of ExPath workflow (details in in (details workflow ExPath of ) Schematic c ) 220 ) 220 1,000 j ) ) 5 ( ( µ µ µ m) m (physical size postexpansion, 900 900 postexpansion, size m (physical m) 25 mMEDTA Digestion (ProK, µ m, inset 1 inset m, 3 h,60°C f 1,500 acquired with a spinning disk confocal confocal disk a spinning with acquired 10 k µ ) ExPath wide-field fluorescence image of image fluorescence wide-field ) ExPath m.
) e i l ) ExPath wide-field fluorescence image image fluorescence wide-field ) ExPath RMS error n
µ RMS error = 5 fields of view from samples from from samples from view of = 5 fields
m; ( m; of measurement (µm)
m of measurement (µm) 0.0 0.1 0.2 0.3 0.4 10 15 20 0 5 k ) ) 5 6 4 2 0 0
n µ = 3 samples from different different from = 3 samples Measurement length Measurement length nature biotechnology nature 1 h,RT m, inset 1 inset m, H 2 O 500 HER2 Supplementary Supplementary µ 1,000 amplification, amplification, m (physical size size m (physical µ m; expansion expansion m; ( ( µ µ 1 8 m) m) 1,500
0 - © 2017 Nature America, Inc., part of Springer Nature. All rights reserved. tured-illumination tured-illumination (SR-SIM) microscopy pre-expansion and confocal struc super-resolution using cells on protocol cultured by this using 2a Fig. Supplementary ( rescence protocol, we obtained excellent isotropic expansion with low autofluoproExM original the in EDTAused mM 1 the EDTAversus mM 25 tocol, even after paraffin removal; but if digestion was performed with pro proExM the under evenly not expand did and liver) skin nodes, expansion. enables water of addition and antibodies) applied e.g., interest, of proteins the spares that extent an (to digestion K teinase linked to the polymer network; then, polymerization followed by pro be to proteins enables modification this and group, functional mide with an on amines acryla biomolecules modify is to used chemically AcX) abbreviated here SE; (Acryloyl-X, acid ryloyl)amino)hexanoic protocol proExM the 1 Fig. min; Supplementary 30 for incubator °C 60 a into samples the transferring diately imme then °C, 100 and 8 pH at citrate sodium mM 20 in samples (placing step antigen-retrieval standard fairly a and rehydration by 208 postexpansion, ( images): postexpansion indicate bars scale (yellow bars Scale mean. the indicates symbol square the median; the indicates rectangle the in segment the respectively; minimum, and maximum the by defined are box the of boundaries lower and upper the s.d.; the by ** 0.1). (s.d. 4.1 factor, expansion Average patients. three from tissue ( postexpansion ( pre-expansion showing (magenta), KRT19 and (green) vimentin against antibodies and (blue) DAPI with labeled tissue breast human normal ( pre-expansion showing (magenta), KRT19 and (green), vimentin ACTA2 (blue), against 2 Figure nature biotechnology nature microscopy postexpansion ( k n f a We found that heavily formalin-fixed human tissues (e.g., lymph lymph (e.g., tissues human formalin-fixed heavily that Wefound
ExPath reduction of tissue autofluorescence. ( autofluorescence. tissue of reduction ExPath Supplementary Note; Supplementary Tables Note; 1 Supplementary Supplementary n – p ) data. ( ) data. µ m; expansion factor 4.6); ( 4.6); factor expansion m; 9 . In proExM, the succinimidyl ester of 6-((Ac of ester succinimidyl the proExM, In . ), could sufficiently prepare FFPE samples for samples FFPE prepare sufficiently could ), – j q ). We validated the low distortion obtained obtained ). We low the distortion validated ) Signal-to-background ratio for pre-expansion (magenta) as well as postexpansion (green) states of of states (green) postexpansion as well as (magenta) pre-expansion for ratio ) Signal-to-background
Supplementary Fig. 3 advance online publication online advance b g l o k – m ) ) 5 ). We next validated a µ – m; ( m; j ) Wide-field images of normal human lung tissue labeled with DAPI (gray) and antibodies antibodies and (gray) DAPI with labeled tissue lung human normal of images ) Wide-field n h and c – p ) ) 5
2 µ ; ; and m (physical size postexpansion, 18 18 postexpansion, size m (physical P < 0.01; * < 0.01; ------m p ized nuclear DNA in postexpansion H&E-stained samples by DAPI by samples H&E-stained postexpansion in DNA nuclear ized ( processing ExPath by removed stain were stains hematoxylin and We eosin ing. antibody both that found fluorescent for important be would removal H&E ( fluorescence ground used we ExM, ( for medium mounting the dissolve and pretreatment coverslip the a remove to xylene as acceptable was treatment xylene that established had we since medium; mounting and erslip cov the remove to had we samples, mounted For protocol. proExM samples. aldehyde-fixed highly paraffin-embedded, expanded levels obtained by earlier ExM protocols ( microns of hundreds to low of levels yielded distortion a of few percent tens over lengthscales ( wide-field a SR-SIM ( found that pre-expansion imaging with either a wide-field ( We preservation. FFPE with prepared tissues breast human normal could prepare samples for proExM by the entire assessing onpipeline EDTA, increased and treatment xylene with pipeline, FFPE this that Supplementary Fig. 1 Fig. Supplementary We samples for next to sought our prepare enhanced H&E-stained P < 0.1, two-tailed paired paired two-tailed < 0.1, a Fig. 1 Fig. – e ) and postexpansion ( postexpansion ) and Fig. 1 Fig. i d f ) microscope followed by postexpansion imaging on imaging by postexpansion followed ) microscope c ) or confocal ( confocal or ) Supplementary Fig. 4 Fig. Supplementary ). H&E-stained tissues exhibited high back high exhibited tissues H&E-stained ). Supplementary Figs. 1 Figs. Supplementary Fig. 1d Fig. µ m; expansion factor 4.0). factor expansion m; t -test. The ends of whiskers are defined defined are whiskers of ends The -test. f , – e j e , ) data. ( ) data. Fig. 1 Fig. ) 45 ) 45 h q
, Signal-to-background ratio i ), similar to the low distortion distortion low the to similar ), 40 50 10 20 30 8 µ 0 , 9 g m; ( m; e . . Thus, this ExPath protocol j k ) microscope, respectively, respectively, microscope, ) – DAPI Pre-expansion Postexpansion p ** j ) 45 ) 45 ) Confocal images of images ) Confocal ), which suggested that that suggested which ), n = 3 samples of breast breast of = 3 samples and and Vimentin µ ticles e l c i rt A m (physical size size m (physical ** 4 ). We visual We ). KRT19 Fig. 1 * k – m ) and ) and b ) ) or - - - - © 2017 Nature America, Inc., part of Springer Nature. All rights reserved. progression and initiation of metastasis of initiation and progression transition vimentin, in the epithelial–mesenchymal and are which critical keratin filaments intermediate the of features sized sub-diffraction-limit- revealed ExPath tissue. human of types ferent dif across performance expansion consistent indicates which 10%, than smaller was above noted specimens for the variation expansion average expansion factor of 4.7 (s.d. 0.2; and ( ovary liver kidney, pancreas, colon, lung, prostate, breast, from tis sues cancer-containing and normal including organs, various from order of channels. magnitude in some spectral that contribute to autofluorescence, can reduce autofluorescence by an biomolecules (potentially including both proteins and small molecules) Thus, the molecular clearing of ExPath, which eliminates unanchored ( red to pathologist’s a channels ranging from in spectral UV inspection) visual by selected regions (from ratios signal-to-background compared we when specimens ExPath-processed images) fluorescence confocal lung ( from autofluorescence reduced substantially observed Wewater. to are >99% water and are thus transparent and refractive-index matched orescence reduction methods ofluorescence and FISH in pathology analysis, despite existing autoflu autofluorescence remains challengingtissue example, For for immunostaining. clinical conventional over applications advantages of immun that are digested by proteinase K treatment), this technique has several proteins nonantibody as (such molecules digested or unanchored of ( fication with expression protein HER2 of correlation the confirmed breast we and protein, the HER2 against antibody an costained with We samples protocol. the in earlier immunostaining with interfere not does it process, the of step final the in performed the in apparent was zation with cancers of no amplification with cancers breast ExPath of specimens breast for DNA into chromosomal with hybridized diffused and samples probes SureFISH that observed We bases ~150 of size average targeting an with oligonucleotides single- stranded of libraries are which probes, SureFISH available mercially com used we so samples, expanded to delivery efficient precludes of length (the targeting probes probes FISH BAC-based FISH (BAC)-based chromosome artificial bacterial traditional of size large The possible. was FISH DNA sion and FISH RNA with identity location their imaging accurately then and samples biological in other each from away RNAs expanding for method a developed assess fixatives. aldehyde with processed not tissues in amines free of number greater the of because perhaps ( samples acetone-fixed of expansion free from 0.1 mg/mL to AcX 0.03 mg/mL enabled moreof consistent and artifact- concentration the lowering that found we fixation; acetone with preserved sections fresh-frozen evaluated we Finally, hyperplasia (ADH). ductal atypical an with tissue using breast human vimentin of slide marker H&E stromal and Hsp60 protein chondrial ( staining s e l c i rt A nanoscale the of examination the be will ExPath for direction future
We applied ExPath to tissue microarrays containing specimens specimens containing microarrays tissue to ExPath applied We elimination in results and apart molecules spaces ExPath Because DNA fluorescent Fig. 2a ERBB2 Fig. Fig. Fig. 2 Fig. Fig. 1l Fig. HER2 Fig. 1j Fig. 3 – ); in all cases we obtained expansions of ~4–5×, with an with of ~4–5×, expansions we obtained cases all in ); q ( j HER2 , , wide-field fluorescence images) and breast ( , , HER2 and (as a control) the centrosome of chromosome 17. n , , m k = 3 normal breast samples from different patients). patients). different from samples breast normal 3 = ), and we applied antibody stains against the mito the against stains antibody we and ), applied ). in in situ mlfcto ( amplification ) ) gene amplification in breast cancer. We recently 1 hybridization (FISH) is hybridization commonly to used 2 ; here, we examined whether postexpan whether examined we here, ; HER2 14– 1 6 . Specimens processed with ExPath . processed Specimens -amplified case. As DNA FISH is is FISH DNA As case. -amplified HER2 i. 1 Fig. Supplementary Supplementary Table 3 HER2 1 Supplementary Fig. 2k Fig. Supplementary is approximately 220 kb) kb) 220 approximately is 8 m ( Fig. Fig. ; oe N hybridi DNA more ); ( Fig. 1 Fig. HER2 3 ). An interesting interesting An ). l ) ) and for breast gene ampli gene Fig. 2k 1 7 , cancer , cancer ). ). The – , 1 l p 3 ), ), ------, ,
convenient way to observe nanoscale morphology of both nucleic nucleic both of morphology nanoscale observe to way convenient and simple a provide will ExPath that Weanticipate cancer. of text architecture of and these other proteins in the and cellular tissue con Table 3 12.5 images, bottom 2.5–3.1 images, 10–12.5 images, top bars: scale (yellow postexpansion but column, middle the in shown are as view of fields same the shows images bar, 2.5 (scale box a by white image 10 bar, (scale view of field a is small which of top the images, two shows bar, 200 (scale microarray a tissue from a core shows images five the of left-most The shown. images tissue a given for images of block each Within etc.). breast, lung, prostate, (e.g., labeled as types tissue different show rows different bottom, to top From patients. human from tissues images) (right cancerous and images) (left normal both for types tissue various of 3 Figure
µ Colon Kidney Liver Ovary Pancreas Breast Lung Prostate m), and the bottom of which zooms into the area outlined in the top top the in outlined area the into zooms which of bottom the and m), for raw data). Blue, DAPI; green, vimentin; magenta, KRT19. magenta, vimentin; green, DAPI; Blue, data). raw for advance online publication online advance ExPath imaging of a wide range of human tissue types. Images Images types. tissue human of range a of wide imaging ExPath Core Normal (human) µ m. Physical size postexpansion: top images, 50 50 images, top postexpansion: size Physical m. µ m; expansion factors 4.0–5.0×; see see 4.0–5.0×; factors expansion m; Pr e µ m). The middle column within the five images images five the within column middle The m). Post µ m). The right column within the five five the within column right The m).
Core nature biotechnology nature x disease type, there are five five are there type, disease Cancer (human Pr Supplementary Supplementary e µ ) m; bottom bottom m; Pos µ m; m; t - © 2017 Nature America, Inc., part of Springer Nature. All rights reserved. which which is beyond the resolution of conventional optical microscopy effacement and appear continuous under EM—a phenomenon called foot process loops like interdigitating fingers, lose their characteristic morphology capillary glomerular of surface the cover normally which processes, nosed or confirmed via EM MCD and focal segmental glomerulosclerosis (FSGS) as are such typicallydiseases kidney diagnephrotic example, For necessary. already is resolution nanoscopic where scenarios some are However, there sis. mechanisms and classification and diagno logical for refined disease automated inspection of key features for both pinpointing novel patho or traditional by followed samples abnormal versus normal of ration Many potential uses of ExPath will likely be discovered by future explo processes foot tertiary podocyte human of visualization enables pathology Expansion organs. human of range wide a from samples biopsy clinical in biomarkers protein and acids nm. 200 inset, 4.2); factor, 4.3 postexpansion, size ( image): postexpansion in as normalized inset, the of cut line the along intensity actinin-4 Below, below. analyzed cut line the indicates inset the within line dotted box; white ( I.). (Norm. intensity maximum to normalized inset, the of cut line the along intensity feature micrograph electron Below, image. the below graph the in analyzed cut line the indicates inset the within line dotted box; black the by outlined region the into zoom kidney. Inset, human ( in analyzed cut line the indicates line IV; Orange gray, collagen DAPI. magenta, actinin-4; green, vimentin; Blue, microscope. confocal disk a spinning with acquired a glomerulus of part 4 Figure nature biotechnology nature ( processes foot cyte e c ) ExPath image of a clinical kidney biopsy sample from the same patient analyzed in in analyzed patient same the from sample biopsy kidney a clinical of image ) ExPath of lines dotted Red and orange the along intensity actinin-4 of ) Profiles a d
Here, we explored whether ExPath could enable imaging of podo imaging enable could ExPath whether Here, we explored Norm. I. 0.0 0.5 1.0 d 0.0 . . (
f ExPath analysis of kidney podocyte foot process effacement. ( effacement. process foot podocyte kidney of analysis ExPath ) As in in ) As 1 0.5 0 . The width of individual foot processes is around 200 nm, Position d , but for a patient with minimal change disease (MCD). ( (MCD). disease change minimal with a patient for , but 1.0 ( µ Fig. Fig. a µ m) ) ) 5 m; expansion factor, 4.3); inset, 200 nm; ( nm; 200 inset, 4.3); factor, expansion m; 1.5 µ 19 4
m; ( m; ). We identified both an anti-actinin-4 anti-actinin-4 an both identified We ). , 2 advance online publication online advance 0 . In MCD, kidney tertiary podocyte foot 2.0 b ) ) 5 Normal c µ . . ( m (physical size postexpansion, 23.5 23.5 postexpansion, size m (physical Norm. I. e 0.0 0.5 1.0 b ) ExPath image of the sample in in sample the of image ) ExPath 0.0 0.5 Position b 1.0 ( µ m) 1.5
f ) ) 1 a 2 2.0 1 ) Pre-expansion confocal image of a normal human kidney sample showing showing sample kidney human a normal of image confocal ) Pre-expansion µ . - - - - - m, inset, 200 nm; ( nm; 200 inset, m, g a a ) As in in ) As and and using the same microscope. Red line indicates the line cut analyzed in in analyzed cut line the indicates line Red microscope. same the using f µ from the same samples ( samples same the from ( cases ( cases normal from kidneys in processes foot of tertiary structure Weultrafine the FSGS. observed and MCD with patients from as well as kidneys normal with from individuals sections kidney of images ExPath fresh-frozen ( sion, foot revealing of ultrafine structures processes podocyte tertiary of in glomeruli normal human samples kidney ( basement capillary (a IV the microanatomy observed and membrane marker); we successfully collagen and marker) glomerular (a tin human stained Wekidney samples with antibodies anti-actinin-4, against vimen formalin. by caused antigenicity actinin-4 degraded of staining anti-actinin-4 for ( slightly decreased samples FFPE- of that preserved samples, kidney frozen acetone-fixed of quality ing ( acetone-fixed that samples were in before immunostaining heat kidney treated frozen processes foot podocyte tertiary label specifically anti-synaptopodin an and 22) (ref. Norm. I. Supplementary Fig. 7 Fig. Supplementary Supplementary Figs. 5 Figs. Supplementary Fig. 4b Fig. 1. 0. 0. m; expansion factor, 4.7); ( 4.7); factor, expansion m; 0 0 5 b 0. . . ( e 0 0 , but for the same patient as in in as patient same the for , but d Fig. 4 Fig. ) Electron micrograph of a clinical biopsy sample from a normal a normal from sample biopsy a clinical of micrograph ) Electron d , c , stained as in in as , stained ) not visible in confocal imaging ( imaging confocal in visible not ) .5 g Position ), consistent with the morphologies seen in EM images images EM in seen morphologies the with consistent ), g ) ) 1 1. 1 0 µ ( µ m (physical size postexpansion, 4.2 4.2 postexpansion, size m (physical m) Fig. 4 Fig. a .5 )—a difference that was presumably a result result a presumably was that difference )—a . Inset, zoom into the region outlined by the the by outlined region the into zoom . Inset, and and Fig. 4d Fig. d e ) ) 1 ) and foot process effacement in MCD MCD in effacement process foot and ) 2. 6 c 0 µ MCD ). Compared with the immunostain the with Compared ).
m; inset, 200 nm; ( nm; 200 inset, m; Normalized intensity 1.0 0.0 0.5 , f f g . Scale bars (yellow indicates a indicates (yellow bars . Scale
). Thus, with ExPath, nanoscale nanoscale ExPath, with Thus, ). Norm. I. 2 0. 0. 1. 0.0 3 0 5 0 antibody, each of which could could which of each antibody, 0. 0 0 .5 0.4 Fig. 4 Fig.
Position Fig. Fig. 4a Position e ticles e l c i rt A 1. ) ) 1 1 0 a ( ). We acquired acquired We ). µ µ , µ ( m) b µ m; expansion expansion m; 0.8 m (physical m (physical m) ) ) postexpan .5 2. 0 c 1.2 . - - -
© 2017 Nature America, Inc., part of Springer Nature. All rights reserved. of the current technology-oriented paper—will be required to deter required be paper—will technology-oriented of current the scope the beyond are these ExPath—although using studies blinded Large-scale diagnosis). for images EM multiple examine a normally from state abnormal or pathologists kidney practice, clinical (in image normal postexpansion single a in sample a the from whether was on image observers between evaluation consistent and rate 0.40) of threshold acceptable clinically the data (0.35 pre-expansion on poor was agreement interobserver whereas 0.68 value kappa with ment, agree images. substantial in were data postexpansion postexpansion of ratings versus Observers’ pre-expansion on cat ratings observers’ for egorical values kappa Fleiss’s calculated we agreement, t when 8%) ( (s.d. used were samples 90% ExPath to significantly increased (s.d. accuracy but 65.7% 17%), only was accuracy classification samples, unexpanded ( patient subjects, an from FSGS normal and one specimen patients MCD from from specimens two specimens three from glomeruli kidney of images immunofluorescence postexpansion ten and pre-expansion states image set in full (see postexpansion and pre-expansion both in glomeruli kidney of images immunofluorescence of nonpatholo set training a three studied and gists—first pathologists FSGS and observers—four MCD seven in cases, effacement process foot of identification rate microscopes. optical diffraction-limited with alized differences between clinical samples of nephrotic diseases can be visu 0.24. DCIS, versus ADH 0.01; DCIS, versus UDH 0.02; ADH, versus UDH 0.24; DCIS, versus * 0.3). (s.d. 4.8 factor, expansion Average (DCIS). images tissue cancer breast noninvasive 38 and ADH) 16 and UDH (15 images tissue breast benign 31 images, tissue breast normal 36 contained that images 105 of consisted sets data Both diseases. breast malignant preinvasive and benign normal, of differentiation computational for images DAPI postexpanded and H&E pre-expanded both on framework classification image this We applied cross-validation. in model classification the by achieved (AUC) curve operator receiver the under area the as measured was accuracy Classification classifier). GLMNET ( green). the around ‘halo’ a blue as expressed was this expert, the than outlines larger yielded segmentation automated the when that (note positives false nuclei, blue-filled negatives; false nuclei, red-filled positives; true nuclei, green-filled column, truth” ground vs. “automated the In visible). barely is outline the thus and high, too is resolution the because row ExPath the in visible not are which outlines, nucleus indicate circles (red respectively algorithm, segmentation automated the or expert the by segmented nuclei are nuclei green-filled columns, segmentation” “Automated and annotation” “Expert the For (ADH). hyperplasia ductal atypical of example samples; ( watershed. marker-controlled by splitting nuclei filter, (LoG) Gaussian of Laplacian multiscale by detection seed thresholding, error minimum by segmentation nucleus equalization, histogram by contrast enhancing correction, rolling-ball using removal noise right, to left From pipeline. segmentation nuclei and preprocessing image 5 Figure s e l c i rt A or confirmation diagnosis the streamline can ExPath whether mine -test; raw data in in data raw -test;
b a
To examine in a blinded study whether ExPath could enable accu enable could ExPath Towhether study in a blinded examine ExPath H&E ± A Expansion and 0.13, 95% confidence level; this value was borderline, given given borderline, was value this level; confidence 95% 0.13, cquir
ExPath improvement of computational diagnosis of early breast lesions. ( lesions. breast early of diagnosis computational of improvement ExPath imaging Supplementary Fig. 8 Fig. Supplementary ed image P < 0.05; bootstrapped paired paired bootstrapped < 0.05; R Supplementary Table 5 Supplementary emo Supplementary Fig. 8 Supplementary ve noise Exper (gr P b ound tr Enhance contrast ) Computational detection and segmentation of nuclei is significantly more accurate in expanded versus pre-expanded pre-expanded versus expanded in accurate more significantly is nuclei of segmentation and detection ) Computational t annotation = 0.0088, = 0.0088, Au ± and and 0.14 at the 95% confidence level; level; confidence 95% the at 0.14 t omat uth) ed nucleisegmentationframe Supplementary Table 4 Table Supplementary n = 7 individuals, two-tailed two-tailed = 7 individuals, Au ). To assess interobserver interobserver To). assess Thr ); then they examined ten ); examined they then t 2 t omat -test. -test. 4 esholding . ExPath enabled accu enabled ExPath . ed segmentation (as in P value for each binary comparison: normal versus UDH, 0.17; normal versus ADH, 0.34; normal normal 0.34; ADH, versus normal 0.17; UDH, versus normal comparison: binary each for value a ) Det ect seeds wo ( rk Au tr ue positiv t ). For For ). omat Wa fa c ed vs ) Classification models were built using L1-regularized logistic regression (the (the regression logistic L1-regularized using built were models ) Classification lse positiv te ------e rs ; he
f . alse neg ground tr d ( images DAPI-stained for postexpansion optimized algorithms tation classification framework updated with nucleus detection and segmen work lesions. of breast preinvasive improveand thus classification the made information accessible by extra ExPath could the lead to a higher that quality of extracted anticipated features we resolution, image increases substantially ExPath Because images. diffraction-limited of efficacy these models is limited by the information extractable from from malignant intraductal proliferative breast lesions benign discriminate can that models pathology computational oped devel previously we issue, first the To addressing start microscopes. images is limited by the optical diffraction limit of the conventionalin contained optical information the second, subjective; and qualitative schemes are due to two issues—first, the diagnostic criteria are largely management clinical guiding for and undertreatment and overtreatment preventing for that is information critical cation of provides diagnostic lesions these gists for nuclear atypia diagnosis in early breast lesions study has shown that there is only patholo ~50% agreement between pathology breast in areas problem challenging of early breast represents classification which one lesions, of the most To explore further the utility of ExPath, we examined the pathological lesions breast early in diagnosis computational improves pathology Expansion pathology. nanoscale known involve that diseases other and disease kidney nephrotic of Fig. 5 Fig. e) We applied our previously developed image classification frame classification image developed previously our Weapplied classification current the with problems the that Wehypothesized a ) Automated nucleus segmentation framework, showing steps of the the of steps showing framework, segmentation nucleus ) Automated 2 ativ 7 uth a to ordinary H&E-stained samples, and we applied an image image an applied we and samples, H&E-stained ordinary to advance online publication online advance e ) to expanded samples. Our image classification framework framework classification image Our samples. expanded to ) ; c Accuracy (AUC)
Nor 0.00 0.5 0.6 0.7 0.8 0.9 1.
mal vs 0 25
, . 2 UDH 6 . Nor P Pr ost mal vs e-e e xpansion xpansion . ADH Nor mal vs
. DCIS
* *
nature biotechnology nature UDH vs
. ADH 1
1 UDH vs . For example, one one example, For .
2 . 1 7 DCIS 1 . However, the . . The classifi ADH vs
. DCIS - - - - - © 2017 Nature America, Inc., part of Springer Nature. All rights reserved. ExPath enhances researchers’ ability to detect and segment nuclei, nuclei, segment and detect to ability researchers’ enhances ExPath that We found classification. sample inform to content information greater providing by diagnostics improve to ExPath of potential the We highlight podocytopathies. of pathology renal diagnostic in EM for need the eliminate would ExM whether unclear it is studies scale situ enables multiplexed localization and identification of biomolecules for required those EM ( skills than the demanding are less ExPath to and perform required and equipment EM, for that than shorter time significantly processing is the ExPath for ExPath, of that to resolution superior EM far Although has MCD). kidney (e.g., pathology nanoscale known exhibit diseases to of has diagnosis it the that in and application types, clinical tissue immediate of diversity wide a on well functions ExPath that Wefound hardware. imaging common on samples cal of clini imaging nanoscale thus enables ExPath tissues. fresh-frozen protocol proExM our of extension an is ExPath information. molecular and precision nanoscale with samples biopsy of clinical interrogation cal opti for method versatile and simple a ExPath, describe Weherein DISCUSSION models. classification more accurate turn, in and, tures segmentation fea informative more nuclear in results images improved postexpansion on the achieved that suggest findings These ( results the affect greatly not did zation beforefactors running our digital pathology expansion pipeline; however, their such normali by normalized were images postexpansion TablesSupplementary 10 in shown are models classification by these selected features nificant ples compared with only 0.71 on pre-expanded samples. The most sig such as lesions ADH an with atypical AUCfrom sam of on 0.93 expanded UDH as such lesions discriminate could pipeline our 0.5), of would classifier perfect (a an AUC achieve would an AUC classifier a random and of 1, achieve positives false versus positives true of When we examined the area under the receiver operator (AUC)curve models over pre-expansion classification data ( diagnosis of performance the improved expansion that substantially found we end, this To analyses. pathology tational compu improved support principle in could segmentation and tion ( Cohen’s(GCE) error consistency in global kappa and a 66% decrease in increase 77% a F-score, in increase 14% a with well, as improved ( and samples nonexpanded over F-score in increase 16% a and value, predictive positive in increase 22% a rate, ( samples in expanded accurate more significantly was nuclei of detection Computational 9 9 and ADH 7 UDH, DCIS; a (6 normal, of 31 images subset for segmentation and detection nuclear on ExPath of impact the (DCIS). The average expansion factorcarcinoma wasductal 38 and 4.8 (ADH)) (s.d. 0.3).hyperplasia Weductal atypical first assessed 16 and (UDH) hyperplasia, ductal usual (15 images (benign) lesion sisted of 105 normal breast images—36 tissue images, 31 proliferative features. texture nuclear and nuclear features intensity features, morphology images—nuclear postexpanded and pre-expanded the both from nucleus segmented each from features ( of kinds three segmentation we extracted framework, of this application Following nuclear and detection seed nucleus tion, detec foreground included images DAPI-stained postexpansion for nature biotechnology nature Supplementary Table 8 Supplementary Each of the two data sets (pre-expansion and postexpansion) con and postexpansion) (pre-expansion sets of data two the Each , , both of which are challenging with EM. Of course, without larger 7 ; ; Supplementary Fig. 9 Fig. Supplementary 9 optimized for clinical samples—FFPE, H&E-stained and and H&E-stained samples—FFPE, clinical for optimized Fig. 5 Fig. Supplementary Table Supplementary 13 ). This improved accuracy of nuclear detec of nuclear accuracy improved ). This and
b advance online publication online advance ), with an 11% increase in true-positive in true-positive ), an with 11% increase ); and segmentation was significantly significantly was segmentation and ); 11 Fig. 5 . . Features from extracted individual c and Supplementary Table 12 Table Supplementary upeetr Tbe 6 Tables Supplementary Supplementary Table 9 ). ). Moreover, ExPath i. 5 Fig. Fig. 5 Fig.
in situ in b a in ). ). ). ). ). ). ). ------
observed between individual pathologists individual between observed discordance significant with pathology diagnostic in area difficult a represents diseases breast preinvasive of classification accurate The types perform better on ExPath images than pheno on pre-expansion morphologic images. nuclear on focus that models classification sis on ExPath performance excellent Weimages. found also that diagno use fast diffraction-limited optics enables the voxel sizes of a super- a of sizes voxel the enables optics diffraction-limited to fast use ability the imaged, be to voxels more requires thus be and to imaged, volume the expands it Although dimension. each in fication choosing. our of environments pure with them surrounds and cal analysis of biomolecules but may also help of microscopes, resolution chemi support existing move biomolecules and labels apart evenly not only helps improve the decades back goes imaging to support in such as hydrogels polyacrylamide specimens biological multiplexed imaging of RNA in developments ongoing with ExPath combine to clinicians and ers In probes). available cially the future, it be will of interest to research DNA(e.g., incorporating FISH into the pipeline, using only commer samples clinical of investigation multimodal for used be can it also but samples, of variety wide a address ExPath can only not that here demonstrate we versatility; its is ExPath of advantage key Another technologies. such implement to groups of multiple ability the lights proExM, two other groups developed related protocols lesions. breast proliferative of analysis the for sifiers clas pathology of computational improves samples the performance tissue breast content of expanded information and that the increased moderate performance on moderate performance images standard histopathological that found We lesions. breast nuclear have which algorithms, segmentation historically shown only early and of morphology classification nuclear the explored analyzed we Here, specimens. ological variation. factor expansion sample-to-sample (<10%) small the image can be mapped onto biologically relevant units, and nulling out factor,of the expansion of size so that the the physical postexpansion pansion images taken at low magnifications enables simple calculation steps toward clinical adoption. Comparing pre-expansion and postex specimens expanded in RNA imaging for single-molecule demonstrated been as has helpful, be may labels gel-anchored of amplification reaction chain ization K proteinase after treatment, non-protein-reliantlost amplification methods such are as hybrid proteins Since imaging. before ampli fication signal implement to desirable be may it targets, abundance the expansion process dilutes that the concentration of fluorophores. is For low- classical ExPath of a property to Another comparison modality. direct super-resolution by validated fully been not have ExM of retention’ forms protein ‘full To ExPath. these of date, forms proteins to be retained most enable that ExM of forms protein-retention future, the In tion. interroga postexpansion prevents this and expansion, even enable to away digested are proteins most ExPath, of implementation rent samples expanded for shown previously been has (as microscopes on imaging light-sheet can fast thus volumetric support matched index to with a transparent that of refractive water, and they are samples ExPath of and proExM, ExM to those Similar and pathology. biology throughout used antibodies and stains of variety wide a with compatible is ExPath optics. diffraction-limited fast of rates acquisition voxel the at acquired be to modality imaging resolution In the current iteration, ExPath enables ~4.5× physical magni physical ~4.5× enables ExPath iteration, current the In of development our to parallel in robust; are protocols ExM ExPath may broadly enhance the computational analysis of path of analysis computational the enhance broadly may ExPath future important are ExPath of automation and Standardization 9 , 1 2 2 9 . may support more information-preserving 3 in situ 4 30– , the use of polyelectrolyte hydrogels to to hydrogels polyelectrolyte of use the , 3 2 and protein , , since ExM separates biomolecules 1 1 . . Accurate is classification 3 3 . Although embedding ticles e l c i rt A 1 28 2 ). In the cur the In ). , 2 9 ; ; this high 3 5 , , show ------
© 2017 Nature America, Inc., part of Springer Nature. All rights reserved. 3. 2. 1. claims jurisdictional in published maps and affiliations. institutional reprints/ R version of t The authors declare competing interests:financial details are available in the manuscript. A.H.B. and the E.S.B.project. supervised for the ExPath experiment. kidney All authors contributed to the writing of the breast lesions. A.W., M.D., V.T. and I.E.S. participated in the single blinded test framework. E.-Y.O., S.J.S. and B.G. performed the and selection annotation of the A.L.S. annotationperformed the expert (ground fortruth) the image analysis computational image analysis framework for the breast neoplasia benign analysis. analyzed ExPath data for breast neoplasia benign experiments. H.I. developed the and O.B. anddesigned acquired ExPath data for all tissues. H.I., O.B. and Y.Z. and analyzed data. F.C. and Y.Z. performed SR-SIM experiments on tissues. Y.Z. Y.Z., O.B., A.H.B. and E.S.B. all contributed key ideas, experiments designed Poitras Fellowship. the Lady Tata Memorial Trust, London. F.C. acknowledges the NSF Fellowship and Harvard and the Ludwig’s members J. Brugge and J. Staunton. O.B. acknowledges centers. A.H.B. and O.B. also acknowledge support from the Ludwig Center at contributions from Harvard University and its affiliated academic healthcare NationalSciences, Institutes of Health Award UL1 andTR001102), financial for Research Resources and the National Center for Advancing Translational Catalyst, the Harvard and Clinical Translational Center Science (National Center McGovern Institute MINT program. A.H.B. acknowledges support from Harvard 1R01EY023173; NIH NIH and1U01MH106011; 1R01MH110932; the MIT Award NIH 1R01GM104948; Director’s Pioneer Award 1DP1NS087724; NIH York Stem Foundation-Robertson Cell Investigator Award; NIH Transformative the W911NF1510548; MIT Brain and Department; the Cognitive Sciences New Laboratory and the US Army Research under Office contract/grant number project; the HHMI-Simons Faculty Scholars Program; the US Army Research For funding, E.S.B. acknowledges the MIT Media Lab; the PhilanthropyOpen online version of the pape Note: Any Supplementary Information and Source Data files are available in the t the in available are references, and codes accession statements of including Methods, and data availability any associated M management. clinical in role important an play will specimens these of classification pathological accurate and lesions, noninvasive small, contain will specimens pathology of tion and continue to colon) improve, prostate, esophagus a propor larger lung, skin, in malignancies (e.g., malignancies common for cedures pro screening cancer as general, In clinic. the in technology this of potential the understanding for critical be will sets sample larger on of diagnosis atypia or Further of validation malignancy). our findings a (for surgery to lesion) nonatypical benign, a (for observation from range can which treatment clinical determines it because important s e l c i rt A COMPETI AUTHOR CO A he pape he
eprints and permissions information is available online at at online available is information permissions and eprints c
ethods kn un, . Bbok H & hag X Bekn te ifato brir super- barrier: In nanoscopy. optical Far-field S.W. diffraction Hell, the Breaking X. Zhuang, & H. Babcock, B., Huang, Huang, B., Bates, M. & Zhuang, X. Super-resolution fluorescence microscopy. Photonics Society Photonics cells. of imaging resolution Biochem. Rev. owledgme index.htm r he pape N . G G FI N TRIBUTIO
N l 78 . . Publisher’s note: r . A (ed. Novak, D.) 3–4 (IEEE, 2010). (IEEE, 3–4 D.) Novak, (ed. , 993–1016 (2009). 993–1016 , N n CIAL CIAL I ts r . N N S Cell TERESTS
143 S pringer Nature remains neutral with neutral Nature remains regard to pringer , 1047–1058 (2010). 1047–1058 , 2010 23rd Annual Meeting of the IEEE the of Meeting Annual 23rd 2010 http://www. online version of version online nature.com/ online Annu. - - 16. 15. 14. 13. 12. 11. 10. 9. 8. 7. 6. 5. 4. 35. 34. 33. 32. 31. 30. 29. 28. 27. 26. 25. 24. 23. 22. 21. 20. 19. 18. 17.
Neumann, M. & Gabel, D. Simple method for reduction of autofluorescence in autofluorescence of reduction for method Simple D. Gabel, & M. Neumann, T.C.,Martins, M.S., Viegas, F.Seco, cost-effective and improved An A. Carmo, do & lipofuscin-like of Reduction M.W. Wessendorf, & W.A. Staines, S.A., Schnell, US patent probes. fish long of Synthesis M. Ruvolo, & S. Chen, E.M., Leproust, F.Chen, J.G. Elmore, Waldman, M. P.W.Tillberg, microscopy.F.,Expansion Tillberg,Chen, P.W.E.S. Boyden, & Gaietta, G. Nixon, R.A. of localization Immunocytochemical J.E. Vaughn, & C.R. Houser, P.E., Phelps, nanoscopy. fluorescence Toward S.W. Hell, rhd H, elad A, ox L & aoen, . ehd fr uli detection, nuclei for Methods D. Racoceanu, & L. Roux, A., Veillard, H., Irshad, thick of microscopy Confocal R.P. Thompson, & R.G. Gourdie, P.G., Germroth, R. Jungmann, RNAimaging. X. &Zhuang, S. Wang, J.R., Moffitt, A.N., Boettiger, K.H., Chen, Single-cell L. Cai, & M. Ahmad, T.,A.F.,Zhiyentayev, Coskun, E., Lubeck, and imaging super-resolution by biology systems Single-cell L. Cai, & E. Lubeck, T.Ku, T.J. Chozinski, F. Dong, K.H. Allison, Degnim, A.C. interpretation, use, studies: reliability in statistic kappa The C.C. Wright, & J. Sim, P. Mundel, S.V. Dandapani, in T.Hirose, syndrome nephrotic causing podocytopathies of Pathology S. Ranganathan, Pease, D.C. Fine structures of the kidney seen by electron microscopy. Maier, J., Traenkle, B. & Rothbauer, U. Real-time analysis of epithelial–mesenchymal Mendez, M.G., Kojima, S. & Goldman, R.D. Vimentin induces changes in cell shape, irsoi suy f hlnri nuos n synapses. and neurons cholinergic of study microscopic electron and light correlated a neostriatum: rat the within acetyltransferase choline (2003). fluorescence microscopy. fluorescence (2007). immunofluorescence tissues. paraffin-embedded formalin-fixed on direct studies in autofluorescence of reduction the for methodology (1999). tissue. 719–730 labeled fluorescently in autofluorescence (2014). A1 20140256575 13 specimens. biopsy outcomes. and (2016). 987–992 antibodies. and proteins fluorescent standard using labeled (2015). 548 Science electronmicroscopy study. (1985). 286–307 and future potential. future status and areview-current histopathology: digital in classification and segmentation, (1995). sections from acrylamide gel embedded embryos. Exchange-PAINT.and PAINT (2015). aaa6090 cells. single in profiling RNA multiplexed highly resolved, Spatially hybridization. sequential by profiling RNA labeling. combinatorial proteinlocalization size-adjustablein tissues. proteins. fluorescent breast. the of proliferations intraductal panel. review consensus (2014). expert an from learned adenosis. sclerosing with requirements. size sample and podocytes. renal and dendrites Chem. Biol. J. diaphragms. slit podocyte of maintenance the children. Cytochem. antibodies. single-domain fluorescent usingtransition 24 transition. mesenchymal to epithelial the during adhesion and motility, , 679–684 (2016). 679–684 , , 1838–1851 (2010). 1838–1851 , advance online publication online advance t al. et
296 et al. et Front Pediatr. Front t al. et et al. et
et al. 3 et al. Multiplexed scalable super-resolution and three-dimensional imaging of , 295–308 (1955). 295–308 , , 503–507 (2002). 503–507 , t al. et et al. et et al. et et al. et al. Nanoscale imaging of RNA with expansion microscopy.expansion with RNA of imaging Nanoscale
t al. et t al. et 282 opttoa ptooy o iciiae ein rm malignant from benign discriminate to pathology Computational An essential role of the universal polarity protein, aPKClambda, on aPKClambda, protein, polarity universal the of role essential An Clin. J. Am. Soc. Nephrol. Soc. Am. J. Clin. Multicolor and electron microscopic imaging of connexin trafficking. Extensive involvement of autophagy in Alzheimer disease: an immuno- t al. et t al. et Adult minimal-change disease: clinical characteristics, treatment, Understanding diagnostic variability in breast pathology: lessons pathology: breast in variability diagnostic Understanding Gene signature model for breast cancer risk prediction for women Diagnostic concordance among pathologists interpreting breast interpreting pathologists among concordance Diagnostic yatpdn a atnascae poen n telencephalic in protein actin-associated an Synaptopodin: J. Am. Med. Assoc. Med. Am. J. , 467–477 (2007). 467–477 , rti-eeto epnin irsoy f el ad tissues and cells of microscopy expansion Protein-retention utpee 3 clua sprrslto iaig ih DNA- with imaging super-resolution cellular 3D Multiplexed Nat. Methods Nat. IEEE Rev. Biomed. Eng. Biomed. Rev. IEEE xaso mcocp wt cnetoa atbde and antibodies conventional with microscopy Expansion lh-cii- i rqie fr oml ooye adhesion. podocyte normal for required is Alpha-actinin-4
Nat. Methods Nat. 4 , 32 (2016). 32 , Breast Cancer Res. Treat.Res. Cancer Breast J. Histochem. Cytochem. Histochem. J. J. Neuropathol.J. Neurol.Exp. Nat. Methods Nat. Phys. Ther. Phys. J. Cell Biol. Cell J.
13
9 , 485–488 (2016). 485–488 ,
, 743–748 (2012). 743–748 , 313 PLoS One PLoS
Nat. Methods Nat.
2
85 , 1122–1132 (2015). 1122–1132 , 11 Nat.Biotechnol. , 445–453 (2007). 445–453 , 139
, 257–268 (2005). 257–268 , , 313–318 (2014). 313–318 , a. Biotechnol. Nat. 7 PLoS One PLoS , 97–114 (2014). 97–114 , nature biotechnology nature , 193–204 (1997). 193–204 , Microsc. Res. Tech.
9
. itce. Cytochem. Histochem. J. 50 152 , e114885 (2014). e114885 , Histopathology Eur.Histochem. J. Sci. Rep. Sci.
, 437–439 (2002). 437–439 , 64 , 687–694 (2015). 687–694 ,
11 , 113–122(2005)., 4 . op Neurol. Comp. J. , e4194 (2009). e4194 ,
, 360–361 (2014). 360–361 , 34 a. Biotechnol. Nat.
Science , 973–981, (2016).
5 , 13402 (2015). 13402 , 21
1347–1355 , 65
Science J. Histochem. 30 Nat. Methods Nat.
240–251 , 347 51 , 513–520 AE J. FASEB , 59–66 , , 543– , in situ in
348 238
47 34 , , , ,
© 2017 Nature America, Inc., part of Springer Nature. All rights reserved. 0.1 mg/ml for samples fixed with aldehyde fixatives) diluted in PBS buffer for at 0.03–0.1 mg/ml AcX (0.03 mg/ml for samples fixed with nonaldehyde fixatives, with incubated were slides Tissue °C. −20 environmentat desiccated a in zen storedand aliquotted fro itwas then atconcentrationDMSO mg/mL, a of10 anhydrous in dissolved was Scientific) Fisher fromThermoAcX, abbreviated protocol proExM reported previously acid, ester,succinimidyl s.e.m. (6-((acryloyl)amino)hexanoic Acryloyl-X, here our of variation a is used method preservation. protein for treatment Chemical in used were A-11008) cat# Scientific, (ThermoFisher 488 Alexa anti-rabbit goat and 3 6 in used was A-11073) cat# Scientific, (ThermoFisher 488 Alexa pig anti-guinea goat except 20341); cat# (Biotium, CF633 mouse anti- goat and A-11010) cat# Scientific, goat (ThermoFisher 546 Alexa anti-rabbit A-11039), cat# Scientific, (ThermoFisher 488 Alexa anti-chicken goat this in used antibodies in work are listed primary the RT.All at each min 10 for times three Medium Washing MAXwash in washed then tissues), thicker for needed be may or duration temperature of incubation optimization (further tissue thick 5 for °C, 37 at h 1 least at for Medium Staining MAXbind in Scientific) nM DAPI (when nuclear imaging was required; DAPI was from Thermo Fisher10 approximately of concentration a at antibodies secondary appropriate with incubated were RT.Samples at each min 10 for and then washed three times with MAXwash Washing Medium (Active Motif) 10 in antibodies MAXbind Staining Medium at (Active Motif) a concentration of primary for with 1 h by at incubation 37 followed Motif) °C, (Active Medium Immunostaining. 60 for°C incubator 30 min. a to transferred immediately was container the then and container, resistant heat- a in °C ~100 at RT) at measured when 8, (pH solution citrate sodium mM 20 in placed were slides tissue Briefly, immunostaining. before treated Sample heat treatment. each. at RT min 5 for solution PBS with 3× washed and melt OCT the let to min 2 RTfor at left were slides the sections, three PBS washes for 10 min each at RT. For before already fixed frozen tissue °C clinical −20 at acetone in min 10 for fixed initially were (Tissue-Tek) tion samples. FFPE as treated then in incubated at xylene further RT until the coverslip Slides were was loosened. were slides the remove, to difficult was coverslip the If blade. razor a as such tools, appropriate with removed carefully were RT. at coverslips xylene Then each. min 3 (RT), doubly deionized water. All of these steps were performed at room temperature 2× xylene, 100% ethanol, 95% ethanol, 70% ethanol, 50% ethanol and (finally) 2× solutions: of series a in placed sequentially were samples samples, clinical recovery. section Tissue specimens does not require informed consent from the subjects. sources (see humantissue samples and tissue microarrays were purchased from commercial were bought from either US Biomax or Abcam ( #2011P002692 to A.W. The rest of the breastWomen’sandBrigham the byand protocol IRB BWH kidneyHospital thearchives under samples used in this study logical samples used in patho kidney frozen TheAbcam). from cases 21 andBiomax US from cases protocolBIDMCunderIRB11 #2013p000410addition,A.H.B. used(in towe from the pathology archives of the Beth Israel Deaconesscases from Medicalthe study Center,on ExPath-based obtained analysis of early breast lesionsHuman ( samples. METHODS ONLINE doi: least 3 h at RT. Note that thicker samples require longer incubation times. , , goat anti-mouse Atto cat# 50185) was 647N (Sigma-Aldrich, used in , and goat anti-chicken Alexa 546 (ThermoFisher Scientific, cat# A-11040) A-11040) cat# Scientific, (ThermoFisher 546 Alexa anti-chicken goat and , Unfixed frozen tissue slides in optimum cutting temperature (OCT) solu (OCT) temperature cutting optimum in slides tissue frozen Unfixed in placed briefly were samples slides, permanent mounted and For stained µ 10.1038/nbt.3892 g/mL g/mL for at least 3 h at RT or 37 °C (in our hands, it did not matter which), Figure Figure Supplementary Table 14 4 . The breast pathological specimens used in Supplementary Table 15 Supplementary Samples were first blocked with MAXblock Blocking Blocking MAXblock with blocked first were Samples Figure 4d All human tissue samples used in this study were heat For formalin-fixed paraffin-embedded (FFPE) (FFPE) paraffin-embedded formalin-fixed For – g and ). The use of unused, unidentified archival Supplementary Figure 8 . Secondary antibodies used were: used antibodies . Secondary Supplementary Table 14 The expansion microscopy microscopy expansion The µ g/mL together with 300 300 with together g/mL Supplementary Figure Figure Supplementary Figure 1j were provided Fig. , k and nine ). Other 5 Figure Figure ) were µ m 9 - - - . for 3 h at 60 °C or until the completion of digestion (i.e., the gelled tissue is is tissue gelled the (i.e., digestion of completion the until or °C 60 at h 3 for incubated were tissues the and NaCl; M 0.8 and X-100 Triton 0.5% EDTA, mM 25 8), (pH Tris mM 50 of consisting recipe) proExM original the from (modified buffer digestion a in Biolabs) England (New K proteinase U/ml 8 expansion. and digestion Sample gelation. complete to atmosphere h at a in °C 37 for humidified 1.5–2 incubated were samples slice Finally, solution. gelling fresh the with filled was chamber gel The tissue. of compression prevent to section tissue the of side either on spacers with sue, tis the of top on coverslip a putting by constructed was chamber gel a Then, gelation. premature preventing while tissues the into solution monomer of diffusion allow to °C 4 at min 30 for solution monomer the with incubated for (w/w) 4HT and for 0.2% (w/w) APS were both Tissue and slides TEMED). the to sequentially added 0.01% concentration, (final solution to gelling the prepare each solution monomer were initiator an as Sigma-Aldrich) (APS, persulfate and as ammonium an accelerator Sigma-Aldrich) (TEMED, amine piperidin-1-oxyl (4HT, Sigma-Aldrich) as an inhibitor, tetramethylethylenedi 4-hydroxy-2,2,6,6-tetramethyl chemicals The of sturdiness. gel expense lower the slightly at protocols, previous than expansion more slightly caused at stored °C before −20 and aliquotted and prepared was Sigma-Aldrich) from (all short) 2.5% acrylate, sodium and N,N (w/w) 0.10% acrylamide (w/w) (w/w) 8.625% NaCl, M 2 PBS, 1× of made solution protocol proExM original our from modified slightly is In situ Figures 2a (ii 2a Figures For (Nikon). NAobjective air 0.2 10× or objective NA air 0.13 4× a with software, AR NIS-Elements by led control (Andor), camera sCMOS Zyla 5.5 a and (Lumencor) engine light X SPECTRA a with microscope epifluorescence Ti-E Nikon a on imaged were ( mens Fluorescent microscopy after expansion. imaging. pre-expansion for matching index (Invitrogen) antifade reagent for suppression of photobleaching and refractive NA 1.40 (Olympus) oil 100× objective. Stained a samples were with imaged with SlowFade Gold microscope SIM Healthcare) (GE Blaze OMX Deltavision a on performed was imaging microscopy illumination structured resolution Super- “Immunostaining.” in described as DAPI and antibodies secondary as well as anti-vimentin) chicken anti-KRT19, (rabbit primary with stained at RT. performed were steps the all stated, specifically ATCC. Unless the of procedures standard via contamination for mycoplasma tested and ticated authen were Cells needed. is line cell of choice the for detailed no justification so scientific conclusions, scientific make to not bed, test technology a as used were cells These each. min 5 for times three PBS in washed then and °C 37 at h 1–4 for Medium Staining MAXbind in DAPI nM 300 and antibodies secondary with incubated then were Specimens each. min 5 for times three 1–4 h at 37 °C and then washed in MAXwash Washing Medium (Active Motif) MAXbind Staining Medium at(Active Motif) a concentration of 10 anti- (rabbit antibodies primary with stained were cells with 0.1% Tritonpermeabilized X-100 for 15 min. Microtubules and in fixed HeLa PBS, imaging. with each min 5 for times three washed min, 10 for pre-expansion formaldehyde microscopy 3 Figure Supplementary illumination Structured 10%. by around factor expansion the may reduce azide of sodium Note addition that for expansion. used water the to 0.002–0.01%) concentration (final azide sodium added sometimes we growth, To bacterial prevent stabilized. sample expanded of the size the until water in to fresh times five three for step was 10 repeated This min to expand. PBS for 10 min at RT. Finally, gels were placed in doubly water deionized at RT nM DAPI in PBS for buffer 20 min at RT; were then they once washed with 1× 300 with stained RTand at min 10 for buffer PBS 1× with once washed were samples Digested solution). the in twisting or bending without flat remains tissue gelled the and transparent, becomes and slide glass the from detached For For polymer synthesis. polymer Figure 1 Figure Fig. 1b Fig. – , in in situ c vi) f ; ‘core’ images of of ‘core’ ; images , a customized 5 , a customized , b ( polymer synthesis. The slightly lower BIS concentration concentration BIS lower slightly The synthesis. polymer ii , iii , HeLa cells (ATCC CCL2) were fixed with , 4% (ATCCwith para HeLa cells were fixed CCL2) , The method for v ,
and Figures 1k Figures µ vi Fig. Fig. m thickness breast TMA was prepared and prepared was TMA breast m thickness After gelation, samples were incubated in in incubated were samples gelation, After ), i , 3 j , , ; ; ′ 6 Supplementary Figs. 2e Figs. Supplementary -methylenebisacrylamide (or BIS for -methylenebisacrylamide and and – Low-magnification images of speci in situ m , 2a 8 nature biotechnology nature , the images were acquired on acquired were images , the – polymer synthesis in ExPath j j and
5 α 8 and and . Briefly, a monomer monomer a Briefly, . -tubulin, Abcam) in Abcam) -tubulin, Supplementary Supplementary – h µ , g/mL g/mL for 4 and and For For 5 ------) © 2017 Nature America, Inc., part of Springer Nature. All rights reserved. Expansion immunoFISH. Expansion patients. three at least from length measurement of function as a plotted was for error measurements and RMS such the sampled, be easily could errors measurement distance points, two any at vectors resulting the subtracting By fields. vector distortion and factors expansion of calculation as well as scaling, uniform and translation by rotation, registration image for the with most SIFT images key was points used and postexpansion expansion pre- of pair The scaling. uniform model and translation rotation, geometric permits a only that with (RANSAC) consensus sample random by filtered library open-source VLFeat the using generated were points key SIFT plane should correspond to one postexpansion tions (note that, since the sample expands along the pre-expansion of pairs of combination invariant feature (SIFT) key transform points Tomatch postexpansion. multiple in view of fields same matching finding field vector distortion the to leads that registration nonlinear the for points control of code, selection manual for improved need the with eliminating calculation error RMS and field vector distortion the square (RMS) error calculation, with minor modifications. We semiautomated method described quantification. error Measurement process. expansion the of variability sample-to-sample natural (<10%) small the out nulls also process normalization This used. be to units length logical’ ‘bio enable thus and state pre-expansion the of that to image postexpansion the of size physical the normalize to used be can which factor, expansion the expansion and and postexpansion then take the ratio of these sizes to calculate pre- sample the of images low-magnification acquire users that here, did we factor expansion the of Measurement ratios. background signal-to- calculate to used and selected were inspection, visual pathologist’s a by judged as signal, autofluorescence only containing area one and signals each For analysis. fluorescent channel, before ten regions of interestregions containing the brightest blank fluorescent from values pixel mean of traction analysis. Autofluorescence (Zeiss). NAobjective air 0.95 40× a with (3DHistech) 1j ( slides H&E of images High-magnification NAobjective. air 0.2 10× or objective air NA 0.13 4× a with illuminator LED white and (Nikon) Camera were acquired on a Nikon Ti-E microscope with a DS-Ri2 sCMOS 16mp Color microscopy. Bright-field 8 Figure Supplementary 0.25 of 7 imaging. before gels the encase to solidify to allowed and gel the around pipetted was w/w) (2% agarose low-melt liquid needed, was immobilization If removed. liquid excess all with plates six-well glass-bottom in placed were filter. emission a 685/40 with and imaged laser nm a 640 with were excited a 561 nm laser and imaged with a 607/36 emission filter. Atto 647N and CF633 laser and imaged with a 525/40 emission filter. Alexa Fluor 546 was excited with imaged with a 450/50 emission filter. Alexa Fluor 488 was excited with a 488 nm 1.15 NA water-immersion objective. DAPI was excited with a 405 nm laser and 40× a with body microscope TI-E Nikon a on Yokogawa)X1 system confocal Cy5-4040C-000. 633, CF or Atto 647N FITC/TXRED-2X-B-NTE; 546, Fluor Alexa GFP-1828A-NTE-ZERO; 488, Fluor Alexa DAPI-11LP-A-000; New York) Rochester, DAPI, used: were (Semrock, cubes filter following The with a the same microscope 40× 1.15 NA (Nikon). objective water-immersion nature biotechnology nature in placed were samples gel digested probing, DNA FISH plus histochemistry and and Figures 1g Figures To prevent they the expansion, following gels during imaging from drifting images All were other fluorescence taken on an Andor spinning disk (CSU- and Supplementary Figure 9 Figure Supplementary
5 µ ; ; m thickness (in pre-ExM distance units). units). distance pre-ExM (in thickness m Supplementary Fig. 9 Fig. Supplementary , 2 and z 8 planes in pre-expansion versus postexpansion states, the the states, postexpansion versus pre-expansion in planes for distortion vector field calculation and root-mean- and calculation field vector distortion for
3 (except core images); core images); (except are MIPs of 2 2 of MIPs are Low-magnification images ( Low-magnification z For for immuno ExPath processed samples being planes pre-expansion and postexpansion, scale- postexpansion, and pre-expansion planes Background was removed from images by sub by images from removed was Background z are maximum intensity projections (MIPs) (MIPs) projections intensity maximum are planes were first imaged pre-expansion and and pre-expansion imaged first were planes ) were acquired on the Panoramic Scan II II Scan Panoramic the on acquired were ) This section is based on our previously previously our on based is section This z µ planes and postexpansion postexpansion and planes and m thickness. m Figure Figure normalization. 3 6 z were for generated all possible projection from 4–5 z 4 Figures 1k Figures axis, one pre-expansion 8 ; ; . Given the challenge of of challenge the Given . Supplementary Figure Figure Supplementary Supplementary Fig. 4 Supplementary We suggest, as as suggest, We – m m and z z planes). projec
3 5 7 Figs. Figs. and and and and z - - - - )
tiles exhibited background fluorescent signals. During image preprocessing, a image preprocessing, During signals. fluorescent background exhibited tiles verlapping image tiles, which required stitching to produce a single image, these Image preprocessing. image preprocessing and nuclei segmentation pipelines are shown in processing, nuclei segmentation, feature extraction and image classification. The imageThisclassification framework consisted fourofcomponents: imagepre breastbenign lesions (UDHand ADH) and noninvasive breast cancer (DCIS). classification of expanded tissue images into different categories: normal breast, cases: 105 all followsused aslisted figures and tables respondingthe below), section 6 listed as follows used 31 cases outfigures andof tables the the totalto leading ofanalyses 105the segmentation, nucleuscases: and tion Computational nuclear atypia analysis. completed. was expansion the until each) at RTmin (5 times multiple SSC 0.02× with washed were samples gel the Finally, each. min 10 for times three at °C 45 SSC 2× with by washes followed min, at for 15 °C 45 carbonate ylene and pH eth 20% 7.0) of citrate, made mM 1× NaCl, 15 (150 SSC mM sodium buffer wash stringency with washed were samples day, next The overnight. at at was preheated 85 45 °C °C incubated were for then 10 The mixtures min. and HER2 and SureFISH probes CEP 17q12 contained (Agilent/Dako) Chr17 85 °C for 30 min; then they were mixed with 30 DNA at sperm salmon single-stranded mg/ml 0.2 and NaCl mM 600 sulfate, dextran 20% carbonate, ethylene 15% PBS, 1× of made buffer hybridization features corresponded to the textural variation inside nuclei. inside variation textural the to corresponded features statistical second-order The kurtosis. and skewness range, interquartile s.d., deviation, absolute mean median, mean, were features statistical first-order computed The nuclei. within values of gray-level distribution corre the to sponded features statistical first-order The radius. major and minor elliptical orientation, solidity, extent, compactness, major axis length, minor axis length, morphological computed eccentricity, perimeter, equivalent perimeter, convex area, were area, features The information. phenotypic nuclear extracted represent which features, and geometrical shape included features phological mor The nucleus. each for features statistical second-order and first-order extraction. Feature nuclei. overlapping and touching rate to sepa algorithm watershed as for were markers used the marker-controlled in the response scale-space were as selected seed points. Last, these seed points filter was applied on a given number of and scales, then maximum local points address the problem of imprecise nuclear seed points, a MSLoG scale-adaptive to order In lesions. neoplasia breast early in considerably varies size nuclear since points, seed nuclear imprecise produces scale constant maxima, local filter (MSLoG) Gaussian of Laplacian multiscale scale-adaptive a used we nuclei, nuclei that touch each other. Second, to separate the touching and overlapping nuclei and This artifacts). segmentation method may undersegment clusters of of fragmented parts vessels, blood (e.g., regions non-nucleus to remove small opening as morphological as well nuclei into single of nuclei fragments small to closing holes that and fill include and hole to filing morphological combine operations morphological of set a by improved then was nuclei of mentation entropy seg and The relative the Poisson initial the image model. histogram mixture between the minimizing by computed was value threshold The els. threshold by the modelling image histogram as a mixture of two Poisson mod the selected algorithm thresholding adaptive our locally issue, this to address order In regions. background and regions nuclei the within variability high is there when method threshold error minimum a as efficient as not are ods method thresholding error mum mini Poisson-distribution-based a using segmented were nuclei First, steps. segmentation. Nuclei 10. radius with by filter a median were smoothed images then enhanced background contrastnucleus-to- was enhancedremoval, by noise adaptive histogrambackground equalization After noise. background remove to algorithm rolling-ball – 8 and 4 1 Supplementary Tables 9 to produce maxima local and seed points select for nuclei. For selecting SupplementaryFigure 9 After nuclei segmentation, we extracted morphological, morphological, extracted we segmentation, nuclei After Due to confocal acquisition (see above) of multiple nono 3 The nuclei segmentation procedure consisted of three of three consisted procedure segmentation nuclei The 8 with size ball set to the average nuclei size was applied – 12 . For. thetask of image classification (seecor 4 and 0 . Standard and global thresholding meth thresholding global and Standard . Figure 5 For the task of evaluating nucleus detec c µ . We proposed a framework for L L of hybridization buffer that Supplementary Tables doi: 10.1038/nbt.3892 Figure 5 3 9 . These a ------. © 2017 Nature America, Inc., part of Springer Nature. All rights reserved. listic classifier based on applying Bayes’ theorem with strong independence independence strong with theorem Bayes’ applying on based classifier listic Bayes Naïve was explored we classifier other The 20. = In our experiments we used numTrees = 100, maxDepth = 30 classifer. and Forest numFeaturesRandom the of performance are the affect that selection parameters three random in used be to (numFeatures) features of number and (maxDepth) tree the of depth (numTrees),maximum trees of Number ting. overfit and to control accuracy to predictive improve the averaging and uses classifier Forest Random A research. biomedical in used commonly are which sifiers, AUCan 0.5. of achieve would classifier an AUCachieve would of classifier 1, a where and perfect a random positives, performance was by assessed computing the AUC of true positives versus false data training set and applied to model this fixed data the Model set. validation that achieved the maximum area of under curve (AUC)value in the cross-validation on selected the we validation, For (6F-CV). cross-validation six-fold with performance model We evaluated glmnet. in setting selected the using construction, model before sets data and validation training in the separately Features data research. cancer in were dimensional translational standardized colinearity parameter number of by nonzero features in is the regularization the model) determined weights of the least predictive features to 0. The amount of the penalty (and the regression the shrinking of effect the has which penalty, regularization L1 an regression. The Lasso procedure consists of performing logistic regression with logistic standard from obtained be would that those than prone to overfitting glmnet package ( R in implemented were analyses The images. tissue malignant preinvasive and benign normal, classify to models regression with to regularization Lasso build multivariate image feature-based Image classification. image. per features summary 315 producing image, per feature each of kurtosis and skewness range, interquartile s.d., deviation, by computing the first-order statistics, including mean, median, mean absolute tures for each features nucleus. these Last, were at summarized the image level fea 45 computed we total, In (LRHGLE). emphasis level gray high run long and (LRLGLE) emphasis level gray low run long (SRHGLE), emphasis level gray high run short (SRLGLE), emphasis level gray low run short (HGLRE), age (RP), low gray level runs emphasis (LGLRE), high gray level runs emphasis ratio-percent (RLN), nonuniformity length run (GLN), gray nonuniformity level (LRE), emphasis run long (SRE), emphasis run short are GLRLM, the from derived features, run-length 11 The them. averaged and directions four R ( GLRLM matrix dimension of the GLRLM is run-length gray-level a constitutes direction, 2. correlation measure 1 and information correlation moment normalized, dissimilarity, maximum probability, information measure energy, entropy, homogeneity, inverse difference normalized, inverse difference cluster prominence, cluster shade, contrast, correlation, were autocorrelation, from the GLCM in order to identify texture more compactly. These 14 features duced one GLCM matrix. Later, we computed 14 features proposed by Haralick was rotationally invariant. So we took the average in all four directions and pro vector, which produced four GLCM Inmatrices. our case, texture information directions (vertical, horizontal, left and right diagonals) with one displacement pixel with gray level that a as probability the pixel gray level with considered were made. Alternatively, we can say that each element of the GLCM is matrix that comparisons such of number total the by divided was matrix whole the value with pixel a where occasions of number total the counting by produced was at in The image. value the gray levels the ( GLCM Haralick co-occurrence doi: the features. assumptions As between value is the predicted class (i.e., we label is the maximum run length. Similar to the GLCM, we computed GLRLMs for We also evaluated our framework using two other machine learning clas learning machine other two using framework our evaluated Wealso given a in collinear level, gray same the with pixels, consecutive of set The We computed two of types features statistical second-order using gray-level 10.1038/nbt.3892 i is adjacent to a pixel with value value with pixel a to adjacent is i 4 , 7 j ; ; fits a number of decision trees on various subsamples of the data set 4 λ d 6 . This method has been shown to perform well in the setting of of setting the in well perform to shown been has method This . and has been widely used to build predictive models from high- from models predictive to build used widely been and has , θ ) is square with dimension dimension with square ) is 4 4 . Lasso regularization j at a distance In the last part of logistic our we framework, performed 4 2 and run-length Ng × R d , , where and angle 4 5 http://www.r-project.org was used to create simpler models less 4 i 3 j th column and th column Ng at a distance distance a at matrices. The co-occurrence matrix Ng , where , where is the number of gray levels, and θ . . We in four adjacency defined Ng 4 is the total number of number total the is 8 i d is to found a be with j , which is a probabi a is which , th th row in matrix the and angle angle and / ), using the the using ), i , j ; ; θ θ . Then Then . ). The The ). λ ------
expanded data as compared expanded to an AUC of value data. A 0.84 for pre-expanded cancer tissue (DCIS), the GLMNET classifier reported an AUC value of 0.95 for breast versus (ADH) noninvasive breast tissue benign atypical discriminating com as data pared to AUC of values expanded 0.71 and respectively, 0.82, for data. For for pre-expanded 0.89 and 0.93 of values AUC reported classifier GLMNET the DCIS), and (ADH tissues breast atypical from (UDH) tissue 0.75, respectively, for pre-expanded data. For differentiating nonatypical breast and 0.82 to AUC of 0.86, compared as data values for expanded 0.94 and 0.96 0.95, of AUC values reported classifier GLMNET the tissue, DCIS and ADH ( classes all for classification binary performed we noninvasive, and benign versus tissue judged to be borderline diagnostic cases. In order to discriminate normalwere breastthey because excluded were 26 and analyzed, were images 105 131; was images of number total The cases. examined 350 than more from S.J.S.) and validated by pathologists and three certified authors of this study (E.-Y.O., V.T. and performed was classification ground-truth The DCIS). and ADH UDH, (normal, classes different four to belonged lower images a 105 is these Thus, this bound). samples, commercial in age and sex by only identified were tissue images from (DCIS) 41 cases (likely different patients, but since patients benign breast 31 noninvasive 38 and ADH) 16 and UDH (15 images images, tissue breast lesion tissue breast normal 36 containing images 105 of sisted to images from both and pre-expanded samples. expanded Both data sets con results. classification Image regression to compared as classification for restrictive less is assumption independence the problem), classification a pursuing are 48. 47. 46. 45. 44. 43. 42. 41. 40. 39. 38. 37. 36. paper. the of authors sionmicroscopy.or at are posted analysis atypia nuclear for used the computational statement. availability Data mean. the indicates symbol square the and median, spans from minimum to maximum; the in segment indicates the the rectangle rectangle central the and s.d., by the defined are whiskers of ends the graphs, boxplot In the curves. operator receiver compare to statistically used was test paired bootstrapped A means. between differences significant determine n as mean are presented Data analysis. Statistical in reported 9 Table ing classifiers (Naïve Bayes and Random Forest) is reported in comparison of GLMNET classification results versus two other machine learn noted in the text, tables and figure legends. Student’s legends. figure and tables text, the in noted
John, G.H. & Langley, P. Estimating continuous distributions in Bayesian classifiers. forests. Random L. Breiman, Dormann, C.F. lasso. the via selection and shrinkage Regression Tibshirani,R. linear generalized for paths T.Tibshirani,Regularization Hastie, &R. J., Friedman, image for features Textural I. Galloway, M.M. Texture analysis using gray level run lengths. Dinstein, & K. Shanmugam, R.M., Haralick, detection automatic Improved B. Roysam, & W. Lee, W., Lassoued, Y., Al-Kofahi, thresholding. error minimum distribution-based Poisson on Notes J. Fan, In equalization. histogram adaptive limited Contrast K. Zuiderveld, processing. image Biomedical S.R. Sternberg, vision computer of library portable and open an - Vlfeat B. Fulkerson, & Vedaldi,A. Lowe, D.G. Distinctive image features from scale-invariant keypoints. Besnard, P. & Hanks, S.) 338–345 (Morgan Kaufmann Publishers, 1995). Publishers, Kaufmann (Morgan 338–345 S.) Hanks, P.& Besnard, In performance. their evaluating study 58 descent. coordinate via models Process. classification. 57 and segmentation of cell nuclei in histopathology images. Lett. Recognit. IV A.D. algorithms. In Vis. (Ed. Heckbert, P.S.), 474–485 (AP Professional, 1994). Professional, (AP P.S.),474–485 Heckbert, (Ed. , 267–288 (1996). 267–288 , (2010). 841–852 , Proc. of the 11th Conference on Uncertainty in Artificial Intelligence Artificial in Uncertainty on Conference 11th the of Proc.
60 . Top-performing features in expanded and pre-expanded data are are data pre-expanded and expanded in features Top-performing . et al et , 91–110 (2004). 91–110 ,
Fig. Fig. 4 Supplementary Tables 10 Supplementary .) 1469–1472 (ACM Press, 2010). Press, (ACM 1469–1472 .) , 172–179 (1975). 172–179 , 5 Proc. 18th ACM International Conference on Multimedia IEEE Trans. Syst. Man Cybern. Man Trans.Syst. IEEE et al.
c 19 ). When discriminating normal breast tissue versus UDH, UDH, versus tissue breast normal discriminating When ). g Statistical analyses were with performed R (version 3.2.5). , 425–431 (1998). 425–431 , . Data are available upon request to the corresponding corresponding the to request upon available are Data . Collinearity: a review of methods to deal with it and a simulation ± The expansion pathology protocol and protocol the code pathology The expansion We applied our image classification framework framework classification We image our applied s.d. (SD) or s.e.m. (SEM) with sample numbers numbers sample with (SEM) or s.e.m. (SD) s.d. Mach. Learn. Mach. J. Stat. Softw. Stat. J. Ecography (Cop.) Ecography and and
45 11
3
, 5–32 (2001). 5–32 , Computer 33 , , respectively. , 610–621 (1973). 610–621 , nature biotechnology nature , 1–22 (2010). 1–22 ,
36
4 , 27–46 (2013). 27–46 , IEEE Trans. Biomed. Eng. 16 8 . Comput. Graph. Image , 22–34 (1983). 22–34 , t -test was used to to used was -test J. R. Stat. Soc. B Soc. Stat. R. J. Supplementary Supplementary http://ex Graphics gems Graphics Int. J. Comput. (eds. Bimbo, Pattern (eds. pan t - - - - -
Supplementary Figure 1
The full workflow of Expansion Pathology.
Nature Biotechnology: doi:10.1038/nbt.3892
Nature Biotechnology: doi:10.1038/nbt.3892 Supplementary Figure 2
Conditions that affect the successful expansion of human tissues.
(A) Images of human skin samples, both taken by a conventional camera (i, iv) and via fluorescence microscopy (ii-iii, v-vi) when stained with DAPI (grey) and antibodies against vimentin (green) and smooth muscle actin (gene name, ACTA2) (magenta). The samples were digested with 8 units/mL proteinase K solution containing 25 mM Tris (pH 8), 1 mM EDTA, 0.25% Triton X-100, and 0.4 M NaCl at 60°C for 0.5 hour (i-iii) or 2 hours (iv-vi). (i and iv) Conventional camera (i.e., non-fluorescent) photographs of human skin- hydrogel hybrid samples in PBS (~2x expansion), after digestion for 0.5 hour (i) or 2 hours (iv). (ii) Widefield fluorescent image from the sample of (i) pre-digestion. (iii) Post-expansion wide-field fluorescent image of the sample of (ii), with a dashed orange box highlighting regions with autofluorescence in the DAPI channel and distorted vimentin networks post-expansion. (v) As in (ii), for the sample in (iv) pre-digestion. (vi) As in (iii) for the sample of (v) with a dashed orange box highlighting regions with autofluorescence in the DAPI channel. (B) As in (A), except that the samples were digested with 8 units/mL proteinase K solution containing 25 mM Tris (pH 8), 25 mM EDTA, 0.25% Triton X-100, and 0.4 M NaCl, at 60°C for 0.5 hour (i-iii) or 2 hours (iv-vi). Note that in Bi and Biv the samples look quite transparent, due to the heavy digestion. (C) Photographs of human liver samples digested with a 1 mM EDTA-based solution – specifically, 8 units/mL proteinase K solution containing 25 mM Tris (pH 8), 1 mM EDTA, 0.25% Triton X-100, and 0.4 M NaCl at 60°C for 0.5 hour (left) or 2 hours (right). (D) Photographs of human liver samples digested with a 25 mM EDTA-based solution – specifically, 8 units/mL proteinase K solution containing 25 mM Tris (pH 8), 25 mM EDTA, 0.25% Triton X-100, and 0.4 M NaCl at 60°C for 0.5 hours (left) or 2 hours (right). Note that the samples of D are significantly more transparent than those of C. (E) Widefield fluorescent image of a human liver sample stained with DAPI (grey) and an antibody against smooth muscle actin (gene name, ACTA2) (magenta) prior to the ExPath process. (F) Post-expansion wide-field fluorescent image of the sample of E, digested with a 1 mM EDTA-based solution for 1 hour. White dashed line outlines an out-of-focus region caused by distortion. (G) Wide-field fluorescent image of a human liver sample stained with DAPI (grey) and an antibody against smooth muscle actin (gene name, ACTA2) (magenta) prior to the ExPath process. (H) Post-expansion wide-field fluorescent image of the same sample as in G. The sample had been digested with a 25 mM EDTA-based solution for 0.5 hour. (I) Post-expansion confocal image of human lymph node tissue with invaded breast cancer stained with DAPI (blue) and an antibody against vimentin (green), and treated with a 1 mM EDTA-based solution for 3 hours. (J) Post- expansion confocal image of a different sample from the same tissue used in I, treated with a 25 mM EDTA-based solution. (K) Post- expansion confocal image of normal human kidney tissue fixed with acetone, stained with an antibody against collagen IV (magenta), and treated with 0.1 mg/ml Acryloyl-X prior to in situ polymerization. Cracks are indicated by white arrows. (L) Post-expansion confocal image of a different sample from the same tissue used in K, treated with 0.03 mg/ml Acryloyl-X prior to in situ polymerization. Scale bars (yellow scale bars indicate post-expansion images: Aii and iii, 9.2 μm (physical size in iii: 40 μm, expansion factor: 4.33). Av and vi, 9.4 μm (physical size in vi: 40 μm, expansion factor: 4.28). Bii and iii, 9 μm (physical size in iii: 40 μm, expansion factor: 4.41). Bv and vi, 8.9 μm (physical size in vi: 40 μm, expansion factor: 4.51). E and F, 119 μm (physical size in F: 500 μm, expansion factor: 4.22); G and H, 109 μm (physical size in H: 500 μm, expansion factor: 4.58). (I-L) 40 μm, physical size.
Nature Biotechnology: doi:10.1038/nbt.3892
Supplementary Figure 3
Distortion evaluation of post-expansion HeLa cells processed by 25 mM EDTA-assisted proteinase K digestion.
(A) Super-resolution structured illumination microscopy (SR-SIM) image of a HeLa cell stained with DAPI and an antibody against α- tubulin. Blue, DAPI; green, α-tubulin. (B) Post-expansion image of the same sample acquired with a spinning disk confocal microscope. (C and D) Root mean square (RMS) length measurement error as a function of measurement length for ExPath vs SIM images (blue solid line, mean of DAPI channel; green solid line, mean of α-tubulin channel; shaded area, standard deviation; n = 3 samples from separate cultures). Scale bars: (A) 4 μm, (B) 4 μm (for yellow scale bar, physical size post-expansion, 19.5 μm; expansion factor: 4.89).
Nature Biotechnology: doi:10.1038/nbt.3892
Nature Biotechnology: doi:10.1038/nbt.3892 Supplementary Figure 4
Fluorescence of unstained tissue vs. H&E stained tissue.
(A) Brightfield image of unstained formalin-preserved human lymph node tissue with invaded breast cancer. (B-E) Widefield fluorescent images of the same sample as in A in four fluorescent channels (listed below). (F) Bright field image of tissue from the same block as in A, stained with H&E. (G-J) Widefield fluorescent images of the sample of F in four fluorescent channels. Filters for each fluorescent channel (Semrock, Rochester, NY): B and G: excitation 628 ± 20 nm, emission 692 ± 20 nm; C and H: excitation 586 ± 10 nm, emission 628 ± 19 nm; D and I: excitation 482 ± 9 nm, emission 520 ± 14 nm; E and J: excitation 387 ± 6 nm, emission 409 nm and longer. Images from a given fluorescent channel are adjusted to have the same contrast. (K) Brightfield and widefield fluorescent images of tissue from the same block as in A, stained with H&E, after each step of ExPath processing. Images from a given fluorescent channel are adjusted to have the same contrast. Samples may be especially bright after the heat treatment and gelation steps due to the buffer compositions present. Scale bars: A-J, 500 μm; K, 40 μm (for yellow scalebars, post-expansion, physical size 184 μm; expansion factor: 4.58).
Nature Biotechnology: doi:10.1038/nbt.3892
Supplementary Figure 5
Heat treatment significantly improves immunostaining of vimentin and actinin-4 on human kidney samples.
Left panel, widefield fluorescent images of acetone fixed human kidney sections with and without heat treatment in citrate buffer. Right panel, zooms into regions corresponding to the regions indicated by the dashed yellow rectangles in the left panels. Scale bars: 1 mm (left panel), 200 μm (right panel). Abbreviations: ACTN4, actinin-4; ColIV, collagen IV.
Nature Biotechnology: doi:10.1038/nbt.3892
Supplementary Figure 6
Anti-actinin-4 specifically stains tertiary podocyte foot processes.
(A-D) Post-expansion widefield images of a human acetone fixed kidney section stained with DAPI and antibodies. Blue, vimentin; green, actinin-4; red, synaptopodin; grey, DAPI. (E) Merged image of (A-D). (F and G) Magnified regions showing actinin-4 (F) and synaptopodin (G) zoomed into the white dashed squares of B and C. (H) Profiles of fluorescent intensity taken along the white dashed line cuts of F and G. Green, actinin-4; red, synaptopodin. Scale bars: 1 μm (4.5 μm physical size, expansion factor 4.5).
Nature Biotechnology: doi:10.1038/nbt.3892
Supplementary Figure 7
Immunostaining images of FFPE kidney samples
A. Post-expansion widefield image of a FFPE normal human kidney sample treated with a citrate antigen retrieval method (20 mM sodium citrate, pH 8.0), with highlighted region (boxed line) magnified in panel B. C. Post-expansion widefield image of a FFPE normal human kidney sample treated with a Tris-EDTA antigen retrieval method (10 mM Tris base, 1 mM EDTA solution, 0.05% Tween 20, pH 9.0), with highlighted region (boxed line) magnified in panel D. E. Post-expansion confocal image of a FFPE normal human kidney sample treated with a citrate antigen retrieval method, with highlighted region (boxed line) magnified in F. G. Post-expansion confocal image of a FFPE human kidney sample with minimal change disease treated with a citrate antigen retrieval method, with highlighted region (boxed line) magnified in H. All the samples were stained with DAPI (gray) and antibodies against vimentin (blue), actinin-4 (green) and collagen IV (magenta). Scale bars: A, C, E and G, 40 μm (physical size). B, D, F, and H, 8 μm (physical size).
Nature Biotechnology: doi:10.1038/nbt.3892
Supplementary Figure 8
Training and and test images for single-blinded nephrotic kidney disease diagnosis.
Frozen kidney sections fixed with acetone were stained with DAPI and antibodies. Blue, vimentin; green, actinin-4; magenta, collagen IV; grey, DAPI. Scale bars are indicated in images. For ExPath images, only physical size is indicated. Expansion factors are listed in Supplementary Table 4.
Nature Biotechnology: doi:10.1038/nbt.3892
Nature Biotechnology: doi:10.1038/nbt.3892
Supplementary Figure 9
Nature Biotechnology: doi:10.1038/nbt.3892 Examples of computational detection and segmentation of the nuclei in pre-expansion H&E images and post-expansion fluorescent images (ExPath).
Computational detection and segmentation of the nuclei is significantly more accurate in expanded samples as compared to pre- expanded samples: examples of normal breast, usual ductal hyperplasia (UDH), atypical ductal hyperplasia (ADH) and ductal carcinoma in situ (DCIS). For the “expert annotation” and “automated segmentation” columns: green filled nuclei are nuclei segmented by the expert or the automated segmentation algorithm, respectively (red circles indicate nucleus outlines, which are not visible in the ExPath row because the resolution is too high and thus the outline is barely visible). In the “automated vs expert” column: green filled nuclei, true positives; red filled nuclei, false negatives; blue filled nuclei, false positives (note that when the automated segmentation yielded larger outlines than the expert, this is expressed as a blue “halo” around the green).
Nature Biotechnology: doi:10.1038/nbt.3892 Expansion Pathology
Yongxin Zhao+1, Octavian Bucur+2,3,4,5, Humayun Irshad2,3,5, Fei Chen1,6,7, Astrid Weins8,9, Andreea L. Stancu2,5, Eun-Young Oh2, Marcello DiStasio2, Vanda Torous2, Benjamin Glass2, Isaac E. Stillman2, Stuart J. Schnitt2, Andrew H. Beck2,3,5*, Edward S. Boyden1,6,7,10*
1MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA 2Department of Pathology and Cancer Research Institute, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA 3Ludwig Center at Harvard Medical School, Boston, MA, USA 4Institute of Biochemistry of the Romanian Academy, Bucharest, Romania 5Broad Institute of MIT and Harvard, Cambridge, MA, USA 6Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA 7McGovern Institute, Massachusetts Institute of Technology, Cambridge, MA, USA 8Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA 9Department of Medicine/Renal Division, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA 10Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
+ made equal contributions. *correspondence to [email protected] (pathological aspects) and [email protected] (technical aspects)
Nature Biotechnology: doi:10.1038/nbt.3892 Supplementary information Supplementary Note Supplementary Table 1. The effects of EDTA concentration on proteinase K digestion of human tissue/hydrogel composites. Supplementary Table 2. The effects of EDTA concentration on proteinase K digestion of human tissue/hydrogel composites, as a function of digestion time. Supplementary Table 3. Expansion factors for samples in Figure 3. Supplementary Table 4. Information about images used in Supplementary Figure 8. Supplementary Table 5. Observations on samples in Supplementary Figure 8 made by observers. Supplementary Table 6. ExPath improves automated nucleus detection and segmentation in expanded breast neoplasia specimens. Supplementary Table 7. ExPath improves nucleus detection in all breast tissue types, as well as on clearly visible and dim nuclei. Supplementary Table 8. ExPath improves nucleus segmentation in all breast tissue types, as well as on clearly visible and dim nuclei. Supplementary Table 9. Binary-classification results (AUC ROC) with three different state of the art classifiers (Naïve Bayes, Random Forest and GLMNET). Supplementary Table 10. List of top-performing features for building classification models for ExPath. Supplementary Table 11. List of top-performing features for building classification models for pre- expansion data. Supplementary Table 12. Binary-classification results (AUC ROC; GLMNET) with or without using normalized expansion factors. Supplementary Table 13. Comparison of technical capabilities of the current version of ExPath vs. electron microscopy.
Supplementary Table 14. Human samples purchased from commercial sources.
Supplementary Table 15. Primary antibodies used.
Supplementary References.
Nature Biotechnology: doi:10.1038/nbt.3892 Supplementary Note The effect of EDTA concentration on tissue digestion. We note that high concentrations of EDTA (>5 mM) are often used for enzyme-free, live-cell tissue dissociation in preparation for flow cytometry1,2. For our fixed tissue experiments, we used 25 mM EDTA to facilitate more effective proteinase K (ProK) digestion. We examined the effects of EDTA at 1 mM vs. 25 mM concentration in the digestion of human samples including skin, liver, breast and lung, including acetone-fixed as well as FFPE samples (Supplementary Fig. 2; Supplementary Tables 1-2). We also investigated the effect of digestion time under both conditions for human skin and liver FFPE samples (which contain distinct extracellular matrix components and exhibit strong autofluorescence in the blue and green channels (Supplementary Fig. 2), most likely due to formalin-fixed extracellular matrix (ECM) proteins3–5). We found that both human skin and liver samples were completely digested and fully expanded in <0.5 hours with 25 mM EDTA-assisted proteinase K digestion. On the other hand, the ECM was not fully digested in either type of tissue with 1 mM EDTA as suggested by residual autofluorescence (Supplementary Figs. 2Aiii and Avi). Incomplete digestion can cause distortions at both micro- and macro- scales. For many other types of tissue, either low or high EDTA will work (Supplementary Tables 1-2). Our results suggest that 25 mM EDTA is preferred for the digestion step, and digestion time could be as short as 0.5 hours for 5 m- thick or thinner human FFPE tissue specimens.
Nature Biotechnology: doi:10.1038/nbt.3892 Supplementary Tables
Supplementary Table 1. The effects of EDTA concentration on proteinase K digestion of human tissue/hydrogel composites. Digestion* with 1 mM EDTA Digestion with 25 mM EDTA Fresh Frozen FFPE Fresh Frozen FFPE Skin Liver Lung Breast *Digestion condition: 8 units/mL proteinase K solution containing 25 mM Tris (pH 8), 0.25% Triton X-100, 0.4 M NaCl, in 60 °C for 3 hours.
Complete digestion.
Incomplete digestion.
Supplementary Table 2. The effects of EDTA concentration on proteinase K digestion of human tissue/hydrogel composites, as a function of digestion time. Human tissue EDTA Digestion time type concentration 0.5 h 1 h 1.5 h 2 h 1 mM Skin 25 mM 1 mM Liver 25 mM
Supplementary Table 3. Expansion factors for samples in Figure 3.
Human Size of yellow scale bar in biological units, in post-expansion images, in m (with organ expansion factor*) Normal Cancer Top Bottom Top Bottom (physical scalebar (physical scalebar (physical scalebar (physical scalebar size, 50 m) size, 12.5 m) size, 50 m) size, 12.5 m) Prostate 10.4 (4.8x) 2.6 (4.8x) 10.2 (4.9x) 2.5 (4.9x) Lung 10.4 (4.8x) 2.6 (4.8x) 11.11 (4.5x) 2.8 (4.5x) Breast 12.5 (4.0x) 3.1 (4.0x) 10.6 (4.7x) 2.7 (4.7x) Pancreas 10.4 (4.8x) 2.6 (4.8x) 10 (5.0x) 2.5 (5.0x)
Nature Biotechnology: doi:10.1038/nbt.3892 Ovary 10.2 (4.9x) 2.5 (4.9x) 10.6 (4.7x) 2.7 (4.7x) Liver 10.6 (4.7x) 2.7 (4.7x) 10.4 (4.8x) 2.6 (4.8x) Kidney 10.4 (4.8x) 2.6 (4.8x) 10.6 (4.7x) 2.7 (4.7x) Colon 10.6 (4.7x) 2.7 (4.7x) 10.4 (4.8x) 2.6 (4.8x) *Average expansion factor: 4.7 (standard deviation: 0.2).
Supplementary Table 4. Information about images used in Supplementary Figure 8.
Image ID (as used Case # State FPE? Expanded? Expansion factor in Supp. Fig. 8) IF01 1 Normal No No - IF02 2 Normal No No - IF03 3 Normal No No - IF04 4 MCD Yes No - IF05 5 MCD Yes No - IF06 6 MCD Yes No - IF07 7 FSGS Yes No - E01 3 Normal No Yes 4.3 E02 1 Normal No Yes 4.4 E03 2 Normal No Yes 4.2 E04 8 Normal No Yes 4.4 E05 6 MCD Yes Yes 4.2 E06 5 MCD Yes Yes 4.3 E07 9 MCD Yes Yes 4.5 E08 7 FSGS Yes Yes 3.8 A01 5 MCD Yes No - A02 3 Normal No No - A03 5 MCD Yes No - A04 2 Normal No No - A05 2 Normal No No - A06 6 MCD Yes No - A07 6 MCD Yes No - A08 7 FSGS Yes No - A09 8 Normal No No - A10 8 Normal No No - B01 8 Normal No Yes 4.4 B02 7 FSGS Yes Yes 3.8 B03 6 MCD Yes Yes 4.2 B04 2 Normal No Yes 4.2 B05 1 Normal No Yes 4.4 B06 9 MCD Yes Yes 4.5
Nature Biotechnology: doi:10.1038/nbt.3892 B07 5 MCD Yes Yes 4.3 B08 3 Normal No Yes 4.3 B09 5 MCD Yes Yes 4.3 B10 8 Normal No Yes 4.4
Abbreviation: FPE=foot process effacement; MCD=Minimal Change Disease; FSGS=Focal Segmental Glomerulosclerosis; Average expansion factor of B and E series: 4.3 (standard deviation 0.2).
Supplementary Table 5. Observation on samples in Supplementary Figure 8 made by observers. Image ID (as used FPE*? Expanded? Observation made by observers in Supp. Fig. 8) 1 2 3 4 5 6 7 Fraction correct Pre-Expansion A01 Yes No Yes Yes Yes Yes Yes No No 71.4% A02 No No No Yes No Yes Yes Yes Yes 28.6% A03 Yes No Yes No Yes No Yes No Yes 57.1% A04 No No Yes Yes No Yes Yes No No 42.9% A05 No No No No No No No No No 100.0% A06 Yes No No No No No No No Yes 14.3% A07 Yes No Yes Yes Yes No Yes Yes Yes 85.7% A08 Yes No Yes Yes Yes Yes Yes Yes Yes 100.0% A09 No No Yes No No Yes No Yes No 57.1% A10 No No No No No No No No No 100.0% Accuracy 70% 60% 90% 40% 70% 50% 80% 65.7% Post-expansion B01 No Yes No No No No No No No 100.0% B02 Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0% B03 Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0% B04 No Yes Yes Yes No Yes No No No 57.1% B05 No Yes No No No No No No No 100.0% B06 Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0% B07 Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0% B08 No Yes No No No No Yes No Yes 71.4% B09 Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0% B10 No Yes No No No Yes No No Yes 71.4% Accuracy 90% 90% 100% 80% 90% 100% 80% 90.0% *FPE: Foot process effacement.
*Observers were two kidney pathologists, one non-kidney pathologist, one pathology resident, two MDs and one non-MD researcher.
Nature Biotechnology: doi:10.1038/nbt.3892
Supplementary Table 6. ExPath improves automated nucleus detection and segmentation in expanded breast neoplasia specimens.
Nucleus Detection Nucleus Segmentation
Total TP FN FP TPR PPV F-Score F-Score Accuracy Kappa GCE
Pre-Exp 9416 5178 4238 1768 0.56 ±0.14 0.76 ±0.17 0.63 ±0.13 0.64 ±0.17 0.78 ±0.06 0.39 ±0.12 0.30 ±0.08
ExPath 8649 5202 3447 438 0.62 ±0.13 0.93 ±0.06 0.73 ±0.1 0.73 ±0.18 0.90 ±0.04 0.69 ±0.08 0.18 ±0.07
Abbreviations: TP=true positive; FN=false negative; FP=false positive; TPR=true positive rate; PPV = positive predictive value; Kappa=Cohen kappa coefficient; GCE=global consistency error6. For the latter scores, shown is mean ± standard deviation; N = 31 fields of view from 12 patients.
Supplementary Table 7. ExPath improves nucleus detection in all breast tissue types, as well as on clearly visible and dim nuclei.
Total TP FN FP TPR PPV F-Score Pre-Exp 9416 5178 4238 1768 0.56 ± 0.14 0.76 ± 0.17 0.63 ± 0.13 All Nuclei ExPath 8649 5202 3447 438 0.62 ± 0.13 0.93 ± 0.06 0.73 ± 0.1 Pre-Exp 5069 3262 1807 - 0.66 ± 0.18 - - Clearly Visible Nuclei ExPath 4925 3476 1449 - 0.74 ± 0.14 - - Pre-Exp 4347 1981 2366 - 0.44 ± 0.18 - - Dim Nuclei ExPath 3597 1899 1698 - 0.65 ± 0.09 - - Abbreviations: TP=true positive; FN=false negative; FP=false positive; TPR=true positive rate; PPV = positive predictive value6. For the latter scores, shown is mean ± standard deviation; N = 31 fields of view from 12 patients.
Supplementary Table 8. ExPath improves nucleus segmentation in all breast tissue types, as well as on clearly visible and dim nuclei.
Pair- Counting Overlap based Metrics Probabilistic Metrics based Metrics
F-Score Accuracy Overlap GCE RI Kappa AUC
All Nuclei Pre-Exp 0.64 ± 0.17 0.78 ± 0.06 0.49 ± 0.17 0.30 ± 0.08 0.67 ± 0.06 0.39 ± 0.12 0.69 ± 0.07
Nature Biotechnology: doi:10.1038/nbt.3892 ExPath 0.73 ± 0.18 0.90 ± 0.04 0.60 ± 0.21 0.18 ± 0.07 0.79 ± 0.07 0.69 ± 0.08 0.85 ± 0.04
Clearly Pre-Exp 0.65 ± 0.16 0.82 ± 0.04 0.50 ± 0.16 0.28 ± 0.06 0.71 ± 0.05 0.43 ± 0.14 0.71 ± 0.10 Visible Nuclei ExPath 0.74 ± 0.17 0.91 ± 0.05 0.61 ± 0.20 0.11 ± 0.07 0.82 ± 0.08 0.74 ± 0.16 0.86 ± 0.06
Pre-Exp 0.61 ± 0.19 0.77 ± 0.06 0.47 ± 0.18 0.32 ± 0.08 0.65 ± 0.07 0.35 ± 0.12 0.60 ± 0.07 Dim Nuclei ExPath 0.70 ± 0.19 0.82 ± 0.06 0.57 ± 0.21 0.25 ± 0.08 0.72 ± 0.08 0.58 ± 0.15 0.73 ± 0.08
Abbreviations: GCE=global consistency error; RI=rand index; Kappa=Cohen kappa coefficient; AUC=area under the curve (receiver operating characteristic curve, a plot of true positive rate against false positive rate)6. Mean ± standard deviation; N = 31 fields of view from 12 patients.
Supplementary Table 9. Binary-classification results (AUC ROC) with three different state of the art classifiers (Naïve Bayes, Random Forest and GLMNET). Naïve Bayes Random Forest GLMNET Combined Results Pre- ExPath Pre- ExPath Pre- ExPath Pre-Exp ExPath Exp Exp Exp Normal vs UDH 0.66 0.81 0.87 0.98 0.86 0.95 0.80 ± 0.10 0.91 ± 0.07 Normal vs ADH 0.84 0.74 0.78 0.89 0.82 0.96 0.81 ± 0.02 0.86 ± 0.09 Normal vs DCIS 0.77 0.74 0.84 0.87 0.75 0.94 0.79 ± 0.04 0.85 ± 0.08 UDH vs ADH 0.81 0.84 0.77 0.94 0.71 0.93 0.76 ± 0.04 0.90 ± 0.04 UDH vs DCIS 0.75 0.90 0.89 0.98 0.82 0.89 0.82 ± 0.06 0.92 ± 0.04 ADH vs DCIS 0.75 0.77 0.79 0.91 0.84 0.95 0.79 ± 0.04 0.88 ± 0.08 Abbreviations: UDH=Usual Ductal Hyperplasia; ADH=Atypical Ductal Hyperplasia; DCIS=Ductal Carcinoma In Situ; mean ± standard deviation; N=3 classification methods.
Supplementary Table 10. List of top-performing features for building classification models for ExPath.
Feature Name Summary Function Feature Classes Orientation Mean Morphology Orientation Median Morphology Eccentricity Skewness Morphology Equivalent Diameter Skewness Morphology Convex Area Kurtosis Morphology Minor Axis Length Skewness Morphology Solidity Kurtosis Morphology
Nature Biotechnology: doi:10.1038/nbt.3892 Solidity Interquartile Range Morphology Ellipse_Minor Interquartile Range Morphology Ellipse_Minor Mean Absolute Deviation Morphology Ellipse_Minor Skewness Morphology Minimum Kurtosis Intensity Minimum Interquartile Range Intensity Standard Deviation Skewness Intensity Median Kurtosis Intensity Mean Absolute Deviation Skewness Intensity Interquartile Range Skewness Intensity Interquartile Range Kurtosis Intensity Kurtosis Standard Deviation Intensity Kurtosis Kurtosis Intensity Kurtosis Interquartile Range Intensity Contrast Interquartile Range Texture Contrast Kurtosis Texture Contrast Mean Absolute Deviation Texture Correlation Kurtosis Texture Autocorrelation Kurtosis Texture Autocorrelation Skewness Texture Information measure Correlation 2 Kurtosis Texture Cluster Prominence Mean Absolute Deviation Texture Dissimilarity Skewness Texture Inverse Difference Moment Interquartile Range Texture LRE Skewness Texture GLN Skewness Texture RLN Skewness Texture RP Skewness Texture HGRE Kurtosis Texture LRHGE Skewness Texture LRHGE Mean Absolute Deviation Texture SRLGE Kurtosis Texture SRLGE Skewness Texture SRHGE Kurtosis Texture
Supplementary Table 11. List of top-performing features for building classification models for pre- expansion data.
Feature Name Summary Function Feature Classes Ellipse Mean Morphology
Nature Biotechnology: doi:10.1038/nbt.3892 Solidity Mean Morphology Orientation Standard Deviation Morphology Maximum Standard Deviation Intensity Maximum Mean Intensity Minimum Standard Deviation Intensity Minimum Mean Intensity Mean Standard Deviation Intensity Standard Deviation Standard Deviation Intensity Standard Deviation Mean Intensity Median Standard Deviation Intensity Mean Absolute Deviation Standard Deviation Intensity Interquartile Range Mean Intensity Skewness Mean Intensity Skewness Standard Deviation Intensity Kurtosis Mean Intensity Kurtosis Standard Deviation Intensity SRE Mean Texture SRE Standard Deviation Texture RLN Mean Texture RP Mean Texture RP Standard Deviation Texture LGRE Mean Texture LGRE Standard Deviation Texture LRLGE Standard Deviation Texture HGRE Mean Texture SRLGE Mean Texture SRHGE Standard Deviation Texture Autocorrelation Mean Texture Autocorrelation Standard Deviation Texture Correlation Standard Deviation Texture Correlation Mean Texture Information measure Correlation 1 Mean Texture Information measure Correlation 2 Mean Texture Cluster Prominence Standard Deviation Texture Cluster Prominence Mean Texture Cluster Shade Standard Deviation Texture Compactness Mean Texture Compactness Standard Deviation Texture Dissimilarity Standard Deviation Texture Dissimilarity Mean Texture
Nature Biotechnology: doi:10.1038/nbt.3892 Homogeneity Standard Deviation Texture Inverse Difference Moment Mean Texture
Supplementary Table 12. Binary-classification results (AUC ROC; GLMNET) with or without using normalized expansion factors.
ExPath without Normalization by the ExPath with Normalization by the Expansion Expansion Factor Factor Normal vs UDH 0.95 0.95 Normal vs ADH 0.96 0.96 Normal vs DCIS 0.92 0.94 UDH vs ADH 0.66 0.93 UDH vs DCIS 0.96 0.89 ADH vs DCIS 0.97 0.95 Overall, we see similar performance with or without normalization by the expansion factor, with the exception of UDH versus ADH, where the performance is significantly improved with normalization.
Supplementary Table 13. Comparison of technical capabilities of the current version of ExPath vs. electron microscopy. ExPath, current version Electron Microscopy Resolution ~70 nm ~0.2 nm Multiplexity Up to 5 colors Only 1 color Tissue processing Total: 14.5 hours (Fresh frozen samples) Total: 59 hours time
Fixation: 0.3 hour Aldehyde fixation: 1 hour
Buffer rinse: 0.5 hour Buffer rinse: 0.5 hours
Heat treatment: 0.5 hour Ethanol dehydration: 1.7 hours
Primary antibody staining: 3 hours Proplylene oxide treatment: 0.7 hours
Buffer rinse: 0.5 hour Infiltration: 30 hours
Secondary antibody staining: 2 hours Polymerization: 24 hours
Buffer rinse: 0.5 hour
Nature Biotechnology: doi:10.1038/nbt.3892 AcX incubation: 3 hours
In situ polymerization: 2.5 hours
Proteinase K digestion: 1 hour
Buffer rinse and DAPI stain: 0.2 hours
Expansion: 0.5 hours
Tissue area that can High Low be imaged Skill requirement Low High Equipment Conventional microscope Ultramicrotome and electron requirement microscope
Supplementary Table 14. Human samples purchased from commercial sources. Sample Manufacturer Catalog No. Figure (Main Text) IHC control tissue array for HER2 molecule Abcam ab178176 Figures 1L and 1M Human adult normal: lung (FFPE) US Biomax HuFPT131 Figure 2A-J Multi-tumor tissue array, 95 cases of 40 types from Abcam ab178234 Figure 3 27 organs/sites (1.5 mm each) Human adult normal: kidney (fresh frozen slides) US Biomax HuFTS241 Figures 4A and 4B. Breast hyperplasia tissue array, 80 cases/80 cores US Biomax BR806 Figure 5 Breast pre-cancerous disease and cancer tissue array US Biomax BR1003 Figure 5 (100 cases/101 cores) Breast common disease tissue array of 102 cases Abcam ab178113 Figure 5 (1.5mm each) Breast cancer tissue array with progressive changes, Abcam Ab178112 Figure 5 48 cases, 96 samples (1.5mm each) Additional non-commercial whole-slide tissue sections and tissue microarrays (TMAs) prepared in our laboratories were used.
Supplementary Table 15. Primary antibodies used. Target (reference in superscript) Host Clonality Manufacturer Catalog No. TOM207 rabbit poly Santa Cruz Biotech sc-11415 Collagen IV8 mouse mono Santa Cruz Biotech sc-59814
Nature Biotechnology: doi:10.1038/nbt.3892 Vimentin9 chicken poly Abcam ab24525 -Tubulin10 rabbit poly Abcam ab15246 VDAC/Porin11 mouse mono Abcam ab14734 KRT1912 rabbit poly Sigma Aldrich HPA002465 ACTN413 rabbit poly Sigma Aldrich HPA001873 Synaptopodin14 guinea pig poly PROGEN Biotechnik GP94-IN Actin (Smooth Muscle)15/ACTA2 mouse mono Agilent Technologies M085129-2
Supplementary References 1. den Haan, J. M., Lehar, S. M. & Bevan, M. J. CD8(+) but not CD8(-) dendritic cells cross-prime cytotoxic T cells in vivo. J. Exp. Med. 192, 1685–96 (2000). 2. den Haan, J. M. M. & Bevan, M. J. Constitutive versus activation-dependent cross-presentation of immune complexes by CD8(+) and CD8(-) dendritic cells in vivo. J. Exp. Med. 196, 817–27 (2002). 3. Wu, Y. & Qu, J. Y. Autofluorescence spectroscopy of epithelial tissues. J. Biomed. Opt. 11, 54023 (2006). 4. Banerjee, B., Miedema, B. E. & Chandrasekhar, H. R. Role of basement membrane collagen and elastin in the autofluorescence spectra of the colon. J. Investig. Med. 47, 326–32 (1999). 5. Hansch, A. et al. Autofluorescence spectroscopy in whole organs with a mobile detector system. Acad. Radiol. 11, 1229–1236 (2004). 6. Taha, A. A. & Hanbury, A. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med. Imaging 15, 29 (2015). 7. Maxfield, K. E. et al. Comprehensive functional characterization of cancer-testis antigens defines obligate participation in multiple hallmarks of cancer. Nat. Commun. 6, 8840 (2015). 8. Knutsson, J., Bagger-Sjöbäck, D. & von Unge, M. Structural tympanic membrane changes in secretory otitis media and cholesteatoma. Otol. Neurotol. 32, 596–601 (2011). 9. Ware, C. B. et al. Derivation of naive human embryonic stem cells. Proc. Natl. Acad. Sci. U. S. A. 111, 4484–9 (2014). 10. Reincke, M. et al. Mutations in the deubiquitinase gene USP8 cause Cushing’s disease. Nat. Genet. 47, 31–38 (2014). 11. Liu, T. F. et al. Sequential actions of SIRT1-RELB-SIRT3 coordinate nuclear-mitochondrial communication during immunometabolic adaptation to acute inflammation and sepsis. J. Biol. Chem. 290, 396–408 (2015). 12. Cheng, J. et al. Tryptophan derivatives regulate the transcription of Oct4 in stem-like cancer cells. Nat. Commun. 6, 7209 (2015). 13. Piskareva, O. et al. The development of cisplatin resistance in neuroblastoma is accompanied by epithelial to mesenchymal transition in vitro. Cancer Lett. 364, 142–155 (2015).
Nature Biotechnology: doi:10.1038/nbt.3892 14. Zhang, X. et al. Essential role for synaptopodin in dendritic spine plasticity of the developing hippocampus. J. Neurosci. 33, 12510–8 (2013). 15. Rizeq, M. N., Van De Rijn, M., Hendrickson, M. R. & Rouse, R. V. A comparative immunohistochemical study of uterine smooth muscle neoplasms with emphasis on the epithelioid variant. Hum. Pathol. 25, 671–677 (1994).
Nature Biotechnology: doi:10.1038/nbt.3892