Precision medicine in nephrology
Digital pathology and computational image analysis in nephropathology
Laura Barisoni 1,2 ✉ , Kyle J. Lafata3,4, Stephen M. Hewitt5, Anant Madabhushi6,7 and Ulysses G. J. Balis 8 Abstract | The emergence of digital pathology — an image-based environment for the acquisition, management and interpretation of pathology information supported by computational techniques for data extraction and analysis — is changing the pathology ecosystem. In particular, by virtue of our new-found ability to generate and curate digital libraries, the field of machine vision can now be effectively applied to histopathological subject matter by individuals who do not have deep expertise in machine vision techniques. Although these novel approaches have already advanced the detection, classification, and prognostication of diseases in the fields of radiology and oncology, renal pathology is just entering the digital era, with the establishment of consortia and digital pathology repositories for the collection, analysis and integration of pathology data with other domains. The development of machine-learning approaches for the extraction of information from image data, allows for tissue interrogation in a way that was not previously possible. The application of these novel tools are placing pathology centre stage in the process of defining new, integrated, biologically and clinically homogeneous disease categories, to identify patients at risk of progression, and shifting current paradigms for the treatment and prevention of kidney diseases.
1Department of Pathology, The advent of whole slide digital imaging systems has Nephropathology in the digital era Duke University, Durham, catalysed new interactive models for pathology edu- Only a decade ago, virtual microscopy was considered to NC, USA. cation, expedited clinical workflows and provided be a fairly novel technology and of insufficient maturity 2Department of Medicine, opportunities for the development of new medically for primary diagnostic purposes. Although the technol- Division of Nephrology, Duke University, Durham, NC, USA. actionable image analysis tools and investigative ogy was rapidly embraced for educational uses, its role techniques. Although still evolving, this technology in clinical practice was less clear given the preference of 3Department of Radiology, Duke University, Durham, has provided the clinical and research community conventionally trained pathologists to work with con- NC, USA. with new solutions that will need to be integrated with ventional light microscopy. This status quo has been 4Department of Radiation established models for contemporary care. Ultimately, shattered by the emergence of rapidly evolving tech- Oncology, Duke University, we anticipate that a new system of care will emerge, nologies such as telepathology and digital pathology, Durham, NC, USA. whereby these innovative technologies are integrated in combination with developments in computational 5Laboratory of Pathology, into routine practice in a rational and fully validated image analysis. The collective subspecialties of pathol- Center for Cancer Research, manner. ogy are now witnessing a far-ranging and transform National Cancer Institute, NIH, Bethesda, MD, USA. In this Review, we discuss how developments in ative set of new practice models, with implications for digital pathology and computational image analysis are both intramural and remote primary and consultative 6Department of Biomedical Engineering, Case Western shaping a new digital era for nephropathology and how diagnosis. Similarly, delivery models for education Reserve University, these technologies might be most expeditiously consid- and companion diagnostics are positively affected by Cleveland, OH, USA. ered for clinical deployment. We provide an overview the emergence of these technologies. In particular, 7Louis Stokes Veterans of the current status of the digital ecosystem in nephro- content-based image retrieval1 — a machine vision tech- Administration Medical pathology, which encompasses human expertise in nique that searches images by analysing their con- Center, Cleveland, OH, USA. renal pathology, artificial intelligence (AI), omics science tent (according to their colour, shape and/or texture) 8 Department of Pathology, and powerful computational tools, as well as potential rather than their associated metadata — is now a real- University of Michigan, Ann Arbor, MI, USA. applications and challenges associated with the emerg- ity that opens fascinating new possibilities to leverage ✉e-mail: laura.barisoni@ ing armamentarium of technologies for image analysis. the vast and under-utilized slide and image archives duke.edu Finally, we present what might be a prescient vision of long held by pathology departments. The main uses https://doi.org/10.1038/ the imminent future of clinical research and practice of digital imaging in nephropathology can be divided s41581-020-0321-6 made possible by these technologies. into three major operational models: telepathology,
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 669 reVieWs
Key points pathology practices and laboratories, particularly in Europe and Canada, are now in the early or mid-phases • The introduction of digital pathology in clinical research, trials and practice has of implementing digital pathology solutions for pur- catalysed the development of novel machine-learning models for tissue interrogation poses other than image archiving or for second-opinion with the potential to improve our ability to discover disease mechanisms, identify consultation23,24. WSIs, which can now be scanned in less comprehensive, patient-specific phenotypes, classify kidney patients into clinically relevant categories, predict disease outcome and, ultimately, identify more targeted than a minute, can serve as an effective surrogate for 20,25 therapies. traditional microscopy-based pathology . However, • The development of computational image analysis tools for tissue interrogation has WSI workflow models have known limitations including brought pathology to the forefront in this process of re-defining kidney diseases. the need for substantial network bandwidth to handle Z-dimensional focus • The new nephropathology ecosystem offers several advantages over conventional large file sizes, the lack of (except in pathology but also brings some challenges that need to be addressed collectively by specialized cases, where a ‘Z-Stack’ image might be gen- all the stake holders, including pathologists, nephrologists, computer scientists, erated for a region of interest (ROI) from the larger total regulatory agencies and patient’s representatives; overcoming these challenges is a slide area), which limits their utility for cytopathologi- pre-requisite for these new machine-learning and computational pathology models cal and renal biopsy evaluation, and imperfect control to be fully deployed for patient care. software in the scanning appliances themselves, which • The development of novel powerful computational tools for image analysis and data can contribute to various imaging artefacts (for exam- integration in kidney diseases has exposed the need to revise the curriculum for ple, out-of-focal plane dust on the glass slide triggering medical professionals to prepare the next generation to fully operate within the new acquisition of the wrong focal plane)26. digital pathology ecosystem. These limitations support the subjective perception • Ultimately, our ability to treat kidney diseases (actionable intelligence) will be largely held by many pathologists that digital pathology can based on the application of artificial (augmenting) intelligence tools and the be less accurate and more time consuming than glass establishment of synergistic human–machine protocols that integrate digital slide microscopy for diagnosis, although direct quanti- pathology data with clinical and molecular data for personalized nephrology. tative comparisons show the opposite. Processes might indeed be slower when digital pathology workflows are digital pathology and computational image analysis, as first adopted; however, following an initial training and outlined below. acclimatization period, overall pathologist performance greatly increases for measurements of both speed and Telepathology accuracy. As a result, perceptions are changing, with Telepathology involves the transmission of one or more digital pathology becoming increasingly accepted, microscopic images from an originating location to an helping the pathology specialty move towards fully imaging workstation at a distant location. Its use has digital workflows and associated platforms for clinical been extensively validated and it is now a common operations26–29. Computational image analysis tool for real-time assessment of tissue adequacy and 2,3 The application of artificial diagnosis . In nephropathology, dynamic telepathology Computational image analysis intelligence for the extraction has been deployed for the remote assessment of donor Advances in scanning technology paired with the of meaningful information implant biopsy samples by trained renal pathologists. increasing availability of large datasets of digital images (output) from the digital images (the input). This approach overcomes some of the limitations of cur- have enabled pathologists to collaborate with technology rent practice, whereby cadaveric donor kidneys are often experts, including data scientists, computational engi- Artificial intelligence harvested and evaluated after-hours, in non-academic neers and imaging physicists, to explore the potential (AI). A branch of computer hospitals and by general surgical pathologists4,5. of a newly established branch in the field of pathology: science that deals with the Telepathology has also enabled renal pathologists to machine vision, a component of computational image ability of computers to mimic human intelligence or cognitive assess the quality of fresh tissue biopsy at the time of analysis. The primary objective of this field is to extract functions. the biopsy procedure without needing to be physically and/or generate quantitative data from digital image present in the interventional radiology suite. Its use subject matter, either in isolation or in tandem with Virtual microscopy gives the subject matter expert (rather than the available other classes of spatially or non-spatially based biolog- The use of microscopy and dimensionality digital technology to capture non-renal pathologist) the ability to triage the kidney ical or omics data. Recognizing the high- an image from a stained tissue biopsy samples for subsequent histology, immuno of the data generated by computational image analysis, section and transmit it in real fluorescence and electron microscopy analysis, without AI and machine-learning (ML)30–33 techniques, which are time as a static image (digital significant disruption of their workflow1. ideally positioned to interrogate such complex datasets pathology) or live image in an exhaustive manner, have been deployed to extract (telepathology) over computer Digital pathology networks, so that the viewer features, patterns and information from histopatholog- can access the image on a Digital pathology is a general term that refers to the ical subject matter that cannot otherwise be analysed remote computer. assemblage of digital workflow and imaging solutions by human-based image interrogation alone. Deep learning that are geared towards creating a digital image-based (DL) — also known as deep structured learning — is a Companion diagnostics whole slide image Generally, indicates a test practice environment in which a (WSI) particularly important branch of ML that provides developed based on a or other digital image is acquired, managed, interpreted opportunities to interrogate images at greater depths than companion biomarker and and searched for specific content. This new approach previously possible30,31,34,35. In pathology, DL has been used as a companion to a is being increasingly adopted in clinical trials6, for edu- used for the detection, annotation, segmentation, registra- medical intervention. cation purposes7–9 and for clinical research1,10–16. Since tion, processing and classification of WSIs. The learning Companion diagnostics can identify patients who are most 2017, the FDA has approved the use of commercial process uses neural networks mimicking the human likely to benefit from a WSI platforms, facilitating the use of digital pathology intelligence and contains multiple layers to progres- therapeutic approach. as a tool for primary diagnosis17–22. In response, many sively extract deep features from the input image, first
670 | November 2020 | volume 16 www.nature.com/nrneph Precision medicine in nephrology
(Table 1) Content-based image transcribing imaging information into structured data, diseases . The common goals of these consor- 10,11,13,44–52 retrieval and then translating them into knowledge. tia are to better understand the pathogenesis Tissue interrogation using AI models, including ML and DL, can be used for of kidney diseases and to improve their classification machine vision technology to various purposes, including the automatic detection, and treatment through comprehensive analysis of clin- search an image dataset by segmentation and quantification of histological param ical, morphological and molecular data — a process analysing information contained and derived from the images of eters and structural changes, and for disease diagnosis. that has required the establishment of digital pathology a dataset. In the context of These tasks are considered to be ‘low-level’ tasks. This is repositories (DPRs) for the banking and organization of digital pathology, the content in contrast to ‘high-level tasks’, in which AI models are digital images from renal biopsies and their associated refers to the colours, shape and used to concurrently interrogate and integrate multiple metadata. texture of the image. classes of primary data — for example, histopathological The establishment of DPRs has driven the concept Machine vision image data spatially coupled to transcriptomic data — of a digital biobank and a convergence of research Settings in which image-based providing opportunities for the prediction of processes on digital rather than physical assets. Conceptually, computational algorithms and such as disease aggression (for example, the biological the formation of a digital biobank is the same as processing pipelines are applied potential of a malignancy), patient outcome, organ forming other types of biobank, starting with rigor- to provide automation to tasks that would otherwise represent engraftment survival and therapeutic response. Thus, ous training of personnel involved in the collection, substantial cognitive tasks for use of AI has the potential to elevate digital images from de-identification and transfer of the pathology material. human observers alone. their basic use as a tool for visual assessment of disease This part of the process requires adherence to quality status to a much more complex and comprehensive role control (QC) steps to protect the privacy of individu- Dynamic telepathology A process whereby a live video as a tool to facilitate prediction of disease trajectory. als and their health information in accordance with the image is viewed with the These new capabilities enable the extraction of useful Health Insurance Portability and Accountability Act, to ensure assistance of an operator at information in a way that was not previously possible study protocol compliance and that the renal biopsy pathol- the site from which the image with the use of conventional microscopy in the context ogy materials are adequate, complete, organized in the is transmitted, or via a robotic of direct human cognitive and visual assessment. appropriate patient and disease category-designated space, mode, where the viewer 11,14 remotely controls the The increased demand for predictive assays for and properly identified (labelled) for searchability . transmission of the image patient profiling is driven not only by the need to better DPRs are created to facilitate shareable data access; thus, at the originating site. stratify patients according to their clinical characteris- considerations are needed at the consortium and regula- tics and outcomes but also by the evolving discovery tory agency level to facilitate secure, seamless and timely Whole slide image (WSI). High-resolution replica of and development of drugs that target specific molecular access for the end user. 36 a glass slide created by a slide pathways . Novel approaches to aid in the understand- scanner that can be viewed on ing of the relationships between structural changes in The digital nephropathology ecosystem a computer screen (a virtual tissue and clinical and/or molecular phenotypes would Digital pathology applications are progressively modify- slide). be particularly beneficial to the nephrology community, ing the ecosystem in which pathologists operate, offering Z-dimensional focus as current classification systems are inadequate in their considerable advantages over traditional pathology The coordinate axis representing ability to capture the heterogeneity of kidney diseases, approaches, as discussed below. However, these appli- the depth of an image. or predict disease outcomes. New, robust algorithms cations also present challenges that require considera- derived from the application of AI approaches and inte- tion of factors such as the redistribution of resources, Dimensionality The number of data attributes gration of pathology data with other relevant datasets standardization and regulation of processes, approaches that make up a dataset; the will have utility for both pathologists and nephrologists, to standardizing and integrating data, the consequences number of degrees-of- with the potential for a meaningful impact on patient of these new approaches for diagnostic paradigms and freedom (i.e. features) in a care and precision medicine. effects on the medical curriculum. machine-learning model. Many parallels can be drawn between machine vision Investment in digital pathology Machine-learning in pathology and developments in radiography, which (ML). A branch of artificial underwent its own digital revolution in the late twenti- DPRs are a cost-efficient way of systematically organ- intelligence that builds eth century with the development of native digital radio izing pathology resources. The investment, although mathematical models based on graphic modalities such as MRI and CT, which allowed initially burdensome, is long term and allows for the training data in order to make predictions or decisions without interrogation of digital anatomically coupled subject mat- establishment of a permanent library of sharable images. being explicitly programmed to ter for the first time. Additional innovations in radiology, In translational research, the return of such invest- perform the task. such as cassette-based storage phosphor image plates ment resides in the nature of the digital library itself, (which allowed X-ray images to be recorded digitally37) which can be accessed by multiple investigators, thereby Image interrogation and the implementation of charge-coupled devices, facil- supporting and facilitating a variety of pathology-based The application of a variety of 38–41 machine vision technologies to itated the rapid development of digital detectors and applications and related new discoveries within and 15,16,53–56 extract quantitative information further accelerated the adoption of digital technologies. across populations . Additionally, DPRs allow bar- from whole slide images. Similar to the current interplay between digital and com- riers that were previously not addressable to be overcome: putational image analysis, the mainstream availability they provide transparency for regulatory agencies and Deep learning A class of machine-learning of digital radiographic images has enabled quantitative enable collaboration both within and between consor- algorithms that has networks imaging analysis techniques to be tested and introduced tia. The reproducibility of observations is also improved, capable of unsupervised for use in clinical trials and for patient monitoring42,43. facilitated by digital annotation11,12,15,16 and computer- learning from data that are aided quantitative assessment, which offers a numerical unstructured or unlabelled. Digital pathology repositories value to the observations and measurements of param- It uses multiple layers to progressively extract higher The nephrology community has witnessed the for- eters (i.e. density, spatial distribution and relationships level features from the input mation of several international consortia that collect of histological primitives) that cannot be achieved by (in this case, the image). biospecimens from patients with a variety of kidney conventional analysis1. Thus, with the increasing role of
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 671 reVieWs
Table 1 | Multicentre digital pathology repositories for kidney disease Name Location Data collection Target diseases DPR (year Consortium website centre/central hub established) NEPhrotic syndrome North University of Michigan, MCD; FSGS; MN 2010 https://neptune-study.org/ sTUdy NEtwork (NEPTUNE) America MI, USA NEPhrotic syndrome China National Clinical MCD; FSGS; MN 2013 NA sTUdy NEtwork – CHINA Research Center of (NEPTUNE-CHINA) Kidney Diseases; Nanjing University, China Cure North Arbor Research, MCD; FSGS; MN, IgAN 2015 https://curegn.org/ GlomeruloNephropathies America; Michigan, MI, USA (CureGN) Europe (Poland, Italy) EURenOmics Europe, University of Heidelberg, Paediatric kidney diseases 2015 https://www.eurenomics.eu/ North Germany America Transformative Research North University of Diabetic kidney disease 2016 https://www.med.upenn. In DiabEtic NephropaThy America Pennsylvania, PA, USA edu/trident/ (TRIDENT) Biomarker Enterprise to Europe Lund University, Sweden Diabetic kidney disease 2018 https://www.beat-dkd.eu/ Attack Diabetic Kidney Disease (BEAt-DKD) European Rare Kidney Europe University of Heidelberg, Glomerulopathies; thrombotic 2018 https://www.erknet.org/ Disease Network Germany microangiopathies; renal and (ERK-Net) urinary tract malformations; tubulopathies; metabolic nephropathies; familial cystic diseases; any rare kidney disease; paediatric transplantation German Focal Segmental Europe University of Cologne, MCD; FSGS 2019 https://clinicaltrials.gov/ Glomerulosclerosis and Germany ct2/show/NCT03949972 Minimal Change Disease Registry (FOrME) Kidney Precision Medicine North University of Washington, Chronic kidney disease; acute 2019 https://kpmp.org/ Project (KPMP) America WA, USA; University kidney injury of Michigan, MI, USA; Mount Sinai School of Medicine, NY, USA Japan Renal Biopsy Japan Niigata University Glomerulopathies; 2019 https://pubmed.ncbi.nlm. Registry (J-RBR) tubulopathies; transplantation nih.gov/21437579/ Human Heredity and Africa University of Michigan Non-communicable 2020 https://h3africa.org/ Health in Africa (H3 Africa) (for the DPR), MI, USA disorders (e.g. heart and kidney disease), as well as communicable diseases (e.g. tuberculosis) Acceleration of Medicine North University of Michigan Lupus nephritis 2020 https://amp-ralupus. Partnership in RA/Systemic America (for the DPR), MI, USA stanford.edu/ Lupus Erythematosus (AMP RA/SLE) APLO1 Long-term Kidney North Wake Forest, NC, USA Kidney transplantation in the Under https://clinicaltrials.gov/ Transplantation Outcomes America African American population consideration ct2/show/NCT03615235 Network (APOLLO) This table lists major, multicentre studies that are known to the authors, but it is not exhaustive. DPR, digital pathology repository; FSGS, focal segmental glomerulosclerosis; IgAN, IgA nephropathy; MCD, minimal change disease; MN, membranous nephropathy; NA, not available.
big data and the ability to integrate separate knowledge approval of whole slide scanners for primary diagnosis domains (pathology, clinical and molecular domains) and tools for the integration of image management software to generate new diagnostic categories and predictors with laboratory information systems, have posi- of outcome, the launching of digital platforms for the tioned digital pathology to be deployed for clinical collection and analysis of tissue structural changes has applications17,19,28. Studies have shown that the implemen- become a necessary investment for clinical research. tation of digital pathology in clinical settings decreases Improvements in image resolution and digital scanner overall operational costs while increasing clinical and capacity, as well as the development and regulatory operational efficiency28. Most recently, the COVID-19
672 | November 2020 | volume 16 www.nature.com/nrneph Precision medicine in nephrology
Segmentation pandemic has forced the practising pathology com- Standardization of analytics One of the most important munity to explore approaches to enabling remote his- The deployment of digital pathology into research, clin- tasks of machine vision. topathological analysis on a large scale. In the USA, a ical trials, and ultimately clinical practice depends on a Manual segmentation critical step in facilitating this move was an amend- variety of elements, from the standardization of analy (annotation) is performed by subject matter experts, who ment to the 1988 Clinical Laboratory Improvement Act tics to allow the sharing of images to the application of use digital annotation tools to (CLIA ’88), which relaxed the location-specific licens- image analysis tools (Table 2). The process of transform- delineate the boundaries of the ing requirements that regulate where a pathologist is ing glass slides into WSIs involves a series of steps, each object (region of interest) or allowed to diagnose cases. This change enabled a num- contributing to the final quality of the digital image that histological primitive. ber of pathology practices and health networks to rapidly is visualized on a computer monitor. Protocols for the Automatic segmentation is performed by the deep employ remote diagnosis workflow models, allowed the processing of tissue into histology sections and the scan- learning model trained to field of pathology to gain greater proficiency in render- ning and transmission of image-associated information detect the boundaries of ing primary diagnoses by digital means and facilitated need to be standardized so that the digital pathology the object. the accelerated adoption of modality-specific workflow applications can augment the work of the pathologist Prediction models and digital consultative review. These develop- on a large scale. Such processes demand a revision of The result of a machine- ments all bode well for the prospect of nephropathology traditional criteria for QC and standardization of ana- learning model when operating benefiting from the expanded use of WSI solutions for lytics beyond current recommendations and regulations on a new data point based on expedited primary diagnosis. that apply to conventional histology approaches, as pro- historical data samples. However, the potential benefit of a fully digital path vided by the College of American Pathologists, CLIA 57 Health Insurance Portability ology operation on clinical applications in nephro ‘88, the Digital Pathology Association and the Renal and Accountability Act pathology extends beyond operational aspects alone. Pathology Society (RPS). In 2018, The Royal College of An Act passed by US Congress Integration of knowledge generated by these new com- Pathologists in the UK published best-practice recom in 1996 to reduce health-care putational tools can lead to new discoveries with the mendations for implementing digital pathology, fraud and abuse, enable transfer of insurance coverage potential to introduce new paradigms in clinical practice. highlighting the diagnostic challenges of WSIs and when changing jobs, standardize health-care | information on electronic Table 2 Current limitations and proposed solutions for computer-based image analysis billing and other formats, and Issue Problem Proposed solution protect confidentiality of health information. Standardization of tissue Variability of harvesting and Revision of histology protocols across centres to analytics tissue processing optimize downstream quantitative analysis Computer-aided Involvement of subspecialty societies quantitative assessment Use of a computer output to Standardization of imaging Scanner variability and Implementation of standard quality assurance guide clinical decision-making analytics image inconsistencies and calibration procedures (e.g. image linearity, and interpretation of clinical uniformity, reproducibility) data. Implementation of DICOM implementation in pathology Histological primitives Discrete normal or abnormal Evaluation of scanner variability across manufacturers structures (e.g. glomerular Involvement of subspecialty societies tufts, globally sclerotic glomeruli, tubules, atrophic Selection of the best approach Quantitative techniques are Parallel training, validation and testing of models and tubules and vessels) or cells for each task or problem specific to the application, techniques region of interest, or the (e.g. lymphocytes, podocytes Prospective evaluation (trials) to assess the clinical clinical question and proximal tubular cells). relevance of the application
Digital scanner Studying rare kidney diseases Inherently small datasets Knowledge-based data science A device that optically scans Data augmentation images or glass slides and converts them into digital Inter-institutional consortia for data sharing images or whole slide images. Data integration Data extraction, Standardization of data collection and storage representation, and fusion Ontology development Image management across different length software scales Development of new data fusion algorithms Software that allows the acquisition, organization and Knowledge integration Lack of didactic training Revision of curricula for health-care providers (e.g. viewing of pathology image across disciplines medical students, residents, fellows) and health data data and its associated scientists metadata. Interdisciplinary nephrology boards Deployment and efficacy in Lack of comprehensive Comprehensive prospective evaluation in clinical trials clinical practice prospective evaluation Development of new regulatory guidelines Data-sharing ethics Lack of regulatory Revision of IRB oversight guidelines Revision of Data and Material Transfer and/or Access Agreements Involvement of subspecialty societies and the medical ethics community DICOM, Digital Imaging and Communications in Medicine; IRB, Institutional Review Board.
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 673 reVieWs
Kidney Cutting Glass slides Curation of Curated Tissue Human assessment biopsy and scanned the image image Integration Reporting processing procurement staining into WSI dataset library Computational image analysis
Procurement step Histology steps Digital steps Data extraction steps Data interpretation steps
Pre-analytical phase Analytical phase Post-analytical phase
Fig. 1 | The analytical phases of digital pathology. The pre-analytical phase involves data extraction, analysis and interpretation of results. Data phase includes steps involved in tissue procurement, processing and extraction is accomplished using human visual analysis and/or machine fixation. The analytical phase involves a histology phase (which includes vision, using human–machine synergistic protocols. Data extracted from the selection of the stain to be used, optimization and validation of the staining images using human and machine vision are integrated and reported. Data procedure) and a digital phase (which includes scanning the slides as whole integration can be achieved using computational tools as well as human slide images (WSIs) and curation of the digital library). The post-analytical intuition and domain expertise.
providing practical solutions for reducing the risk of High-quality datasets can be achieved by proactively discordant diagnoses58,59. In the past few years, the gaining tighter control over analytical practices within College of American Pathologists has also provided and across laboratories, and by re-modelling current cri- guidelines for validating digital pathology approa teria for the minimum QC threshold based on the needs ches60, and the National Society for Histotechnology of current technologies and their applications61. As the has initiated a programme to improve the quality of use of image-based data by AI technologies requires WSIs60,61. However, we and others recommend that all the conversion of analogue data into numerical values stake-holders participate in regulating and monitoring (that is, digitization), understanding of the intimate asso- the use and performance of digital pathology-based ciation between stain quality and image quality becomes protocols62. essential. Similar to the field of radiology, where imaging As outlined below, standardization of pathology ana- physicists are employed to ensure QC of clinical images lytics requires consideration of three major components and imaging systems, standardization and quality assur- (Fig. 1): a pre-analytical phase, which includes the steps ance at both the tissue staining phase and the image from tissue procurement to fixation, processing, cutting acquisition phase will likely require on-site technical and drying time; an analytical phase, which includes a expertise. This support will ensure patient safety and histology phase (involving stain selection, optimization reliable diagnostics. We expect that this area of optics and validation) and a digital phase (scanning, consid- and light microscopy will be an important focus of eration of image and monitor resolution, number of discovery and standardization in the coming years. colours, colour distribution, compression ratio and One relevant issue that requires better understanding image format63); and a post-analytical phase, which before computational tools are deployed for image analy involves the analysis and interpretation of results, the sis in the clinical setting is how variation in image recording and reporting of data, and pathologist and ML quality caused by the physical abstraction layer of the performance metrics1,10,12,16,64–66 (Fig. 1). WSI scanning platform itself affects the performance of AI approaches. Different commercial WSI solutions Standardization of the pre-analytical and analytical digitally reproduce colours based on a reference colour phases. The current criteria for ensuring QC of the source slide with uneven variability. This variability can histology preparations allow for broad variation of pre- extend to inter-device variability, even within a single analytical and analytical artefacts across and within model of a WSI scanner. Although colour correction Generalizable laboratories. The spectrum of artefacts varies in severity software can be used to standardize WSIs across dif- The ability of a and quality. Some can affect the glass slide (for example, ferent scanning platforms, this approach is still in its machine-learning algorithm to pen marks, dirt and bubbles), whereas others affect the infancy. Recognizing that AI algorithms can be (and remain effective across a wide range of data examples and tissue (for example, the presence of tissue folds, or dif- often are) dependent on consistent colour representa- applications. ferences in thickness or stain intensity) or the scanning tion, standardization of colour becomes essential to process (for example, differences in focus or the gridding establish robust and reproducible quantitative measure- Analogue data effect of the WSI). While pathologists train themselves to ments and analysis63,67,68. This issue is of critical impor- Data that are represented and make an interpretation using the stain protocol they are tance in the context of multicentre DPRs and will force analysed in a physical way (e.g. a glass slide) as opposed accustomed to and read through artefacts on the glass investigators to weigh the pros and cons of central versus to a digital way (e.g. a whole slide, tissue section or digital image, computers must be local scanning. slide image). Analogue data are trained to adapt to such heterogeneity in image presenta- QC of image datasets can also be achieved retro- stored in physical structures. tion. For computer-aided algorithms to leverage digital spectively (that is, after WSI acquisition) using com-
Physical abstraction layer pathology and enable quantification, classification and putational post-processing techniques. QC tools have The physical computational prognostication in a scalable manner, the algorithms now been developed to automatically assess WSI qual- machines, which represent a must be generalizable across datasets. Although variabil- ity, capturing variations in image presentation such as hard stop with respect to the ity in WSI training sets may actually represent a poten- stain distribution, brightness, or glass, tissue and scan- computational scale and tial advantage for the generalizability of the resulting ning artefacts. One example of such a QC algorithm is storage capacity of a computer and typically cannot be algorithms, the performance of algorithms is intimately HistoQC, an open source QC tool that quantitatively overcome by the addition of linked to the control of analytics and homogeneity of assesses the heterogeneity of WSI image datasets and, more software. the datasets63. similarly, identifies artefacts present on the source glass
674 | November 2020 | volume 16 www.nature.com/nrneph Precision medicine in nephrology
slides or on the scanned image69. In the context of clin- As discussed above, the standardization of pathology ical research, HistoQC is currently being evaluated in vocabulary is a prerequisite for the interoperability of NEPTUNE, CureGN and Kidney Precision Medicine image data collection and reporting. Use of standardized Project (KPMP) for curation of the image dataset prior vocabulary and DPR protocols has facilitated the har- to experimental analysis (Table 1). In clinical practice, monization of data collected across multiple consortia, the ability to identify compromised WSIs before they providing insights into the structural manifestations of are assessed by the pathologist or computer-aided tools kidney diseases within and across populations10,53. For reduces downstream delays in clinical diagnosis. example, one study that used visual assessment of sam- ples from three cohorts in North America and China Standardization of the post-analytical phase. The digital showed that the age-adjusted percentage of global glo- ecosystem has the potential to facilitate the harmoni- merulosclerosis is a useful predictor of kidney disease zation of data collection both within and between con- progression across diseases and populations53. Similarly, sortia via the use of shared protocols10, and to improve the percentage of interstitial fibrosis was associated accuracy15 and concordance among pathologists12,16. with the expression of specific genes, transcriptional However, agreement among nephropathologists in regulators and urinary biomarkers across cohorts from diagnostic and tissue assessment remains the limit- North America and Europe54,55. However, visual (that ing factor and a pre-requisite for the training of AI is, manual) assessment of WSIs is time consuming and algorithms5,12,16,70–73. Awareness that the standardization has limited reproducibility71. Computational imag- of pathology definitions and use of a controlled vocab- ing algorithms are likely to provide more efficient and ulary are central elements that drive the reproducibility reproducible automated quantitative assessment across and accuracy of observations and/or of data generated diseases and populations, increasing the robustness of for both human and computer-aided analysis is reflected the observations and scalability of such studies. The inte- by several recent initiatives. For example, NEPTUNE gration of computational image analysis and AI into the and the international digital nephropathology net- digital pathology ecosystem will induce further shifts in work, INTEGRATE, have generated a list of individual renal pathology paradigms towards the integration of observational lesions (descriptors) and their associated structural changes, with multiaxial datasets to redefine definitions, obtained by international consensus, for the categories and subcategories of patients with similar bio- assessment of WSIs10,12. Similarly, the Renal Pathology logical mechanisms, morphological profiles and clinical Society has undertaken an initiative to develop a univer- trajectories. sal lexicon by compiling detailed and unambiguous defi- nitions of pathological lesions and structures with the Integration and interoperability goal of harmonizing classifications, whereas the KPMP Multi-domain datasets generated by systems biology investigators have applied a controlled vocabulary to approaches are represented by different length scales; assemble libraries of conventional diagnoses, disease cat- for example, serum creatinine is measured quantita- egories and descriptors. In KPMP, the descriptor library tively at different time points during the disease course, will be relevant to informing and guiding the assembly morphology data are collected using different method- of the KPMP feature vector library, a ML approach to the ologies, human or machine vision, and metrics, at the automatic detection of normal and abnormal histologi- time of the biopsy, genetic data do not change with time, cal primitives, and to generate a functional ontology and whereas the expression of genes and associated regula- atlas of kidney diseases. tors are dependent on disease status. The extraction, alignment, integration, interpretation and reporting of Shifting paradigms in renal pathology such multi-scale information is an emerging challenge in The transformative approach to the study of kidney dis- nephrology (Table 2). In the new data integration ecosys- Feature vector eases introduced by systems biology science is calling tem, foundational, structural and semantic interopera- Typically, one or more spatial for a change of paradigms in renal pathology for the bility across and within datasets and studies is intimately constructs selected by a subject matter expert that interrogation of WSIs. The current ‘one-size-fits-all’ linked to the standardization of analytics, data sharing serves as a training template approach, whereby pathologists generate classifications and data integration approaches. The interoperability of for any class of machine- using a few preselected parameters and test them against image analysis alone carries challenges, from annotation learning algorithm, including disease outcome or disease biomarkers, provides nei- to algorithms and image features. For example, image convolutional neural networks. ther sufficient granularity nor a quantitative metric that annotations created by vendor-supplied software pro- Ontology adequately represents the complexity of structural and grams in one community are often incompatible with Represented sharable cellular changes that can exist in the kidney. The new those created by other communities who might make use knowledge, a specification of a digital ecosystem and the development of novel tech- of a different program. A valuable approach to facilitat- conceptualization, in the form nologies offers an opportunity to establish new protocols ing the interoperability of heterogeneous data within and of concepts and categories for a specific domain, associated and methodologies — either visual or computer-aided across domains is to use ontologies, which encompass properties and relationships — to retrieve and represent, as interpretable data, the and represent properties and relations between data, cat- between them. Ontologies information contained in kidney biopsy samples. egories, concepts and entities, and to organize the data function using a controlled The introduction of digital pathology and use of into information and knowledge74,75. The Quantitative vocabulary to connect terms descriptors to identify discrete elements of struc- Histopathology Image Ontology is one example of an and use links, logical definitions, concepts and tural changes in tissue has enabled a more compre- effort to establish standardized approaches to ontologi- networks of well-defined hensive visual assessment of the kidney on annotated cal pathology image data for their integration with data relationships. WSIs compared with conventional methodologies10,11. from other domains — such as clinical or demographic
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 675 reVieWs
Data aggregators data. The Quantitative Histopathology Image Ontology However, the concept of data sharing raises several Aggregation or compiling of is aimed at standardizing terminology at multiple legal and ethical issues. Some of these, including issues information from databases levels, including data input and output, parameters for related to inclusiveness, ethically responsible research, with the goal of preparing such image acquisition and analysis, and the execution phase privacy and return of results, are not new and apply to information for data 76 processing. whereby data are recorded and reported . Integration many forms of research. Consideration must also be and interoperability are also centre-stage concepts in given to the sharing of metadata, including validated KPMP, where the development of a kidney cell atlas will feature detector ensembles and associated image sets be based on a comprehensive ontology harmonized with that would be submitted in the curation process in par- pre-existing ontologies. Much can be learned by study- allel with the bio-specimens themselves. In addition, ing other disciplines, such as oncological pathology42,77–85 although best practices relating to data sharing and and radiology, where considerable progress in integrat- material transfer agreements have been articulated in ing information has already been achieved42, and where many documents, particularly in the context of rare ML approaches have been used to combine pathology disease research89–95, guidance specific to the sharing of and molecular data for improved prediction of cancer pathology images and use of DPRs does not yet exist and recurrence81,82,86. is urgently needed (Table 2). An additional consideration is the use of WSI data The data ecosystem for retrospective analysis, beyond the original stated The confluence of large-scale studies, often involving research intent. The digital permanent image libraries clinical data from electronic health records or data that will originate at the institutional level, or under indi- repositories, and the capacity to represent entire research vidual consortium oversight, will naturally outlive the studies in electronic form, has altered the way in which scope of the originally stated informed consent, the spe- researchers collect, store, use and integrate data. The cific aims of the original study, and technology that was oncology field has integrated data from different available at the time of the image data collection. Given domains for quite some time, with the advent of secure that new technologies empower investigators to gener- data environments paired with flexible database design ate hypotheses and ask scientific questions that were not and cloud computing technology aiding the evolution conceivable at the time of the collection, it will be essen- of the data ecosystem. Rather than attempt to place data tial to establish standard processes to enable permission in a single database in which a single framework uni- lists of original data use to be extended. These extensions fies disparate data types, the trend is to now store data will require a standardized adjudicative process that can in individually optimized databases that are then con- operate without the need to re-consent participants, as nected by data aggregators. Functionally, such a set-up their location and identity may no longer be known. enables end-users to perform a search from one data- Such broad consent documents, which essentially base as simultaneous queries against multiple databases. ‘future-proof’ possible unforeseen uses of curated tissues Moreover, this process enables image data to be linked and images are already used by researchers in various with both clinical and molecular data. omics fields. One controversy from the oncology field96, involving the alleged exclusive sharing of pathology Data sharing. Data sharing encompasses several datasets originally intended for general, not-for-profit diverse concepts, from transparency and regulation, use with a commercial AI developer, underscores the to interoperability (discussed above). A key aspira- need for the pathology field to rethink not only how tional goal for data sharing has been, and remains the informed consents should be formulated but also who attainment of interoperability, under the umbrella of should benefit from the sharing of digital datasets and the now-recognized FAIR principle (Findable, associated health information. Accessible, Interoperable and Reusable)87. The past dec- It is imperative that guidelines for the responsible ade has seen the emergence of several policies relating sharing of data in the digital pathology ecosystem are to data sharing and research transparency by the Office formally established. Such guidelines could be modelled of Research Integrity, National Institute of Health or on previously published documents, such as the interna- other organizations88. Providing access to bio-specimen tional code of conduct for genomic and health-related repositories and research data increases their potential data sharing, the founding principles of which are utility for health discovery and validation, and opti- based on the goals of promoting health and advancing mizes their long-term value89–95. This benefit is par- research, and associated ethical elements (for exam- ticularly relevant in the context of rare diseases, where ple, respect of the parties involved and their privacy, data sharing is often needed to obtain sufficient sample distribution of benefits, trust, integrity, accountability, sizes. Historically, renal pathologists have used local reciprocity, and data security and quality)97. datasets for research purposes, with limited opportuni- ties for external QC or data exchange. The availability Image analysis of digital WSIs fosters opportunities for collaboration Two main approaches are used for the interrogation of in a transparent environment. Transparency is not only WSIs: human visual assessment and machine vision. relevant to QC for internal research purposes, for exam- ple, when clinical trials are conducted using a digital Human visual assessment pathology platform for the pathology analysis, the rel- A variety of methods have been used for the visual evant pathology data elements can be easily audited by assessment of WSIs, including conventional diagnostic5 regulatory agencies6. and descriptor-based approaches10, quantification of
676 | November 2020 | volume 16 www.nature.com/nrneph Precision medicine in nephrology
6,12 Pathomics structural and cellular elements , and morphometric Pathomic signatures, which are digital imaging sig- 56 The high-throughput extraction analysis . The collection and quantification of visual data natures obtained through hand-crafted pathomic feature and analysis of features can be further facilitated by digital or manual annotation extraction, are typically engineered to capture a num- derived from digital pathology tools of the regions or objects of interest on the WSIs, ber of features including morphology, intensity, texture, images, typically achieved using techniques from the resulting in increased accuracy and reproducibility of higher-order and spatial features. Morphology features 15 machine vision field, image the observations . Visual assessment of WSIs has been quantify the size and shape of the ROI, such as its com- processing and machine used to identify features that correlate with outcomes or pactness, border irregularity or maximum diameter. learning. the presence of biomarkers, and for the discovery of clin- Intensity features capture the distribution of pixel values ically relevant disease categories. As mentioned earlier, within the ROI, based on the image’s histogram, which is Hand-crafted pathomics Pathomics with feature features such as extent of age-adjusted global glomeru- a probability distribution that encodes the frequency of engineering — a process that losclerosis or interstitial fibrosis can be reliably assessed occurrence of all pixel intensities. Of note, intensity fea- uses domain knowledge to visually and are associated with outcomes including tures do not capture spatially encoded information. For extract features (attributes or disease progression53,54. Other quantitative information instance, the entropy of an image can explain the degree properties) from raw data. that can be obtained from WSIs of renal biopsy samples, of randomness in its pixel intensities but not where the Discovery pathomics such as the interstitial capillary and podocyte density, randomness occurs. Texture features quantify the spatial An approach that uses might enhance our ability to predict predisposition to distribution of pixel information in the image, based on deep-learning- based feature chronic kidney disease progression, hypertension98 and joint-probability distribution functions. These features extraction to learn complex the development of global glomerulosclerosis99–105 but quantify aspects such as overall image brightness, local patterns in the data or image. It commonly uses artificial cannot be easily visually assessed, at least not without a heterogeneities and non-linear relationships between neural networks. significant amount of labour. pixels. Fine- and coarse-texture features capture hetero- geneities at different orders of magnitude. Higher-order Hand-crafted features Machine vision features capture information specific to certain fre- Mathematically well-defined wavelet decomposition features derived from data Machine vision refers to the computational techniques quency bandwidths such as and using feature engineering that are used to convert digital pathology images into multi-scale invariance such as fractal geometry. Spatial techniques. mineable data, using AI techniques such as ML, DL and features capture the spatial organization or architecture pathomics. This approach is driving a fundamental par- of specific histological primitives (for example, glomer- Digital image signature adigm shift in digital pathology in which digital renal uli or tubules) in the tissue. These typically involve graph A mathematical vector, the Voronoi tessellation components of which denote biopsy samples are interpreted as quantitative data- network-based algorithms such as the various quantitative imaging sets. The performance of the AI techniques depends or minimum spanning tree. features that collectively on the quantity and quality of the primary data, where represent an image. ‘quality’ refers to the cleanliness, provenance, accuracy, Discovery pathomics with deep learning. In contrast to Wavelet decomposition signal-to-noise ratio and comprehensiveness to achieve hand-crafted pathomics, discovery pathomics uses DL A signal processing technique the maximum predictive performance. Obtaining to extract features directly from images, without explicit whereby an input signal (e.g. an primary data of sufficient quality or quantity can be mathematical definitions. DL algorithms are often able to image) is passed through a challenging for datasets associated with rare kidney uncover novel and important patterns from sets of input cascade of digital filters (i.e. diseases. images with case-level annotations and without human high-pass filters or low-pass filters) to generate a set of The two main categories of machine vision that are guidance, given their ability to excel in unsupervised bandwidth-limited signal applicable to WSIs of renal biopsy samples are supervised feature extraction. This approach can be particularly components (e.g. filtered learning and unsupervised learning. In unsupervised powerful for applications that remain as unstructured images), which collectively machine vision, features are identified based on their imaging data (for example, generating segmentations of contain the full information of the original signal. similarity to each other and group or category assign- a histological primitive), or when a problem is too com- ment is based on their relative proximity. By contrast, plex for conventional feature engineering. Although DL Fractal geometry supervised learning uses an annotated training dataset can be interpreted as a complete classifier, it can also be A non-regular geometric shape, to learn the relationship between individual features used as a type of image representation analogous to the where the degree of and categorical labels. Supervised and unsupervised hand-crafted pathomic approach. In fact, DL approaches non-regularity is invariant across different spatial scales. learning can be used to convert digital pathology commonly encode an input signal into structured data images into minable data, using two general techniques: form (that is, a transcription process similar to that Voronoi tessellation ‘hand-crafted pathomics’ (also known as conventional used for feature engineering), followed by a decoding A mathematical approach pathomics) and ‘discovery pathomics w i t h D L’. of the data to generate an output signal (that is, a trans- where a plane (e.g. a whole slide image) is partitioned into lation process akin to that used by conventional ML sub-regions based on their Hand-crafted pathomics. This approach involves the seg- algorithms). The most common type of DL used with relative proximity to a given set mentation (by manual or automated means) of an ROI images is based on convolutional neural networks (CNNs), of objects (e.g. image on an image (for example, glomeruli, tubules, arteries whereby a series of shift-invariant filters are used to gen- landmarks, histological or interstitial capillaries), from which high-throughput erate abstract feature maps to learn hierarchical patterns primitives, etc.). quantitative features known as hand-crafted features embedded within the input image. CNN architectures are extracted to derive a pathomic feature space, or have been used to automatically detect and segment digital image signature. This approach is essentially a normal and sclerotic glomeruli, nodular glomeruloscle- transcription process, whereby unstructured imag- rosis, tubules, interstitial space and fibrosis, peritubular ing data (that is, pixels within the ROI) are converted capillaries and arteries72,73,106 (Fig. 2). Other popular DL into structured imaging data (that is, the pathomic architectures (that is, artificial neural networks) include feature space) so that it can be recognized by learning fully convolutional networks, recurrent neural networks and algorithms. generative adversarial networks42 (Table 3).
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 677 reVieWs
Trade-off between hand-crafted and discovery pathomics. Hand-crafted pathomic features are mathematically well Hand-crafted pathomics and discovery pathomics defined, interpretable and can be easily combined with are complementary approaches to quantitative WSI other structured health data, such as radiomics, tran- representation. Each technique has advantages and dis- scriptomic or electronic health record data, to develop advantages, and their proper implementation therefore multi-modal digital biomarkers. However, hand-crafted depends on the specific clinical case and intended use. features are limited to human intuition (that is, human a Automatic detection b Disease staging c Morphometry 70
35
0 70
35
0 d Prediction 8
6 Patient 1 4 Patient 2
Serum creatinine 2
0 1 2 3 4 5 6 7 8 9 10 Days e Support tool for other technologies Mapping of histological primitives identified Laser capture by discovery and hand-crafed pathomics microdissection
RNA Spatially extraction encoded pathogenomic signatures Biopsy sample
RNA extraction
Fig. 2 | Machine vision technology as a support tool in nephropathology. interstitial fractional space. The superimposition of a digital grid on the Machine vision tools — including discovery pathomics with deep learning image enables digital morphometry. d | Machine vision can also be used to and hand-crafted pathomics — can be used to convert digital pathology build models to aid prognostication. For example, qualitative and images into minable data and provide support to pathologists. a | Machine quantitative automatic detection of features of acute tubular injury may vision can be used to automatically detect histological primitives. For predict the course of the disease and response to therapy: the presence of example, a convolutional neural network can be used to detect glomeruli only a few areas of vacuolization with specific qualitative characteristics in frozen kidney sections stained with haematoxylin and eosin. b | Machine could predict rapid recovery from an episode of acute renal failure, with vision has also been used to establish a classifier for glomerular disease normalization of serum creatinine levels, compared with a renal biopsy stage in patients with diabetic glomerulosclerosis. The left panel shows a containing much greater levels of vacuolization. e | Computational imaging paraffin-embedded section of a normal glomerulus (top) and a glomerulus tools can be combined with other methodologies for parallel discovery. For with nodular glomerulosclerosis (bottom) stained with periodic acid Schiff. example, computational image analysis tools can be applied to guide laser The middle panel shows detection of the cellular/nuclei (blue) and matrix capture microdissection by identifying structures with similar pathomic (red) component of the normal (top) and diseased (bottom) glomerulus. The signatures within the same biopsy sample and across biopsy samples. The right panel shows the measurements of the glomerular characteristics for structures with similar pathomic signatures can be captured and analysed both the normal and the diseased glomerulus. c | Application of artificial separately, allowing for spatial mapping of pathogenomic signatures. intelligence-guided morphometry can provide quantitative assessment of Panel b reprinted courtesy of P. Sander and B. Ginley, University at Buffalo, the interstitial fractional space. On the left is a paraffin-embedded section NY, USA. Panel c reprinted courtesy of J. Hodgin, University of Michigan, stained with trichrome. The right panel shows the automatic detection of MI, USA.
678 | November 2020 | volume 16 www.nature.com/nrneph Precision medicine in nephrology
involvement is required to define the features based Furthermore, DL-based approaches are particularly on relevant domain knowledge) and can often lead to useful when the input (for example, manual segmenta- high-dimensional feature spaces that are non-trivial and tion of a histological primitive) and output (for example, computationally expensive to analyse. On the other automatic segmentation) of a problem are both images hand, unsupervised feature generation does not require or when conventional approaches cannot perform the classical feature engineering (that is, features are instead task at hand. However, the trade-off of these benefits generated automatically based on intrinsic patterns is loss of mathematical interpretation and the need for in the data that do not require direct human input). relatively larger datasets.
Table 3 | Use of methods for image analytics in digital nephrology studies Methodology Stains Histological primitive Number of WSIs or cases Task Refs CNN PAS (paraffin Interstitial fibrosis, tubular atrophy, 65 WSIs from transplant Segmentation of multi-classes 119 sections) global glomerulosclerosis kidney biopsies of histological primitives PAS (paraffin Glomeruli, empty Bowman capsule, 142 WSIs from transplant Segmentation of multi-classes 73 sections) globally sclerotic glomeruli, kidney biopsies; 15 WSI of histological primitives proximal tubules, distal tubules, from nephrectomies atrophic tubules, not otherwise identified tubules, arteries PAS (paraffin Non-sclerotic glomeruli, globally WSIs from mouse kidneys; Segmentation of multi-classes 109 sections) sclerotic glomeruli, podocyte nuclei, WSIs from human biopsies of histological primitives other nuclei, interstitial fibrosis/ (number of WSIs not tubular atrophy provided) TRI (paraffin Glomeruli 275 WSIs from 171 renal Glomerular segmentation and 117 sections) biopsies classification H&E (frozen Non-sclerotic glomeruli; globally 40 WSIs from donor kidney Glomerular segmentation 120 sections) sclerotic glomeruli biopsies TRI (paraffin Interstitial fibrosis 171 WSIs from native Prediction of clinical 106 sections) kidney biopsies phenotype Feature-engineering PAS (paraffin Nuclei; glomerular capillary lumina; 54 WSIs from human Glomerular segmentation; 72 RNN sections) glomerular matrix renal biopsies and glomerular nucleus; nephrectomies; 25 WSIs glomerular component from mouse kidneys detection; glomerular feature extraction; diabetic nephropathy classification/ prediction FCN (two-stage PAS (paraffin Glomeruli 22 WSIs from mouse Glomerular segmentation 110 semi-supervised sections) kidneys approach) CNN, SW-CNN and PAS (paraffin Glomeruli 24 WSIs from mouse Glomerular detection and 107 FCN sections) kidneys segmentation CNN and GAN PAS, COL3, Glomeruli, interstitial space, 20 WSIs from mouse Stain-dependent 108 CD31, AFOG interstitial capillaries kidneys supervised segmentation; (paraffin stain-independent sections) unsupervised segmentation Region-based CNN TRI (paraffin Glomeruli 87 WSIs from rat kidneys; Glomerular localization/ 111 sections) 6 WSIs from human kidney detection biopsies Local binary H&E, PAS, TRI, Glomeruli 17 WSIs from mouse Glomerular detection; 110 patterns image SIL, CR (paraffin kidney; 10 WSIs from rat glomerular sub-classification feature vector and sections) kidney; 9 WSIs from native CNN kidney biopsies CNN, SW-CNN and PAS (paraffin Glomeruli 24 WSIs from mouse Glomerular segmentation 114 FCN sections) kidneys Segmental Desmin Glomeruli 20 WSIs from rat kidneys Glomerular detection 113 histogram of (paraffin oriented gradients sections) Colour H&E (paraffin Glomerular urinary space WSIs from mouse kidneys Glomerular detection 135 segmentation sections) and perceptual organization AFOG, Acid Fuchsin Orange G; CNN, convolutional neural network; COL3, collagen type III; CR, Congo red; FCN, fully convolutional network; GAN, general adversarial network; H&E, haematoxylin and eosin; PAS, periodic acid Schiff; RNN, recurrent neural network; SIL, silver; SW-CNN, sliding window convolutional neural network; TRI, trichrome; WSIs, whole slide images.
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 679 reVieWs
Minimum spanning tree Table 4 | Machine vision interrogation of nephropathology samples versus surgical pathology specimens The subset of the edges of a weighted graph that connects Consideration Nephropathology (kidney biopsy) sample Surgical pathology all vertices together by Tissue size Generally small specimens (needle biopsies), which Resection specimens containing large minimizing total edge weight. contain a variable amount of cortex and with limited fragments of tumours ± surrounding For example, if the physical sample of the objects of interest (e.g. focal lesions) non-tumoural tissue distances (edges) between a set of histological primitives Tissue staining A diversity of stains are routinely used. Several Routinely only haematoxylin and eosin (vertices) on a whole slide sections (levels) are stained for each biopsy sample, image is represented as a and each may contain a different representation of graph, then its minimum the objects of interest. The object of interest may spanning tree is the not be represented in all the sections and stains combination of edges that Tissue complexity The kidney parenchyma contains a variety of Generally homogeneous collection minimize total physical histomorphological structures (glomerular unit, of tumoural cells ± surrounding distance, while still connecting proximal and distal tubular segments, collecting non-tumoural tissue all histological primitives. ducts, interstitial space, glomerular and interstitial microvasculature, arteries, veins and lymphatics) Feature engineering and cell types (glomerular and tubular epithelial The process of computationally cells, glomerular/interstitial capillaries, venular and deriving features from arterial endothelial cells) databases on domain knowledge. These features act Histological Quantitative and qualitative heterogeneity of Limited histological heterogeneity. The as mathematical inputs to complexity of disease disease manifestation at the structural and cellular histological manifestation of the disease machine-learning algorithms. manifestation level, often involving more than one structure can generally be graded based on a or cell type at the same time. The association few pre-selected parameters. Although Classifier of diverse structural changes is not necessarily assessment of aggressiveness is based A mathematical function, predictable on the most aggressive cell type, there typically formulated via may be some variability of the tumour machine learning, that maps a grade within each case set of input variables to Clinical data A variety of clinical parameters are often Outcome is evaluated by time to discrete output variables, complexity present; the natural history of kidney diseases is response, progression or death known as classes. heterogeneous across and within the same disease; genetic heterogeneity exists, and there is lack of Convolutional neural standardization of therapies during the course of networks many diseases (CNNs). Types of deep, feedforward networks, Data analysis Most kidney diseases are rare; thus, the collection Many oncological diseases are fairly comprising multiple layers to of samples from multiple institutions or laboratories common. It is therefore easier to collect infer an output from an input is necessary to obtain a sufficient number of a large number of cases across only a few (the image). CNNs use building samples for meaningful studies institutions or laboratories for analysis blocks to learn and extract feature maps from an image, and filters between the Applications and challenges of the pathological features and the conventional use of input and the output that are connected only to a fixed Pathomics and DL are complementary approaches to multiple stains for renal histology preparations, make region from the previous layer computational image analysis with both operationally computer-aided image analysis of renal biopsy samples (and not the entire layer). It and research-driven applications. These approaches challenging (Table 4). In order to produce a complete map also comprises pooling layers facilitate the discovery of new scientific knowledge and or atlas of the entire renal parenchyma and its patholog- to reduce the dimensionality of the features. push the boundaries of clinical intuition. By contrast, ical variations, subject matter experts must manually operationally driven applications provide longer term, annotate a large number of normal and abnormal histo Artificial neural networks stable solutions to existing clinical use-cases, facilitate logical primitives (covering all normal structures and A set of connected input more efficient clinical workflows and enable the use cells, and all possible pathological variations) to train (image) and output (a defined of existing knowledge in new ways72. Research- and the DL algorithms. Definition of the object boundaries class or number) units where each connection has a weight operationally driven applications are far from inde- (for example, glomerular tuft versus glomerular unit) is associated with it. During the pendent of one another, as an inherent feedback loop an obvious important pre-requisite. Although oncolog- learning phase, the network exists between them. Operationally driven applications ical pathologists can focus their manual annotation on a learns by adjusting the weights designed on the basis of existing clinical needs often single monomorphic group of cells that form a specific to predict the correct class label (for categorical data) or lead to unanticipated tertiary findings, which may tumour and then apply pathomic tools to identify sub- regression value (for motivate new research-driven applications, resulting in visual variations, each form of kidney and glomerular continuous data) of the input updated scientific knowledge to be eventually deployed disease is represented by a very heterogeneous aggrega- image. There are different in the form of new operationally driven applications. In tion and variable combination of normal and abnormal types of artificial neural the next decade, nephropathology will be indisputably cells, even within the same disease category. networks (e.g. convolutional, recurrent, generative altered by this feedback loop between AI-driven research The choice of methodology for each purpose is an adversarial). discoveries and their application in clinical practice. additional challenge. For example, several methodologies have been proposed for the segmentation of glomeruli: Research-driven applications supervised classification with a separate detection and In contrast to oncology42,77–85, only a few examples exist segmentation stage, which is highly dependent on the of research-driven applications of computational image precise annotation of the glomerulus; segmentation of analysis in the nephrology literature72,106–114 (Table 3). The the white sickle space surrounding the Bowman capsule, complexity of the renal parenchyma, the heterogeneity which is highly dependent on the glomerulus relative
680 | November 2020 | volume 16 www.nature.com/nrneph Precision medicine in nephrology
cutting plane; and domain adaptation methods, which Similarly, the combination of laser capture microdis- are dependent on a number of annotated target and section techniques to isolate structures and cells can be source domain training data (Table 3). Generalization of enhanced via computational imaging techniques122. The the selected approach is virtually impossible, as the num- pairing of computational image analysis with laser cap- Fully convolutional networks ber of manual annotations required for each histological ture microdissection can be purposed to automatically Convolutional networks that lack fully connected layers and primitive can also vary and is dependent on the intrin- detect the ROI as predefined by the pathologist, thus comprise only a hierarchy of sic structural heterogeneity of the histological primi- substituting the pathologist for the task, and/or to aug- convolutional layers. They tive. Furthermore, as each stain might provide unique ment the process with information that would otherwise can be used to learn information, the algorithms will need to be generated not be captured by the pathologist, but that can offer representations from every pixel and make pixel-level and tested on all stains before current practice can be unique opportunities for omics discovery. For example, predictions. changed to limit AI applications to a single stain. Finally, automatic detection of histological primitives followed as the spatial distribution of different pathological struc- by cluster analysis of those primitives may identify struc- Recurrent neural networks tures is uniquely important to renal diagnostics, future tures that share subvisual features independently of their Acquire inputs over different applications in 3D WSI reconstruction115,116 will facili- morphological similarity at the human eye. Visualization time points, taking into account the status of the input at the tate the accurate assessment of histological primitives tools to map these discoveries back to changes at the tis- different time points and (for example, glomerular number in a biopsy sample) sue level might improve our understanding of disease learning from the discrete and the spatial interplay between pathological features mechanisms123. earlier inputs, displaying (for example, interstitial fibrosis and inflammation) and Arguably the most complex task currently underway dynamic behaviours. associated molecular signatures. in the field of nephropathology is the development of a Generative adversarial To date, the majority of studies using DL technology functional kidney cell atlas by KPMP investigators. The networks with kidney tissue have targeted low-level tasks, such as development of a histology-based atlas of normal and Learn from implementing two the detection and segmentation of normal histological pathological renal biopsy samples, where an array of simultaneous neural networks, primitives (for example, glomeruli, tubules, interstitial validated morpho-genomic and morpho-transcriptomic one producing the data and the other evaluating capillaries and arteries) and universally recognized data extractions will be spatially coupled to the WSI data, the agreement between the pathological primitives (for example, segmental and will provide an interactive computational framework generated data and the global glomerulosclerosis, nodular glomerulosclerosis, in which interrogation can take place across all data original input. interstitial fibrosis and tubular atrophy) on WSIs from domains, within an anatomic framework. This interac- 72,73,111,117–119 120 (Table 3; Radiomics paraffin and frozen sections tive framework is expected to provide deeper insights The high-throughput extraction Fig. 2). More complex tasks have been executed by using into cell and disease classification and lead to the iden- and analysis of features a DL architecture to stage diabetic glomerulosclerosis72 tification of actionable targets at the molecular level. derived from radiological or to determine the association of interstitial fibrosis Unlike most prior efforts, the KPMP effort makes medical images, typically using with clinical phenotypes such as chronic kidney disease digital histology data a centre point for possible initial techniques in machine vision, image processing and machine stage, serum creatinine level, nephrotic range proteinu- discovery modes. learning. ria at the time of the biopsy and renal survival at 1, 3 and 5 years (Fig. 2). Interestingly, the trained CNN models Operationally driven applications High-dimensional feature outperformed the pathologist-estimated fibrosis score As research-driven applications become operationalized, spaces 106 A set of descriptive data in predicting the output classes . AI tools for image analysis are likely to take a role in attributes (i.e. features) Many pathomic tools for the prediction of outcomes, decision support through computer-assisted diagnosis measured across a cohort of genotype and disease mechanisms have not yet been (which, importantly, differs from computer-generated samples, where the number applied to renal pathology. The current lack of train- diagnosis). These decision support tools may be gen- of features (known as the ing and knowledge of the principles of computer-aided erated by processes such as DL algorithms, pathomics dimension of the data) is significantly larger than the image analysis and AI among pathologists for manual with feature engineering and computational integration number of samples. Feature annotation and of data scientists for algorithm develop- of image data with data from other domains (Fig. 3). For spaces are typically ment are just two factors that have prevented this field example, automated detection of histological primitives represented as matrices, where from moving forwards faster. has potential operational applications owing to its ability one dimension (i.e. matrix columns) represents different The development of computational imaging tools to increase the reproducibility of estimates of the distri- features and the other for nephropathology and pathology can exemplify bution, severity and quantity of structures and lesions dimension (i.e. matrix rows) the concept of parallel discoveries (Fig. 2). For exam- that are known predictors of outcome or molecular represents different samples. ple, high-throughput gene expression analysis and signatures but which have limited reproducibility when comparison of multiple samples at the same time can assessed manually (for example, globally sclerotic glo- Use-cases Descriptions of how to perform be achieved by applying tissue microarrays to glass meruli, interstitial fibrosis, diabetic glomerulosclerosis 4,5,12,16,71 a task using examples, slides containing fragments of tissue from numerous stage) ; and to provide a quantitative estimate of outlining the system’s patients121. This technique provides an opportunity to histological primitives that can be visualized but cannot behaviour in response to the apply computational imaging techniques to capture the be reliably counted manually (for example, podocytes, request. desired histological primitive from an image, match his- interstitial capillaries)98,100–104. Thus, DL quantifica- Image microarray tological and cytological features across a range of length tion of these histological primitives can augment the The assembly of uniformly scales, and combine the information to create a single ability of the pathologist to interpret and diagnose sized and resolution-matched construct from multiple images in an array-based format kidney diseases. images, representing key (an image microarray), on multiple patients at the same The information contained in WSIs that the human morphological features and fields of view, and aggregated time. The combination of tissue and image microarrays cognitive system cannot directly perceive, known as into a single monolithic digital enables the assemblage of many fields, reflecting highly subvisual features, provide an opportunity to pose new image file in an array format. specific morpho-omic signatures122. scientific questions and novel insights that have not yet
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 681 reVieWs
126–130 Cluster analysis been conceived or imagined based on conventional process . Our understanding of the key drivers Statistical data analysis pathology approaches. Although the implementation behind AI-based determinations is still incomplete, and technique that is aimed of these approaches and integration of new knowledge we still do not possess sufficiently developed tools to at grouping objects in arising from these findings into clinical practice will be assess performance metrics or enforce QC measures in homogeneous groups (clusters). complex and require extensive validation, future ML the clinical setting. As algorithms become available for pipelines will be an integral component of the patholo- clinical deployment, regulatory bodies will need to work Subvisual features gist’s workflow as decision support tools. Indeed, some closely with the other stake-holders (such as patholo- Features contained in the of these algorithms are already making their way into gists, computer scientists, nephrologists, pathology lab- digital images that cannot be routine practice in the field of radiation oncology42,124,125. oratories and institutions) to establish a framework for captured by the human eye, and require the application The ultimate challenge will be the integration and imple- their approval and implementation. of machine vision technology. mentation of computational image analysis-generated Synonymous with pathomic data with other knowledge domains to establish fused Limitations of sample size feature extraction. datasets and associated morpho-omic classifications, As digital pathology becomes more widely accessible, the Fused morpho-omic prognostications and predictions for kidney diseases analytical tools of AI and ML will reveal new opportu- signatures at the individual patient level (that is, fused morpho- nities with which to develop large-scale, kidney-centric The integration of omic signatures). These applications will also have a data ecosystems, to combine datatypes and datasets for patient-specific data derived role in drug discovery36, in the identification of partic- discovery and validation, and for the practice of precision from multiple omic domains ipants for enrolment in clinical trials and in rationally nephrology (Table 2). However, ML models are based on (e.g. pathomics, genomics, transcriptomics and determining the dosage of therapeutic agents. learning patterns in data and require sufficient numbers metabolomics) to define Despite the excitement about these new approaches, of training examples, with the added observation that patient-specific phenotypes. pathologists and data scientists are advocating for cau- most AI-driven solutions require large amounts of data tious scepticism in the interpretation of quantitative during their training process to be both successful and results derived from the application of image analysis generalizable. This requirement could prove to be a chal- techniques115–119. Similar caution is warranted when lenge in their application to rare kidney diseases, where considering the potential clinical implementation of sample sizes are inherently small. Fortunately, novel AI such approaches, recognizing that vigorous validation and ML approaches that can be applied to small train- should always be an integral step of the deployment ing sets are now becoming available. These approaches
Convolution and activation Discovery pathomics Pooling Up-sampling
Clinical knowledge
Data transcription Data translation
Hand-crafed pathomics Patient Kidney Whole Digital biopsy slide Pathomic Machine- pathology- procurement image feature space learning models derived knowledge Data transcription Data translation
Visual assessment Other omics knowledge
Analogue to digital conversion Knowledge extraction Knowledge integration The renal biopsy glass slide The WSI (input) is transformed into Fused knowledge from WSI and other (input) is converted into a meaningful knowledge (output) domains (input) becomes actionable digital WSI (output) using AI and visual assessment intelligence for patient care (output)
Digital pathology Artificial intelligence Human intelligence Actionable intelligence
Fig. 3 | The nephropathology digital ecosystem. The digital ecosystem (input image data) are visually assessed or scored by a trained pathologist covers three phases: the digital pathology phase (analogue to digital to generate diagnoses or morphological profiles as digital pathology- conversion), the knowledge extraction phase, which relies on human derived knowledge (output data). AI-based machine vision comprises both and/or artificial intelligence (AI), and the actionable intelligence phase, in hand-crafted pathomics and discovery pathomics. For hand-crafted which integrated knowledge is applied to patient care. Each phase begins pathomics, image data (input) is transcribed into pathomic signatures with input data and ends by generating output data that represent the (the pathomic feature space) and then translated into digital pathology- input for the successive phase. In the digital pathology phase, glass slides derived knowledge (the output data) using machine-learning models. For from the renal biopsy sample (that is, the analogue input data), are discovery pathomics, the image data (the input data) are transcribed or converted into whole slide images (WSIs) (that is, digital output data). The encoded into structured data and then decoded into an output signal (the WSIs represent the input data for the knowledge extraction phase, from output image data) using deep learning. Finally, in the actionable which useful information is generated (output data). Knowledge intelligence phase, the knowledge obtained from the digital images extraction can be broken down into two types: human cognition and AI. (input data) is integrated with other data types, for example, omics and AI techniques can be implemented as companion diagnostic tools clinical data, and used to diagnose, prognosticate and select targeted alongside human cognition. Human cognition is employed when WSIs treatments (output data) for the patient.
682 | November 2020 | volume 16 www.nature.com/nrneph Precision medicine in nephrology
Learning health-care use methodologies such as data augmentation, whereby It will be critical for pathologists to be well-versed in systems computational perturbations (usually of a spatial or AI and ML applications and methods, including their Systems built around colorimetric nature) are used to increase the variation limitations, so that these powerful tools can be applied evidence-based medicine and of a training set to improve model generalization; new to clinical practice quickly, effectively and reliably. on knowledge generation 131 processes embedded in daily approaches to high-dimensional data clustering , Owing to the interdisciplinary nature of integrated clinical practice. They rely on whereby samples are grouped into similar categories computational pathology, the need for nephrologists to interdisciplinary approaches on the basis of intrinsic data properties without the acquire new knowledge and skills is equally germane, as to generating large electronic need for labelled training examples; or unsupervised these individuals represent the penultimate consumers health datasets, which are used machine-learning processes, which provide reasonably of this new class of information as generated by these to generate new knowledge by research, resulting in quality robust image segmentation performance when used to approaches. Acknowledgment that AI will be integral 100,118 improvement of patient care. identify normal renal histological primitives . All part of medicine is the first step towards the re-design these capabilities, taken together, represent a promis- of curricula for medical schools and residency or fel- Computational pathology ing future for these new computational tools, and we lowship training programmes to address the current The science that includes big data generation and analysis, expect that they will augment the clinical practice of inadequacy in preparing medical professionals for the 138 image processing, data mining both nephrology and pathology in the near future. imminent future (Table 2). and data fusion of digital The limitations of ML approaches for use on small pathology data. datasets will also be addressed by the development of Conclusions robust learning health-care systems. Aided by AI technol- We can borrow concepts and analogies from biology to ogy, learning health-care systems rely on the data-driven summarize the transformative process that is beginning generation of knowledge to enable the practice of to affect the nephropathology community. This trans- evidence-based medicine132–136. Implementation of this formative process is based on a variety of events that technology-driven health-care model will facilitate represent the first decisive steps in the long anticipated the sharing of data and ML hyper-parameters (that is, pathway towards a new digitally enhanced ecosystem139. parameters that refer to the model selection task or to Humans are learning to operate within the confines of the quality and speed of the learning-process algorithm) this new paradigm, as represented by digital avatars between institutions, leading to aggregate models with of living components, the operability of which is closely better potential for generalizability. Furthermore, the linked to the non-living components of their environ- continued establishment and management of large ment. These biotic and abiotic constituents are linked international nephrology consortia will also have a key together by new, interdependent cycles, dynamisms role in the success of AI applications in rare kidney dis- and flows. The next-generation nephropathology is eases. Given the nature of ongoing consortium-driven expected to operate within this new digital ecosystem kidney research, which is most typically based on and apply artificial (augmenting) intelligence tools to cross-institutional and multi-omics data13,137, it is highly implement human–machine synergistic protocols in likely that these consortia will serve as the blueprint for clinical practice to better categorize, predict and prog- future collaborative AI endeavours in the renal space nosticate kidney diseases, and to identify candidates for that operate at scale. conventional or novel therapeutic approaches. The crit- ical steps needed for the success of this transformational Training of medical professionals process ultimately depend on practitioners in the field Renal pathologists will have a critical role in establish- being open to acquiring new skills, and the adaptation ing the digital nephropathology ecosystem, developing of practice patterns to support the new discovery envi- clinically relevant ML and AI algorithms, and deploy- ronments made possible by these technologies. Most ing such algorithms in routine clinical practice. Renal importantly, it will be essential for renal pathologists pathologists are the subject matter experts in renal his- and nephrologists to understand both the strengths and tology and, as such, they will need to work closely with inherent limitations of digital pathology technologies data and machine vision scientists to design studies and and, at the same time, be willing to invest the time and protocols, ensure that relevant WSI datasets are made effort needed to shoulder their adoption. available, annotate WSI libraries, and finally curate, val- idate and interpret the data that arise from these efforts. Published online 26 August 2020
1. Barisoni, L. & Hodgin, J. B. Digital pathology in 6. Barisoni, L. et al. Novel quantitative method to evaluate 11. Barisoni, L. et al. Digital pathology evaluation in nephrology clinical trials, research, and pathology globotriaosylceramide inclusions in renal peritubular the multicenter nephrotic syndrome study network practice. Curr. Opin. Nephrol. Hypertens. 26, capillaries by virtual microscopy in patients with Fabry (NEPTUNE). Clin. J. Am. Soc. Nephrol. 8, 1449–1459 450–459 (2017). disease. Arch. Pathol. Lab. Med. 136, 816–824 (2012). (2013). 2. Pantanowitz, L. et al. Experience with multimodality 7. Pantanowitz, L., Szymas, J., Yagi, Y. & Wilbur, D. 12. Barisoni, L. et al. Reproducibility of the NEPTUNE telepathology at the University of Pittsburgh Medical Whole slide imaging for educational purposes. descriptor-based scoring system on whole-slide images Center. J. Pathol. Inf. 3, 45 (2012). J. Pathol. Inf. 3, 46 (2012). and histologic and ultrastructural digital images. Mod. 3. Dietz, R. L., Hartman, D. J. & Pantanowitz, L. 8. Saco, A., Bombi, J. A., Garcia, A., Ramirez, J. Pathol. 29, 671–684 (2016). Systematic review of the use of telepathology during & Ordi, J. Current status of whole-slide imaging in 13. Mariani, L. H. et al. CureGN study rationale, intraoperative consultation. Am. J. Clin. Pathol. 153, education. Pathobiology 83, 79–88 (2016). design, and methods: establishing a large prospective 198–209 (2019). 9. Kumar, N., Gupta, R. & Gupta, S. Whole slide imaging observational study of glomerular disease. Am. J. 4. Azancot, M. A. et al. The reproducibility and (WSI) in pathology: current perspectives and future Kidney Dis. 73, 218–229 (2019). predictive value on outcome of renal biopsies from directions. J. Digit. Imaging https://doi.org/10.1007/ 14. Nast, C. C. et al. Morphology in the digital age: expanded criteria donors. Kidney Int. 85, 1161–1168 s10278-020-00351-z (2020). integrating high-resolution description of structural (2014). 10. Barisoni, L. et al. Digital pathology imaging as a novel alterations with phenotypes and genotypes. Semin. 5. Liapis, H. et al. Banff histopathological consensus platform for standardization and globalization of Nephrol. 35, 266–278 (2015). criteria for preimplantation kidney biopsies. Am. J. quantitative nephropathology. Clin. Kidney J. 10, 15. Rosenberg, A. Z. et al. The application of digital Transpl. 17, 140–150 (2017). 176–187 (2017). pathology to improve accuracy in glomerular
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 683 reVieWs
enumeration in renal biopsies. PLoS One 11, e0156441 radiographic applications: a comparison of ZnSCdS:Ag 64. Bellur, S. S. et al. Reproducibility of the Oxford (2016). with Gd2O2S:Tb and Y2O2S:Tb screens. Phys. Med. classification of immunoglobulin A nephropathy, 16. Zee, J. et al. Reproducibility and feasibility of strategies Biol. 42, 1351–1373 (1997). impact of biopsy scoring on treatment allocation and for morphologic assessment of renal biopsies using the 41. Antonuk, L. E. et al. A real-time, flat-panel, amorphous clinical relevance of disagreements: evidence from the nephrotic syndrome study network digital pathology silicon, digital x-ray imager. Radiographics 15, VALidation of IGA study cohort. Nephrol. Dial. Transpl. scoring system. Arch. Pathol. Lab. Med. 142, 993–1000 (1995). 34, 1681–1690 (2019). 613–625 (2018). 42. Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V. 65. Working Group of the International IgA Nephropathy 17. US Food and Drug Administration. FDA allows & Madabhushi, A. Artificial intelligence in digital Network et al. The Oxford classification of IgA marketing of first whole slide imaging system for pathology — new tools for diagnosis and precision nephropathy: rationale, clinicopathological correlations, digital pathology. FDA https://www.fda.gov/news- oncology. Nat. Rev. Clin. Oncol. 16, 703–715 (2019). and classification. Kidney Int. 76, 534–545 (2009). events/press-announcements/fda-allows-marketing- 43. Khorrami, M. et al. Changes in CT radiomic features 66. Working Group of the International IgA Nephropathy first-whole-slide-imaging-system-digital-pathology associated with lymphocyte distribution predict overall Network et al. The Oxford classification of IgA (2017). survival and response to immunotherapy in non-small nephropathy: pathology definitions, correlations, 18. Amin, M. et al. Integration of digital gross pathology cell lung cancer. Cancer Immunol. Res. 8, 108–119 and reproducibility. Kidney Int. 76, 546–556 images for enterprise-wide access. J. Pathol. Inf. 3, 10 (2020). (2009). (2012). 44. Gadegbeku, C. A. et al. Design of the nephrotic 67. Leo, P. et al. Evaluating stability of histomorphometric 19. Leica Biosystems. Leica biosystems receives FDA syndrome study network (NEPTUNE) to evaluate features across scanner and staining variations: 510(k) clearance to market a digital pathology primary glomerular nephropathy by a multidisciplinary prostate cancer diagnosis from whole slide images. system for primary diagnosis. PR Newswire https:// approach. Kidney Int. 83, 749–756 (2013). J. Med. Imaging 3, 047502 (2016). www.prnewswire.com/news-releases/leica-biosystems- 45. Lochmuller, H. et al. RD-connect, NeurOmics and 68. Yoshida, H. et al. Meeting report: the international receives-fda-510k-clearance-to-market-a-digital- EURenOmics: collaborative European initiative for workshop on harmonization and standardization of pathology-system-for-primary-diagnosis-300857825. rare diseases. Eur. J. Hum. Genet. 26, 778–785 digital pathology image, held on April 4, 2019 in html (2019). (2018). Tokyo. Pathobiology 86, 322–324 (2019). 20. Evans, A. J. et al. US food and drug administration 46. Consortium, H. A. et al. Research capacity. Enabling 69. Janowczyk, A., Zuo, R., Gilmore, H., Feldman, M. approval of whole slide imaging for primary diagnosis: the genomic revolution in Africa. Science 344, & Madabhushi, A. HistoQC: an open-source quality a key milestone is reached and new questions are 1346–1348 (2014). control tool for digital pathology slides. JCO Clin. raised. Arch. Pathol. Lab. Med. 142, 1383–1387 47. Alyodawi, K. et al. Compression of morbidity in a Cancer Inf. 3, 1–7 (2019). (2018). progeroid mouse model through the attenuation of 70. Bajema, I. M. et al. Revision of the International 21. Imaging Technology News. FDA grants breakthrough myostatin/activin signalling. J. Cachexia Sarcopenia Society of Nephrology/Renal Pathology Society designation to Paige.AI. Imaging Technology News Muscle 10, 662–686 (2019). classification for lupus nephritis: clarification of https://www.itnonline.com/content/fda-grants- 48. Heinzel, A. et al. Validation of plasma biomarker definitions, and modified National Institutes of Health breakthrough-designation-paigeai (2019). candidates for the prediction of eGFR decline in activity and chronicity indices. Kidney Int. 93, 22. Menter, T., Nicolet, S., Baumhoer, D., Tolnay, M. & patients with type 2 diabetes. Diabetes Care 41, 789–796 (2018). Tzankov, A. Intraoperative frozen section consultation 1947–1954 (2018). 71. Dasari, S., Chakraborty, A., Truong, L. & Mohan, C. by remote whole-slide imaging analysis — validation 49. Kammer, M. et al. Integrative analysis of prognostic A systematic review of interpathologist agreement in and comparison to robotic remote microscopy. J. Clin. biomarkers derived from multiomics panels helps histologic classification of lupus nephritis. Kidney Int. Pathol. 73, 350–352 (2019). discrimination of chronic kidney disease trajectories Rep. 4, 1420–1425 (2019). 23. Brachtel, E. & Yagi, Y. Digital imaging in pathology — in people with type 2 diabetes. Kidney Int. 96, 72. Ginley, B. et al. Computational segmentation and current applications and challenges. J. Biophotonics 5, 1381–1388 (2019). classification of diabetic glomerulosclerosis. J. Am. 327–335 (2012). 50. Perco, P., Pena, M., Heerspink, H. J. L., Mayer, G. Soc. Nephrol. 30, 1953–1967 (2019). 24. van Diest, P. J. et al. Pathology image exchange: the & Consortium, B. E.-D. Multimarker panels in diabetic 73. Hermsen, M. et al. Deep learning-based histopathologic Dutch digital pathology platform for exchange of kidney disease: the way to improved clinical trial assessment of kidney tissue. J. Am. Soc. Nephrol. 30, whole-slide images for efficient teleconsultation, design and clinical practice? Kidney Int. Rep. 4, 1968–1979 (2019). telerevision, and virtual expert panels. JCO Clin. 212–221 (2019). 74. Smith, B. et al. Biomedical imaging ontologies: a Cancer Inf. 3, 1–7 (2019). 51. Freedman, B. I. & Julian, B. A. Evaluation of potential survey and proposal for future work. J. Pathol. Inf. 6, 25. Madabhushi, A. et al. Integrated diagnostics: living kidney donors in the APOL1 era. J. Am. Soc. 37 (2015). a conceptual framework with examples. Clin. Chem. Nephrol. 29, 1079–1081 (2018). 75. Erdal, S. et al. A knowledge-anchored integrative Lab. Med. 48, 989–998 (2010). 52. Freedman, B. I. & Moxey-Mims, M. The APOL1 image search and retrieval system. J. Digit. Imaging 26. Baidoshvili, A. et al. Validation of a whole-slide image- long-term kidney transplantation outcomes network 22, 166–182 (2009). based teleconsultation network. Histopathology 73, — APOLLO. Clin. J. Am. Soc. Nephrol. 13, 940–942 76. Gurcan, M. N. et al. Developing the quantitative 777–783 (2018). (2018). histopathology image ontology (QHIO): a case study 27. Baidoshvili, A. et al. Evaluating the benefits of digital 53. Hommos, M. S. et al. Global glomerulosclerosis with using the hot spot detection problem. J. Biomed. Inf. pathology implementation: time savings in laboratory nephrotic syndrome; the clinical importance of age 66, 129–135 (2017). logistics. Histopathology 73, 784–794 (2018). adjustment. Kidney Int. 93, 1175–1182 (2018). 77. Ali, S., Lewis, J. & Madabhushi, A. Spatially aware cell 28. Hanna, M. G. et al. Implementation of digital 54. Mariani, L. H. et al. Interstitial fibrosis scored on cluster(spACC1) graphs: predicting outcome in pathology offers clinical and operational increase in whole-slide digital imaging of kidney biopsies is a oropharyngeal pl6+ tumors. Med. Image Comput. efficiency and cost savings. Arch. Pathol. Lab. Med. predictor of outcome in proteinuric glomerulopathies. Comput. Assist. Interv. 16, 412–419 (2013). 143, 1545–1555 (2019). Nephrol. Dial. Transpl. 33, 310–318 (2018). 78. Ali, S., Veltri, R., Epstein, J. A., Christudass, C. & 29. Retamero, J. A., Aneiros-Fernandez, J. & 55. Ju, W. et al. Tissue transcriptome-driven identification Madabhushi, A. Cell cluster graph for prediction of Del Moral, R. G. Complete digital pathology for of epidermal growth factor as a chronic kidney disease biochemical recurrence in prostate cancer patients routine histopathology diagnosis in a multicenter biomarker. Sci. Transl. Med. 7, 316ra193 (2015). from tissue microarrays. SPIE Med. Imaging https:// hospital network. Arch. Pathol. Lab. Med. 144, 56. Lemley, K. V. et al. Morphometry predicts early GFR doi.org/10.1117/12.2008695 (2013). 221–228 (2019). change in primary proteinuric glomerulopathies: 79. Ertosun, M. G. & Rubin, D. L. Automated grading 30. Yao, X. Evolving artificial neural networks. Proc. IEEE a longitudinal cohort study using generalized of gliomas using deep learning in digital pathology 87, 1423–1447 (1999). estimating equations. PLoS One 11, e0157148 images: a modular approach with ensemble of 31. Haykin, S. Neural Networks (Prentice Hall, 1994). (2016). convolutional neural networks. AMIA Annu. Symp. 32. McCarthy, J., Minsky, M. L., Rochester, N. 57. Abels, E. et al. Computational pathology definitions, Proc. 2015, 1899–1908 (2015). & Shannon, C. E. A proposal for the Dartmouth best practices, and recommendations for regulatory 80. Heindl, A. et al. Relevance of spatial heterogeneity of summer research project on artificial intelligence, guidance: a white paper from the Digital Pathology immune infiltration for predicting risk of recurrence August 31, 1955. AI Mag. 27 12–14 (2006). Association. J. Pathol. 249, 286–294 (2019). after endocrine therapy of ER+ breast cancer. J. Natl 33. McCarthy, J. J., Minsky, M. L. & Rochester, N. Artificial 58. Cross S., et al. Best practice recommendations for Cancer Inst. https://doi.org/10.1093/jnci/djx137 intelligence (Research Laboratory of Electronics (RLE) at implementing digital pathology (The Royal College (2018). the Massachusetts Institute of Technology (MIT), 1959). of Pathologists, 2018). 81. Lee, G. et al. Supervised multi-view canonical 34. Deng, L. Deep learning: methods and applications. 59. Williams, B. J. & Treanor, D. Practical guide to correlation analysis (sMVCCA): integrating histologic Trends Signal. Process. 7, 197–387 (2014). training and validation for primary diagnosis with and proteomic features for predicting recurrent 35. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. digital pathology. J. Clin. Pathol. 73, 418–422 prostate cancer. IEEE Trans. Med. Imaging 34, Nature 521, 436–444 (2015). (2019). 284–297 (2015). 36. Vamathevan, J. et al. Applications of machine learning 60. Pantanowitz, L. et al. Validating whole slide imaging 82. Lee, G. et al. Co-occurring gland angularity in localized in drug discovery and development. Nat. Rev. Drug. for diagnostic purposes in pathology: guideline from subgraphs: predicting biochemical recurrence in Discov. 18, 463–477 (2019). the college of American Pathologists Pathology and intermediate-risk prostate cancer patients. PLoS One 37. Moore, R. Computed radiography. Med. Electron. 11, Laboratory Quality Center. Arch. Pathol. Lab. Med. 9, e97954 (2014). 78–79 (1980). 137, 1710–1722 (2013). 83. Lucas, M. et al. Deep learning for automatic Gleason 38. Zhao, W. & Rowlands, J. A. X-ray imaging using 61. Newitt, V. N. Program zeroes in on histology digital scan pattern classification for grade group determination amorphous selenium: feasibility of a flat panel self- connection. CAP Today https://www.captodayonline.com/ of prostate biopsies. Virchows Arch. 475, 77–83 scanned detector for digital radiology. Med. Phys. 22, program-zeroes-in-on-histology-digital-scan-connection/ (2019). 1595–1604 (1995). (2018). 84. Osareh, A., Shadgar, B. 2010 5th IEEE International 39. Neitzel, U., Maack, I. & Gunther-Kohfahl, S. Image 62. Yagi, Y. & Gilbertson, J. R. Digital imaging in pathology: Symposium on Health Informatics and Bioinformatics, quality of a digital chest radiography system based the case for standardization. J. Telemed. Telecare 11, 114–120 (IEEE, 2010). on a selenium detector. Med. Phys. 21, 509–516 109–116 (2005). 85. Saltz, J. et al. Spatial organization and molecular (1994). 63. Yagi, Y. Color standardization and optimization in correlation of tumor-infiltrating lymphocytes using 40. Kandarakis, I., Cavouras, D., Panayiotakis, G. S. whole slide imaging. Diagn. Pathol. 6 (Suppl 1), 15 deep learning on pathology images. Cell Rep. 23, & Nomicos, C. D. Evaluating X-ray detectors for (2011). 181–193.e7 (2018).
684 | November 2020 | volume 16 www.nature.com/nrneph Precision medicine in nephrology
86. Ginsburg, S. B., Lee, G., Ali, S. & Madabhushi, A. whole kidney sections. J. Am. Soc. Nephrol. 29, 137. Sampson, M. G. et al. Integrative genomics identifies Feature importance in nonlinear embeddings (FINE): 2081–2088 (2018). novel associations with APOL1 risk genotypes in applications in digital pathology. IEEE Trans. Med. 112. Simon, O., Yacoub, R., Jain, S., Tomaszewski, J. E. black NEPTUNE subjects. J. Am. Soc. Nephrol. 27, Imaging 35, 76–88 (2016). & Sarder, P. Multi-radial LBP features as a tool for 814–823 (2016). 87. Kush, R. D. et al. FAIR data sharing: the roles of rapid glomerular detection and assessment in whole 138. Kolachalama, V. B. & Garg, P. S. Machine learning and common data elements and harmonization. J. Biomed. slide histopathology images. Sci. Rep. 8, 2032 medical education. NPJ Digit. Med. 1, 54 (2018). Inform. 107, 103421 (2020). (2018). 139. Baltscheffsky, H. Major “anastrophes” in the origin 88. Medicine, I. O. Sharing Clinical Research Data: 113. Kato, T. et al. Segmental HOG: new descriptor for and early evolution of biological energy conversion. Workshop Summary (The National Academies Press, glomerulus detection in kidney microscopy image. J. Theor. Biol. 187, 495–501 (1997). 2013). BMC Bioinforma. 16, 316 (2015). 89. Mascalzoni, D. et al. International charter of principles 114. Gadermayr, M., Dombrowski, A.-K., Acknowledgements for sharing bio-specimens and data. Eur. J. Hum. Genet. Klinkhammer, B. M., Boor, P. & Merhof, D. CNN L.B. has received research support grants from the National 23, 721–728 (2015). cascades for segmenting sparse objects in gigapixel Institute of Diabetes and Digestive and Kidney Diseases of the 90. Sampson, M. G. & Kang H. M. Using and producing whole slide images. Comput. Med. Imaging Graph. 71, National Institutes of Health (U54 DK083912, UM1 publicly available genomic data to accelerate discovery 40–48 (2019). DK100845, 1R01-DK118431-01, R01 DK108805, in nephrology. Nat. Rev. Nephrol. 15, 523–524 115. Jansen, I. et al. Three-dimensional histopathological M1701051) and the Nephcure Foundation. A.M. has received (2019). reconstruction of bladder tumours. Diagn. Pathol. 14, research support grants from the National Cancer Institute of 91. Knoppers, B. M. et al. A P3G generic access 25 (2019). the National Institutes of Health (1U24CA199374-01, agreement for population genomic studies. 116. Jansen, I. et al. Histopathology: ditch the slides, R01CA202752-01A1, R01CA208236-01A1, R01 CA216579- Nat. Biotechnol. 31, 384–385 (2013). because digital and 3D are on show. World J. Urol. 36, 01A1, R01 CA220581-01A1, 1U01 CA239055-01); the 92. Ness, R. B. & American College of Epidemiology Policy 549–555 (2018). National Institute for Biomedical Imaging and Bioengineering Committee. Biospecimen “ownership”: point. Cancer 117. Kannan, S. et al. Segmentation of glomeruli within (1R43EB028736-01); the National Center for Research Epidemiol. Biomarkers Prev. 16, 188–189 (2007). trichrome images using deep learning. Kidney Int. Rep. Resources under award number 1 (C06 RR12463-01); the VA 93. Official Journal of the European Union. Council 4, 955–962 (2019). Merit Review Award IBX004121A from the United States recommendation of 8 June 2009 on an action in 118. Gallego, J. et al. Glomerulus classification and detection Department of Veterans Affairs Biomedical Laboratory the field of rare diseases. Official Journal of the based on convolutional neural networks. J. Imaging 4, Research and Development Service; the DOD Breast Cancer European Union https://eur-lex.europa.eu/LexUriServ/ 20 (2018). Research Program Breakthrough Level 1 Award LexUriServ.do?uri=OJ:C:2009:151:0007:0010:EN:PDF 119. Ginley B. et al. Neural network segmentation (W81XWH-19-1-0668); the DOD Prostate Cancer Idea (2009). of interstitial fibrosis, tubular atrophy, and Development Award W81XWH-15-1-0558; the DOD Lung 94. OECD. https://www.oecd.org/sti/inno/38500813.pdf glomerulosclerosis in renal biopsies. Preprint at arXiv Cancer Investigator-Initiated Translational Research Award (2007). https://arxiv.org/abs/2002.12868 (2020). (W81XWH-18-1-0440); the DOD Peer Reviewed Cancer 95. Toronto International Data Release Workshop Authors. 120. Marsh, J. N. et al. Deep learning global Research Program (W81XWH-16-1-0329); the Ohio Third Prepublication data sharing. Nature 461, 168–170 glomerulosclerosis in transplant kidney frozen sections. Frontier Technology Validation Fund; the Wallace H. Coulter (2009). IEEE Trans. Med. Imaging 37, 2718–2728 (2018). Foundation Program in the Department of Biomedical 96. Thomas, K. & Ornstein, C. Sloan Kettering’s cozy deal 121. Avninder, S., Ylaya, K. & Hewitt, S. M. Tissue Engineering and The Clinical and Translational Science Award with start-up ignites a new uproar. The New York microarray: a simple technology that has revolutionized Program (CTSA) at Case Western Reserve University. U.G.J.B. Times https://www.nytimes.com/2018/09/20/health/ research in pathology. J. Postgrad. Med. 54, 158–162 has received research support grants from the National Cancer memorial-sloan-kettering-cancer-paige-ai.html (2018). (2008). Institute of the National Institutes of Health (U24CA199374, 97. Sugano, S. International code of conduct for genomic 122. Hipp, J. D. et al. Computer-aided laser dissection: a U2CDK114886, U54(MMDHHS), NIH1RC2DK01). The con- and health-related data sharing. HUGO J. 8, 1 (2014). microdissection workflow leveraging image analysis tent is solely the responsibility of the authors and does not 98. Afsar, B. et al. Capillary rarefaction from the kidney tools. J. Pathol. Inf. 9, 45 (2018). necessarily represent the official views of the National Institutes point of view. Clin. Kidney J. 11, 295–301 (2018). 123. Corredor, G., Whitney, J., Arias, V., Madabhushi, A. of Health, the U.S. Department of Veterans Affairs, the 99. Venkatareddy, M. et al. Estimating podocyte number & Romero, E. Training a cell-level classifier for Department of Defense, or the United States Government. and density using a single histologic section. J. Am. detecting basal-cell carcinoma by combining human Soc. Nephrol. 25, 1118–1129 (2014). visual attention maps with low-level handcrafted Author contributions 100. Hodgin, J. B. et al. Glomerular aging and focal global features. J. Med. Imaging 4, 021105 (2017). All authors contributed to the general design of this Review glomerulosclerosis: a podometric perspective. J. Am. 124. Lafata, K. J. et al. Association of pre-treatment with specific areas of emphasis or contribution enumerated Soc. Nephrol. 26, 3162–3178 (2015). radiomic features with lung cancer recurrence as follows: K.J.L. contributed to the sections on machine 101. Naik, A. S. et al. Quantitative podocyte parameters following stereotactic body radiation therapy. Phys. vision and provided technical insight for the reference tables predict human native kidney and allograft half-lives. Med. Biol. 64, 025007 (2019). and figures. S.M.H. and A.M. provided historical context and JCI Insight 1, e86943 (2016). 125. Lafata, K. J. et al. An exploratory radiomics approach general fact-checking throughout the manuscript. L.B. 102. Ding, F. et al. Accelerated podocyte detachment and to quantifying pulmonary function in CT images. and U.G.J.B. carried out a thematic review of the current progressive podocyte loss from glomeruli with age in Sci. Rep. 9, 11509 (2019). state of digital pathology, providing substantial narrative con- Alport syndrome. Kidney Int. 92, 1515–1525 (2017). 126. Glassy, E. F. Digital pathology: quo vadis? Pathology tent and final editorial review. 103. Kikuchi, M., Wickman, L., Hodgin, J. B. & Wiggins, R. C. 50, 375–376 (2018). Podometrics as a potential clinical tool for glomerular 127. Van Es, S. L. Digital pathology: semper ad meliora. Competing interests disease management. Semin. Nephrol. 35, 245–255 Pathology 51, 1–10 (2019). L.B. is a consultant for Vertex, Sangamo and Protalix. She is (2015). 128. Van Es, S. L. & Madabhushi, A. The revolving door for also on the scientific advisory board for Vertex. Additionally, 104. Nishizono, R. et al. FSGS as an adaptive response to AI and pathologists — docendo discimus? J. Med. she is the chair of the pathology committee for NEPTUNE, growth-induced podocyte stress. J. Am. Soc. Nephrol. Artif. Intell. 2, 12 (2019). director of the Digital Pathology Repository for CureGN and 28, 2931–2945 (2017). 129. Dietz RL, P. L. The future of anatomic pathology: deus curator for the COVID-19 digital pathology repository hosted 105. Kather, J. N. et al. Predicting survival from colorectal ex machina? J. Med. Artif. Intell. 2, 4 (2019). at the NIH/NCI. She is involved as a co-investigator in KPMP, cancer histology slides using deep learning: a 130. Flotte, T. J. & Bell, D. A. Anatomical pathology is at and as co-PI in an R01 that uses digital pathology material. retrospective multicenter study. PLoS Med. 16, a crossroads. Pathology 50, 373–374 (2018). A.M. is an equity holder in Elucid Bioimaging and in Inspirata e1002730 (2019). 131. Lafata, K. J. et al. Data clustering based on Langevin Inc. In addition, he has served as a scientific advisory board 106. Kolachalama, V. B. et al. Association of pathological annealing with a self-consistent potential. Q. Appl. member for Inspirata Inc, AstraZeneca, Bristol Meyers-Squibb fibrosis with renal survival using deep neural networks. Mathematics 77, 591–613 (2019). and Merck. Currently, he serves on the advisory board of Kidney Int. Rep. 3, 464–475 (2018). 132. McLachlan, S. et al. The heimdall framework for Aiforia Inc. He also has sponsored research agreements with 107. Gadermayr, M. et al. Segmenting renal whole slide supporting characterisation of learning health Philips and Bristol Meyers-Squibb. His technology has been images virtually without training data. Comput. Biol. systems. J. Innov. Health Inf. 25, 77–87 (2018). licensed to Elucid Bioimaging. He is also involved in a NIH Med. 90, 88–97 (2017). 133. National Academies Press. The Learning Healthcare U24 grant with PathCore Inc, and three different R01 grants 108. Gadermayr, M. et al. Generative adversarial networks System: Workshop Summary (National Academies with Inspirata Inc. U.G.J.B. is a member of the advisory for facilitating stain-independent supervised and Press (US), 2007). Board for Inspirata Inc, a company that provides digital unsupervised segmentation: a study on kidney histology. 134. Institute of Medicine. Digital Infrastructure for the pathology workflow solutions. The other authors declare no IEEE Trans. Med. Imaging 38, 2293–2302 (2019). Learning Health System: The Foundation for competing interests. 109. Lutnick, B. et al. An integrated iterative annotation Continuous Improvement in Health and Health Care: technique for easing neural network training in Workshop Series Summary (Institute of Medicine, Peer review information medical image analysis. Nat. Mach. Intell. 1, 112–119 2011). Nature Reviews Nephrology thanks M. Hermsen, P. Sarder (2019). 135. Samsi, S., Jarjour, W. & Krishnamurthy, A. Glomeruli and the other, anonymous, reviewer(s) for their contribution 110. Gupta, L., Klinkhammer, B. M., Boor, P., Merhof, D. segmentation in H&E stained tissue using perceptual to the peer review of this work. & Gademayr, M. Iterative learning to make the most organization. Proceedings of the IEEE Signal of unlabeled and quickly obtained labeled data in Processingin Medicine and Biology Symposium, 1–5 Publisher’s note histology. Proc. Mach. Learn. Res. 102, 215–224 (IEEE, 2012). Springer Nature remains neutral with regard to jurisdictional (2019). 136. McLachlan, S. et al. Learning health systems: the claims in published maps and institutional affiliations. 111. Bukowy, J. D. et al. Region-based convolutional neural research community awareness challenge. J. Innov. nets for localization of glomeruli in trichrome-stained Health Inf. 25, 981 (2018). © Springer Nature Limited 2020
NaTure Reviews | NePHRoLogy volume 16 | November 2020 | 685