Deep Learning for Organ Segmentation in Radiotherapy: Federated Learning, Contour Propagation, and Domain Adaptation

Deep learning for organ segmentation in radiotherapy: federated learning, contour propagation, and domain adaptation Eliott Brion Institute of Information and Communication Technologies, Electronics and Applied Mathematics (ICTEAM) Universite´ catholique de Louvain A thesis submitted in partial fulfillment of the requirements for the degree of Doctor in Applied Sciences February 22, 2020 2 PhD committee Thesis supervisors Prof. Benoit Macq Institute of Information and Communication Technologies, Electronics and Applied Mathematics, Universitécatholique de Louvain Ecole´ polytechnique de Louvain, Universitécatholique de Louvain Prof. John A. Lee Molecular Imaging, Radiotherapy and Oncology, Universitécatholique de Louvain Ecole´ polytechnique de Louvain, Universitécatholique de Louvain President of the jury Prof. Jean-Pierre Raskin Institute of Information and Communication Technologies, Electronics and Applied Mathematics, Universitécatholique de Louvain Ecole´ polytechnique de Louvain, Universitécatholique de Louvain 3 4 Members Prof. Christophe De Vleeschouwer Institute of Information and Communication Technologies, Electronics and Applied Mathematics, Universitécatholique de Louvain Ecole´ polytechnique de Louvain, Universitécatholique de Louvain Prof. Xavier Geets Institut Roi Albert II, Radiothérapieoncologique Dr. Rudi Labarbe Ion Beam Applications SA (IBA), Louvain-la-Neuve External members Prof. Romain Hérault LITIS, INSA Rouen, Normandie Université Prof. Bernard Gosselin Universitéde Mons Abstract External radiotherapy treats cancer by pointing a source of radiation (either photons or protons) at a patient who is lying on a couch. While it is used in more than half of all cancer patients, this treatment suffers from two major shortcomings. First, the target sometimes receives less radiation dose than prescribed, and healthy organs receive more of it. Although some dose to healthy organs is inevitable (since the beam must enter the body), part of it is due to poor management of anatomical variations during treatment. As a consequence, the tumor can fail to be controlled (possibly leading to decreased quality of life or even death) and secondary cancers can be induced in the healthy organs. Second, the slowness of treatment planning escalates healthcare costs and reduces doctors' face-to-face time with their patients. Coupled with steady improvement in the quality of the medical images used for treatment planning and monitoring, deep learning promises to offer fast and personalized treatment for all cancer patients sent to radiotherapy. Over the past few years, computation capabilities, as well as digitization and labeling of images, have been increasing rapidly. Deep learning, a brain-inspired statistical model, now has the potential to identify targets and healthy organs on medical images with unprece- dented speed and accuracy. This thesis focuses on three aspects: slice interpolation, CBCT transfer, and multi-centric data gathering. The treatment planning image (called computed tomography, or CT) is volumetric, i.e., it consists of a stack of slices (2D images) of the patient's body. The current radiotherapy workflow requires contouring the target and healthy organs on all slices manually, a time-consuming pro- cess. While commercial suites propose fully automated contouring with deep learning, their use for contour propagation remains unexplored. 5 6 In this thesis, we propose a semi-automated approach to propagate the contours from one slice to another. The medical doctor, therefore, needs to contour only a few slices of the CT, and those contours are automati- cally propagated to the other slices. This accelerates treatment planning (while maintaining acceptable accuracy) by allowing neural networks to propagate knowledge efficiently. In radiotherapy, the dose is not delivered at once but in several small doses called fractions. The poorly measured anatomical variation between fractions (e.g., due to bladder and rectal filling and voiding) ham- pers dose conformity. This can be mitigated with the Cone Beam CT (CBCT), an image acquired before each fraction which can be considered a low-contrast CT. Today, targets and organs at risk can be identified on this image with registration, a model making assumptions about the nature of the anatomical variations between CT and CBCT. However, this method fails when these assumptions are not met (e.g., in the case of large deformations). In contrast, deep learning makes few assumptions. Instead, it is a flexible model that is calibrated on large databases. More specifically, it requires annotated CBCTs for training, and those labels are time-consuming to produce. Fortunately, large databases of contoured CTs exist, since contouring CTs has been part of the workflow for decades. To leverage such databases we propose cross-domain data augmentation, a method for training neural networks to identify targets and healthy organs on CBCT using many annotated CTs and only a few annotated CBCTs. Since contouring a few CBCTs may already be chal- lenging for some hospitals, we investigate two other methods { domain adversarial networks and intensity-based data augmentation { that do not require any annotations for the CBCTs. All these methods rely on the principle of sharing information between the two image modalities (CT and CBCT). Finally, training and validating deep neural networks often requires large, multi-centric databases. These are difficult to collect due to tech- nical and legal challenges, as well as inadequate incentives for hospitals to collaborate. To address these issues, we apply TCLearn, a federated Byzantine agreement framework, to our use-case. This framework is shown to share knowledge between hospitals efficiently. Acknowledgments Most theses are too large enterprises to be completed by any single indi- vidual. I would like to express my gratitude to the people who supported me along the journey. First, my family. Maman, papa, Elsa, Mamy, Boris, thank you for believing in me. Second, my supervisors. Benoit, thank you for having trusted me four years ago and ever since. I appreciated your invariable enthusiasm and the opportunities you provided, including joining you for one month during your sabbatical at McGill University. John, thank you for your availability and precious advice. Then come the friends. Corentin, Christophe, Adrien, and Gaël, thank you for having supported me during the lows and for having cel- ebrated the highs. My lab mates, a.k.a. \Team jprod". Jean, there is no one else I would rather have shared the seat with during the emotional roller- coaster ride of this research project. Paul, Umair, Antoine, Sylvain, Gaetan, Damien, Ana, and Simon, thank you for having made this lab a fun, supporting, and stimulating place to be. Thanks to my housemates Nicolas, Corentin, Laury, and Annelise, for our \repas coloc" and other activities providing precious relaxing time. Without data, no deep learning. I would like to thank our hospital partners who trusted us with theirs: Dr. Jean-Fran¸coisDaisne and Dr. Vincent Remouchamps, from CHU-UCL-Namur, Nicolas Meert, from CHU-Charleroi, as well as the teams of doctors and physicians from both centers who welcomed us for several weeks. 7 8 I am not best known for my ability at handling administrative stuff. Patricia, thank you for your patience. Similarly, thank you to Brigitte and Fran¸cois,UCLouvain's system administrators, for your support. This work was made possible thanks to Sara, who annotated a large amount of data used in the studies, and Gabrielle, who carefully revised this manuscript. Thanks to the two of you. I also had the chance to be followed by a thesis committee of bright and helping scientists. Rudi, thank you for guiding and inspiring me from my internship at IBA when I was still a master's student to editing the present document. Christophe, thank you for your numerous ideas and guidance. Several chapters of this document have been greatly im- proved thanks to your valuable feedback. Finally, I had the chance to start this Ph.D. with an inspiring internship at IBM Almaden. Mehdi and Hongzhi, thank you for your guidance on-site and your invitation to social activities offsite. This stay helped to put me on the right track. List of publications Related papers in peer-reviewed journals and conference proceedings. Contour propagation in CT scans with convolutional neural networks [102] Léger,J., Brion, E., Javaid, U., Lee, J., De Vleeschouwer, C., Macq, B. (2018, September). In International Conference on Advanced Concepts for Intelligent Vision Systems (pp. 380-391). Springer, Cham. Using planning CTs to enhance CNN-based bladder segmentation on cone beam CT [18] Brion, E., Léger,J., Javaid, U., Lee, J., De Vleeschouwer, C., Macq, B. (2019, March). Using planning CTs to enhance CNN-based bladder segmentation on cone beam CT. In Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling (Vol. 10951, p. 109511M). International Society for Optics and Photonics. Secure architectures implementing trusted coalitions for blockchained distributed learning (TCLearn) [112] Lugan, S., Desbordes, P., Brion, E., Tormo, L. X. R., Legay, A., Macq, B. (2019). Secure architectures implementing trusted coalitions for blockchained distributed learning (TCLearn). IEEE Access, 7, 181789-181799. 9 10 Cross-domain data augmentation for deep learning based male pelvic organ segmentation in cone beam CT [101] Léger,J., Brion, E., Desbordes, P., De Vleeschouwer, C., Lee,

Load more