2017 IEEE 13th International Conference on eScience Towards a Fully Automated Diagnostic System for Orthodontic Treatment in Dentistry Seiya Murata∗, Chonho Lee†, Chihiro Tanikawa‡ and Susumu Date† ∗Graduate School of Information Science and Technology, Osaka University Email: [email protected] †Cybermedia Center, Osaka University Email: {leech, date}@cmc.osaka-u.ac.jp ‡Graduate School of Dentistry, Osaka University Email: [email protected] Abstract—A deep learning technique has emerged as a success- healthcare. For example, such imaging enables anyone to self- ful approach for diagnostic imaging. Along with the increasing check the degree of malocclusion and jaw abnormality from demands for dental healthcare, the automation of diagnostic oral and facial images, which are the causes of masticatory imaging is increasingly desired in the field of orthodontics for many reasons (e.g., remote assessment, cost reduction, etc.). dysfunction, apnea syndromes, and pyorrhea, etc. Moreover, However, orthodontic diagnoses generally require dental and it leads to objective diagnosis that is important for dental medical scientists to diagnose a patient from a comprehensive and medical scientists and their patients because the diagnosis perspective, by looking at the mouth and face from different directly affects to the treatment plan, treatment priorities, and angles and assessing various features. This assessment process insurance coverage. takes a great deal of time even for a single patient, and tends to generate variation in the diagnosis among dental and However, dental and medical scientist are struggling against medical scientists. In this paper, the authors propose a deep time and various limits against accuracy when assessing their learning model to automate diagnostic imaging, which provides patients. In an orthodontic clinic, a dentist generally carries an objective morphological assessment of facial features for out the examination, consultation, and treatment. In addition, orthodontic treatment. The automated diagnostic imaging system other dentists and specialists meet and spend a great deal of dramatically reduces the time needed for the assessment process. It also helps provide objective diagnosis that is important for time assessing patients and creating their medical records. For dental and medical scientists as well as their patients because example, in Osaka University Dental Hospital1, more than a the diagnosis directly affects to the treatment plan, treatment hundred patients visit every day, and dentists regularly spend priorities, and even insurance coverage. The proposed deep two to three hours creating a medical record for just one learning model outperforms a conventional convolutional neural patient. network model in its assessment accuracy. Additionally, the authors present a work-in-progress development of a data science Furthermore, an orthodontic diagnosis has a specific diffi- platform with a secure data staging mechanism, which supports culty in that dentists must diagnose a patient comprehensively computation for training our proposed deep learning model. The by looking at the entire face while assessing multiple parts of platform is expected to allow users (e.g., dental and medical the face from different angles, rather than simply targeting one scientists) to securely share data and flexibly conduct their data analytics by running advanced machine learning algorithms (e.g., part of the mouth and face. For instance, a dentist must first deep learning) on high performance computing resources (e.g., a look at the frontal face of a patient and examine the patient GPU cluster). for drooping eyelid and/or distortion of the nose; the dentist must confirm maxillary protrusion and/or prognathism from I. INTRODUCTION the side of the face; check the alignment of the teeth from The rapidly increasing availability of medical and dental the oral exam; and then, finally give the patient a severity data is becoming a driving force for the adoption of data- score. This complex assessment process consequently brings driven approaches, which generates the motivation to automate variation into the diagnosis among different dentists. medical tasks such as diagnostic imaging, disease progression For this reason, and in order to improve the assessment modelling, and cohort analysis. In particular, diagnostic imag- speed and accuracy, the authors propose a deep learning ing has been achieving success in diagnosing the presence of model to automate diagnostic imaging. This model provides tuberculosis in chest x-rays [1], detecting diabetic retinopathy an objective morphological assessment of facial features. To from retinal photographs [2] as well as locating breast cancer the best of our knowledge, this is the first attempt to apply in pathology images [3], all by utilizing deep learning tech- deep learning technique to an orthodontic assessment. This niques [4]. automation dramatically reduces the assessment workload for Automated diagnostic imaging is eagerly desired in the field the dentists and also prevent variation in the diagnosis. of orthodontics as well, along with the increasing demands for dental healthcare, becoming one of the factors for all forms of 1http://hospital.dent.osaka-u.ac.jp 978-1-5386-2686-3/17 $31.00 © 2017 IEEE 1 DOI 10.1109/eScience.2017.12 To train the proposed model, computational support for a (e.g., patients’ facial images) and generates multilabels (e.g., workload on a large-sized dataset is necessary. Therefore, the the morphological assessment of facial features). To the best authors are developing a data science platform with a secure of our knowledge, this is the first attempt to apply a deep data staging mechanism via VPN connection between Dental learning technique to an orthodontic assessment. Hospital1 and Cybermedia Center2 in Osaka University. The Transfer Learning: In practice, it is very difficult to collect platform is expected to allow us to securely transfer training and prepare a large number of medical images (e.g., X-rays, data (i.e., patients’ facial images) to compute nodes, and then CT scans and MRIs) to train a deep learning model from to process such data on a GPU cluster, and finally, to obtain scratch. An insufficient number of training data causes low the trained model. After the completion of training, the data is performance of analytics, e.g., classification and prediction deleted, and the network session is closed. While the promise accuracy. In such a case, a transfer learning approach may help of big data analytics is materializing, there is still a non- to improve performance [5], [6]. Transfer learning attempts negligible gap between its potential and usability in practice to gain knowledge from one or more source tasks and apply due to various factors that are inherent in the data itself such the knowledge to a target task. There are a few CNN-based as scale, heterogeneity and privacy. The authors aim to fill the models pretrained on a very large dataset, e.g., ImageNet, gap and make it possible for authorized users (e.g., dentists, for image classification. The size of our dataset is relatively clinical experts and researchers) to securely share data among small compared to those models. We initialize our model with themselves and to flexibly conduct their data analytics by the weights of ImageNet and fine-tune layers (especially later running advanced machine learning (ML) algorithms on high layers) by using our dataset. performance computing (HPC) resources. In this paper, we present the proposed deep leaning-based Multi-label Image Classification: In general, medical images diagnostic imaging system, and also introduce the envisioned may contain multiple regions of interest to be evaluated. As data science platform. The remainder of this paper is organized described above, an orthodontist diagnoses a patient based on as follows. Section II introduces some preliminaries and re- the assessment results of various facial parts from several dif- lated work. Section III describes the deep learning model, and ferent facial images. Thus, for automated diagnostic imaging, a Section IV shows its performance accuracy and discusses its typical single-label (binary or multi-class) image classification practical use. Finally, we present a data science platform that model should be extended to solve the problem of a multi- supports the computation of the automated diagnostic system label image classification. The multi-labels correspond to the in Section V, followed by our conclusions. assessment results of facial parts. A common approach to the multi-label image classification II. PRELIMINARIES AND RELATED WORK is to extend CNN so that it handles multiple single-label Deep Learning: In recent years, the deep learning technique image classification problems [7]. However, this approach has has become the most popular and successful approach for drawbacks in its computational efficiency as follows. First, the problems in machine learning and image recognition. There number of parameters to be learned keeps increasing, which are two well-known models, called the convolutional neural is caused by a large number of label combinations. The label network (CNN) and the recurrent neural network (RNN). The space with m labels is exponentially expanded from O(2) to CNN is generally composed of convolutional layers, pooling O(2m). Secondly, the approach fails to learn the dependency layers and fully-connected layers. The convolution of the between multiple labels. It does