Automatic pixel classification and segmentation of biofilm forming bacteria from fluorescence microscopy images. Koen Hendriks STUDENT NUMBER: 2018158 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DATA SCIENCE & SOCIETY DEPARTMENT OF COGNITIVE SCIENCE & ARTIFICIAL INTELLIGENCE SCHOOL OF HUMANITIES AND DIGITAL SCIENCES TILBURG UNIVERSITY Thesis committee: Dr. Sharon Ong Dr. Marie Postma Tilburg University School of Humanities and Digital Sciences Department of Cognitive Science & Artificial Intelligence Tilburg, The Netherlands June 2019 Preface I would like to thank Dr. Sharon Ong for her excellent support during this project and Dr. Marie Postma for her valuable feedback and comments on the first version. Furthermore, I would like to thank my parents for always being supportive of my decisions, providing me with the resources which allowed me to pursue this master’s degree and a loving home on which I can always fall back. Table of contents 1. Introduction .................................................................................................... 5 2. Related Work ................................................................................................. 6 2.1 Brief history of biofilms........................................................................... 6 2.2 Analyzing time-lapse fluorescence microscopy images .......................... 6 2.2.1 Bacteria detection.............................................................................. 7 2.2.2 Bacteria Tracking ............................................................................ 10 3. Experimental Setup ...................................................................................... 10 3.1 Data ........................................................................................................ 10 3.1.1 Creating ground truth ...................................................................... 11 3.1.2 Feature engineering ......................................................................... 11 3.1.3 Balancing classes ............................................................................ 15 3.2 Method / Models .................................................................................... 17 3.2.1 Software and Libraries .................................................................... 18 3.2.2 Models............................................................................................. 18 3.2.3 Hyperparameter Tuning .................................................................. 19 3.2.4 Evaluation ....................................................................................... 21 3.2.5 Baseline ........................................................................................... 21 4. Results .......................................................................................................... 23 5. Discussion .................................................................................................... 27 6. Conclusion ................................................................................................... 27 Automatic pixel classification and segmentation of biofilm forming bacteria from fluorescence microscopy images. Koen Hendriks Antimicrobial resistance is considered a threat to global health. One cause of this phenomenon is biofilm formation, of which knowledge is limited. Experiments of biofilm forming bacteria can generate a significant amount of time series fluorescence microscopy images. However, at present, these images require extensive manual labor in order to analyze, and results are often subjected to human error and biases. In addition, current research on image analysis of bacteria often does not examine in detail the important first step in bioimage analysis – pixel classification. As a result, this research investigated feature importance and three different classifiers in terms of their predictive power of background, border, and interior pixels in a dataset engineered from 55 raw fluorescence microscopy images of Pseudomonas Aeruginosa. Results reveal a small subset of useful features and that the Random Forest classifier outperformed both the k-NN and Support Vector Machine classifiers in terms of F1-score. Accurately classifying these pixels is an essential pre-requisite for labeling single bacteria. When comparing labeled images of commonly used segmentation techniques to this method we found it is less prone to segmentation error. Any segmentation errors translate into errors further down the bioimage analysis pipeline, making this experiment an important first step in making an algorithm which automatically segments and tracks bacteria. 1. Introduction The World Health Organization (WHO) mentions antimicrobial resistance (AMR) as one of the main threats to global health in 2019. As it hinders both prevention and effective treatment of an expanding variety of common infections caused by microbes, AMR is a relevant threat to all layers of modern-day civilization (World Health Organization, 2019). One phenomenon which has previously been explained as causing bacteria to be increasingly resistant to antimicrobial drugs, is the formation of biofilms (Hall-Stoodley & Stoodley, 2009). Knowledge of the early stages of bacterial biofilms formation, in which single free-swimming bacteria attach to a surface and continue to form antimicrobial resistant microcolonies, is relatively limited (Conrad et al., 2011; Gibiansky et al., 2013; Tolker-Nielsen, 2015). Studying this phenomenon almost always consists of the analysis of time-lapse fluorescence microscopy images (Tolker-Nielsen, 2015). Technological advances resulted in increased amounts of data being available (M. Wang, 2010). However, manual analysis is laborious and time-consuming, and automation is often not generalizable (Haubold et al., 2016a; Q. Wang, Niemi, Tan, You, & West, 2010). Therefore, the aim of this project is assisting in the development of a tool which automatically segments individual bacterium in time-lapse microscopy images of the early stages of biofilm development. Potentially the most important characteristic of biofilms is their resistance to antimicrobial drugs. There are, however, several applications of biofilms which also are worth discussing. Firstly, biofilms forming on indwelling medical devices are causing device associated infections. For example, biofilms have been found on both central venous and urinary catheters, prosthetic heart valves, and artificial hip prosthesis. Secondly, several diseases have been reported as caused by microorganisms forming biofilms. These include native valve endocarditis, otitis media, chronic bacterial proctitis, cystic fibrosis, and periodontitis. Thirdly, they have also been linked to food and water contamination when forming in the environments were these products are processed (Petersen, 2009). Finally, even though biofilms are often considered as negatively influencing human life, there are also cases where they can be linked to more positive outcomes. Examples include wastewater treatment, remediation of contaminated soil and groundwater, and microbial leaching (extracting precious metals from ores with biofilms instead of chemicals) (Cunningham, Lennox, & Ross, 2009) Considering the importance to study biofilms, limited knowledge of the early stages of their formation and the challenges in analyzing fluorescence microscopy images, this research focusses on answering the following research questions: RQ1: Which set of features is most important for classifying pixels in fluorescence microscopy images of pseudomonas aeruginosa? RQ2: What is the best classifier for segmenting pseudomonas aeruginosa, and how accurate is it? 2. Related Work This section provides an overview of previous work related to the analysis of images of the early stages of biofilm development. 2.1 Brief history of biofilms Dutch businessman and scientist Antonie van Leeuwenhoek, also sometimes referred as the father of Microbiology, is believed to have first discovered biofilms when he observed the bacteria in his own dental calculus in the 17th century (Costerton, Geesey, & Cheng, 1978; Slavkiny, 1997;Costerton, Stewart, & Greenberg, 1999). It was not until 1969, when Jones et al. developed a technique for staining and examining biofilms by electron microscopy, that a polysaccharide matrix surrounding the individual cells was found (Jones et al., 1969; Rodgers, Zhan, & Dolan, 2004). Today, biofilms are described as a colony of microorganisms encased in an, primarily, extracellular polymeric substance (EPS) matrix which are irreversibly attached to a surface (Costerton et al., 1999; Hall-Stoodley & Stoodley, 2009; Pamp & Tolker-Nielsen, 2007; Rodgers et al., 2004). Biofilms have been previously found on many different surfaces including teeth, medical implants, and streambeds(Rodgers et al., 2004; Vidyasagar & Nagarathnamma, 2018). 2.2 Analyzing time-lapse fluorescence microscopy images Time-lapse microscopy is a widely used technique in analyzing pseudomonas aeruginosa on the single-bacteria level(Chang, Yokota, Abe, Tang, & Tasi, 2017; Lee et al., 2018; M. Wang, 2010). By recording observations at a set interval (e.g. 15 minutes), it allows for studying processes over a long period of time. Time-lapse microscopy is also considered the only technique allowing the accurate observation of dividing cells by enabling researchers to construct lineage trees(Etzrodt, Endele, & Schroeder, 2014; Ulman et al., 2018). Analysis of the images