A Multi-Resolution Textural Approach to Diagnostic Neuropathology Reporting

Supporting Materials for

A Multi-resolution Textural Approach to Diagnostic Neuropathology Reporting

Mohammad Faizal Ahmad Fauzia*, Hamza Numan Gokozanb*, Brad Elderc, Vinay K. Puduvallid, Christopher R. Piersonb,e,f, José Javier Oterob, Metin N. Gurcang

aFaculty of Engineering, Multimedia University, Cyberjaya, Selangor, Malaysia; bDepartment of Pathology, The Ohio State University, Columbus, Ohio, USA; cDepartment of Neurological Surgery, The Ohio State University, Columbus, Ohio, USA; dDivision of Neuro-oncology, The Ohio State University Wexner Medical Center, Columbus, Ohio, USA; eNationwide Children’s Hospital Department of Pathology and Laboratory Medicine, Columbus, Ohio, USA; fDivision of Anatomy, The Ohio State University, Columbus, OH, USA; gDepartment of Biomedical Informatics, The Ohio State University, Columbus, Ohio, USA

*These authors contributed equally to the work.

Corresponding Authors:

Dr. Mohammad Faizal Ahmad Fauzi Dr. Metin Gurcan Faculty of Engineering 250 Lincoln Tower Multimedia University 1800 Cannon Drive Jalan Multimedia Columbus, OH 43210 63100 Cyberjaya Office: 320-K Lincoln Tower Selangor, MALAYSIA. Tel: (614) 688-9857 Tel: +603-8312 5330 Fax: (614) 688-6600 Fax: +603-8318 3029 Email: [email protected] Email: [email protected] lab: http://bmi.osu.edu/~cialab

Dr. José Javier Otero Assistant Professor The Ohio State University College of Medicine Department of Pathology Division of Neuropathology 4169 Graves Hall 333 W 10th Avenue Columbus, OH 43210 Tel: 614-685-6799 Fax: 614-292-5849 Email: [email protected] Supplemental Materials and Methods

Discrete Wavelet Frames

Two-dimensional wavelet transform performs a spatial/spatial-frequency analysis on an image by repeatedly decomposing the image in the lower frequency bands, followed by sub-sampling [35]. A full wavelet decomposition of an image results in an array of wavelet coefficients, the same shape and size as the original image. One level wavelet decomposition of an image results in four separate channels, namely LL (low frequency band), LH, HL (medium frequency bands) and HH (high frequency band) channels. The decomposition can be performed repeatedly on the LL channel until some specific criteria is met. Because of the decomposition structure, the wavelet transform is also known as pyramidal wavelet transform (PWT). The number of channels is given by 3l+1, where l is the number of decomposition level.

The discrete wavelet frames [36-39] is nearly identical to the standard wavelet transform, except that it upsamples the filters, rather than downsampling the image. While the frame representation is over- complete and computationally more intensive than PWT, it holds the advantage of being translationally invariant. This is particularly important in our work since the glial processes (and the nuclei) can appear anywhere in the image. In our previous work [40-42], we found that DWF is the best texture feature method compared to several others such as PWT, tree-structured wavelet transform (TWT), discrete cosine transform(DCT), Gabor filter, Laws’ filter, and multiresolution simultaneous auto regressive model. Given an image, the DWF decomposes its channel using the same method as the wavelet transform, but without the subsampling process. This results in four filtered images with the same size as the input image. The decomposition is then continued in the LL channels only as in the wavelet transform, but since the image is not sub-sampled, the filter has to be upsampled by inserting zeros in between its coefficients. To compute the features, the mean energy of each channel is used and is given as:

in which M and N are the number of rows and columns of the image, and Wk is the k-th channel or filtered images. The wavelet frames allow the algorithm to analyze the content of the image at a different frequency range, thus making texture classification possible.

k-Nearest Neighbor Classification

The k-nearest neighbor algorithm (k-nn) is a non-parametric method for classifying images or objects based on closest training examples in the feature space. The advantage of k-nn is that it can provide a degree of confidence on the classification. As an example, using k=5, for an image to be classified into a particular class, at least three of the five nearest neighbors must be of the particular class. This constitutes to 60%, 80% or 100% of the neighbors, which can be used as a reference to gauge the confidence level of the classification. The normalized Euclidean metric is used so as to prevent a particular dimension to dominate the distance measure, and is given as:

in which i and j denote two image patterns, f(k) is the k-th component features, and σ(k) is the standard deviation of the distribution of features f(k) in the entire database and is used to normalize the individual feature components.

Sensitivity, Precision, and Accuracy:

Sensitivity and precision are defined by:

(3) (4)

The classification accuracy is defined by:

(5) Supplemental Classification of Glioblastoma and Metastatic Neoplasm

Region of Interest Segmentation

A key diagnostic feature capable of determining if cells represent neoplastic cells of a metastasis versus a high grade glioma is the presence of glial processes between cell bodies that appear as a meshwork of eosinophilic fibers in H&E stained cytologic preps. From an image analysis perspective, these glial processes appear as anisotropic thin linear structures interspersed between cells as shown in the first 2 rows of Figure 1(a). Metastasis tissue, on the other hand appears more homogeneous in the background region, as can be seen in the last 2 rows of Figure 1(a). The homogeneous background is due to lack of glial-type filaments; therefore, our approach to classification was first to segment our region of interest, which is the nuclear free region of the tissue. For GB tissue, these refer to the anisotropic thin line segments, while for the metastasis tissue, the homogeneous segments. Since nuclei in H&E stained images correspond to dark blue pixels, the segmentation can be done by the visually meaningful decomposition-based segmentation procedure [23-24]. To ensure proper feature extraction, morphological erosion is carried out on the segmented regions of interest. In this workflow, we use a disk structuring element with radius 5. We conclude that the segmentation stage is critical since the inclusion of the nuclei regions during feature extraction contaminates the features, ultimately resulting in a less accurate classification.

Feature Extraction and Classification

Four levels of decomposition are carried out using Daubechies four-tap wavelet, which results in 13 channels of filtered images. Because there is no preferred orientation of the glial processes, the LH and

HL channels at each level can be combined, reducing the number of channels and dimension of the feature vectors to nine. The four-tap wavelet basis combined with the disk-structuring element of radius five ensures that none of the coefficients from the non-ROI region are considered in the feature vector computation. Given the segmented region of interest from the previous stage, discrete wavelet frames

(DWF) is applied to the image, but the mean energy of each channel is calculated only on the regions of interest. For our nine-dimensional feature vectors derived from the DWFs, the distances or dissimilarities between the test and training images are computed based on the normalized Euclidean distance metric and k smallest distances are considered to classify the test image. We conclude that the discrete wavelet frames texture features, fine-tuned as above, is able to accurately distinguish the glial-type filaments present in GB from the more homogeneous background in metastasis tissues. Supplemental p53 Immunohistochemistry Reporting

Automated Nuclear Detection

Supplemental Figure 3(a) shows the flowchart of our cell detection process. Given a digitized p53 stained image, we first convert the image into gray scale by only considering the luminance channel. Otsu thresholding is then applied to convert the luminance image into binary, separating the cells (blobs) from the background. The binary image is then cleaned by filtering very small and isolated blobs (blobs smaller than a certain minimum threshold, minarea). While Otsu thresholding is a very simple and straightforward technique in detecting dark objects from a light background, due to the lack of spatial information, it is prone to grouping closely connected cells together as a single cell, which will affect the cell counting process significantly. To address this, we developed a novel adaptive thresholding scheme for any detected regions with total area greater than a particular maximum threshold maxarea. The adaptive thresholding checks if a large detected blob consists of multiple cells by searching for potential valley within the blob, and proceeds to segment the blobs further if they do. For a 40x magnification image used in our experiment, a suitable minarea and maxarea threshold was found to be 300 and 1000 pixels (~70 and 230 µm2), respectively.

Classification of Positive and Negative Cells

Supplemental Figure 3(b) shows the flowchart of our positive-negative cell classification process. The classification of the cells as positive or negative was based on the intensity and color of the cells. Hence, the image is first converted into HSV (Hue-Saturation-Value) color model. For each of the detected cells found in the previous section, their centroid is determined and 32x32 pixel blocks are extracted around each centroid. The weighted Hue and Value are calculated for each block and are used for classifying the cells. The weights used are inversely proportional to the pixels’ distance to the centroid, with those closer to the center of the block receiving higher weight, and those further from the center receiving less weight. Negative p53-stained cells tend to be blue (higher Hue) and less intense (higher Value), while positive p53-stained cells tend to be brown (lower Hue) with varying intensities. Based on these properties, we developed a two-step classification rule:

1. If the weighted Value (wV) for a block is less than a particular threshold (darker), the block will

be classified as containing positive cell, regardless of its weighted Hue (wH).

2. Otherwise, the classification depends on weighted Hue, with wH less than a particular threshold

means the block contains positive cells, and wH greater than the threshold means it contains

negative cells.

From our experiments, suitable threshold value for both wH and wV for positive-negative cells classification is found to be 70.

Strong-Moderate-Weak Cells Classification

Supplemental Figure 3(c) shows the flowchart of our strong-moderate-weak cell classification process.

Unlike the positive-negative classification (Section 4.3), which can be distinguished based on intensity as well as hue information, classifying the strong-moderate-weak cells are much more challenging due to the more subjective appearance of the cells’ intensities. Nevertheless, we note that strongly stained cells exhibit more homogeneous intensities within the cell’s boundary, while weakly stained cells exhibit more varying intensities, hence are more “textured”. We therefore propose to utilize texture features on top of the intensity information to distinguish the three classes of cells’ strength, especially between the moderate and weakly stained cells. As mentioned previously, another challenge in classifying positive cells is that the staining intensities differ between images. Because of this, cells with similar intensities may belong to different strength class in different tissue. To address this, an adaptive thresholding approach in which the threshold value to be used varies depending on the image content is used. We propose a threshold that is inversely proportional to the average intensity of all the detected cells.

Cells with average intensity above the calculated threshold are automatically classified as strongly-stained cells. The remaining cells are then classified into the three classes by means of discrete wavelet frames to distinguish their “texturedness”. Three levels of decomposition are carried out for each

32x32 cell block, and the mean energy from each channel is used to create 10-dimensional features. The features are then compared to the training features by means of k-nearest neighbor classification. Unlike the leave-one-out strategy employed in the intra-operative consultation experiment in the previous section due to limited samples, we base our k-nearest neighbor classifier against 50 training samples each from the three classes. The 150 samples are extracted from images different from the test images. Several values of k was tested and k=23 is found to yield the highest accuracy. Supplemental Figures & Tables

(a) (b) (c)

(d) (e) (f)

Supplemental Figure 1: Example of incorrectly classified tissues (a)-(c) Glioblastoma classified as metastasis,

(d)-(f) Metastasis classified as Glioblastoma (a)

(c) (d)

Supplemental Figure 2: Training samples for the (a) strong, (b) moderate, (c) weak, and (d) negatively stained cells Detected p53 Images Cells

Determine the centroid for Otsu thresholding each detected cells

Filter small blobs Obtain 32x32 pixel blocks (

Compute the weighted hue (wH) Blobs> Yes and value (wV) for each 32x32 maxarea? block

Separate cells through adaptive thresholding No wV

Yes Yes wH

Detected Positive Negative Cells Cells Cells

WŽƐŝƟǀ Ğ t ĞĂŬ ĞůůƐ ĞůůƐ

ĞƚĞĐƚĞĚ ĂůĐƵůĂƚĞ t &ĂŶĚ Ŭ-ŶŶ D ŽĚĞƌĂƚĞ ǁ s хƚŚ ĞůůƐ ƚŚƌĞƐŚŽůĚ Ě ƚŚ ĐůĂƐƐŝĮ ĐĂƟŽŶ ĞůůƐ

^ƚƌŽŶŐ ĞůůƐ

Supplemental Figure 3: (a) Nuclear detection process from digitized pathology slides, (b) Positive-negative nuclear classification

process, (c) Strong-moderate-weak nuclear classification process Supplemental Table 1A: Demographic information from cytologic preparations (“smears”)

Smears Ag Sex Location Intraop diagnosis Final Diagnosis Notes

e S1 50 F Right frontal Infiltrating glioma (at Anaplastic S1*Smear Concerning

least grade 2)* oligodendroglioma for higher grade S2 63 M Left frontal Malignant neoplasm* GB S2* defer to permanent S3 68 M Right frontal High grade glioma GB S4 60 M Left occipital Malignant neoplasm* GB S4* defer to permanent S5 61 M Left temporal High grade glioma* GB S5* GB concerned S6 37 M Left frontal Infiltrating glioma Anaplastic

oligodendroglioma S7 54 F Right frontal Malignant neoplasm Metastatic S7* consistent with

* melanoma melanoma S8 66 F Left frontal Metastatic Metastatic colon S8* colon primary

adenocarcinoma* adenocarcinoma S9 56 F Right frontal Metastatic Metastatic breast S9* breast primary

carcimoma* carcinoma S10 62 M Spinal cord Metastatic carcinoma Metastatic renal cell

carcinoma S11 48 M Right Metastatic epithelioid Metastatic

occipital/ tumor pulmonary

Parietal adenocarcinoma S12 74 M Right GB GB

temporal S13 26 F Right frontal Slightly Oligodedroglioma S13* few atypical cells,

Hypercellular* Grade2 non diagnostic S14 62 F Right frontal Infiltrating glioma Anaplastic

astrocytoma S15 50 F Right frontal Malignant neoplasm* Metastatic lung S15* consistent with

carcinoma carcinoma S16 48 M Right frontal GB GB S17 82 M Left temporal Anaplastic GB

astrocytoma(G3) S18 68 M Lateral/ High grade glioma GB

medial

temporal lobe S19 48 M Right frontal GB GB S20 64 M Left frontal High grade glioma GB S21 68 M Left parietal GB GB S22 25 M Right Metastatic tumor Metastatic mixed

parietal/ malignant germ cell

occipital tumor S23 52 F Right Metastatic carcinoma Metastatic lung S23* neuroendocrine

cerebellar carcinoma* carcinoma consistent

with pulmonary primary

lesion S24 67 M Left clival Neoplastic tissue* Chondrosarcoma S24* Differential

includes

chondrosarcoma and

chordoma S25 60 M Left frontal Malignant neoplasm* Lymphoma S25* smear consistent

with lymphoma

Supplemental Table 1B: Demographic information for the p53 analysis

Ag Sex Location Final p53 stain reported in Notes

e diagnosis pathology report G1 46 M Left GB p53+(no further info)

frontal G2 58 M Left GB Scattered giant cells are +

frontal G3 77 F Left GB Immunoreactivity ranging from

occipital weak to strong G4 39 M Right GB Strongly diffusely positive

frontal G5 60 F Left GB Immunoreactivity to p53

temporal

-occipital G6 61 M Left GB Diffuse immunoreactivity to

temporal p53 G7 53 M Left GB Weak to moderate- 30%

frontal G8 70 F Right GB expression noted in 50% of tm

parietal cells G9 74 M Right GB expression noted in 80% of tm

temporal cells G10 50 F Left GB expression noted in 60% of tm

frontal cells G11 48 M Right GB majority of tm cells- strong

frontal reactivity G12 55 M Right GB majority of tm cells- moderate

frontal to strong reactivity G13 82 M Left GB strong reactivity- over 70%

temporal G14 64 M Left GB less than 10%

frontal G15 63 F Right GB moderate staining in occasional

frontal tumor nuclei G16 69 F Left GB diffusely positive

temporal G17 54 M Left GB diffusely positive

frontal G18 31 M Brain Anaplastic diffusely positive Location info not

mass* astrocytoma available-outside

consultation slide G19 82 M Right GB diffusely positive

frontal G20 73 M Right GB diffusely positive, strong

frontal reactivity G21 54 M Right GB majority of tumor cells stained-

temporal weak to moderate intensity