Improving the Efficiency of Manual Ground Truth Labeling Using Automated Segmentation

Hongzhi Wang PhD, Prasanth Prasanna MD, Jose Morey MD, Tanveer F. Syeda-Mahmood PhD Medical Sieve Group, IBM Almaden Research Center Anatomy Segmentation: labeling anatomical structures of interest in medical images

• Corner stone for quantitative image analysis Gold standard: manual segmentation

Manual tracing in ITK-SNAP Manual tracing in Amira Gold standard: manual segmentation

Manual tracing in ITK-SNAP Manual tracing in Amira • Very time consuming, hours or even days to annotate a single 3D volume Techniques for Assisting Manual Segmentation

• Interactive manual segmentation enhanced with interpolation techniques, e.g. MITK and Amira • Only a subset of 2D slices are manually annotated • Annotation for the full 3D volume is generated through interpolation • Annotation time is a fraction of standard manual segmentation, depending on percentage of manually annotated slices

• Semi-automatic segmentation, e.g. Pluta et al. 2009, Daisne & Blumhofer 2013 • Automatic segmentation produces initial annotation • Mistakes corrected by human experts • Most focusing on single anatomical structure • ~ 40% time reduction comparing to standard manual segmentation( Daisne & Blumhofer 2013) Aims

• Investigate full potential in time reduction by semi-automatic segmentation • Employing state of the art automatic anatomy segmentation algorithm

• Challenging multi-structure segmentation task • cardiac CT anatomy segmentation with 20 anatomical structures

• Comparison with interpolation-based interactive manual segmentation Data Description

• 33 cardiac CT studies 28 cases used for training automatic o segmentation o 5 testing cases for experimental validation

• 20 structures studied o Bone: sternum, vertebrae Artery/vein: pulmonary artery (left/right/trunk), o aorta (root/ascending/arch/descending), Superior/inferior vena cava Cardiac structure: Left/right ventricle/atrium, o left ventricular myocardium Valve : aortic valve, tricuspid valve, pulmonary o valve, mitral valve Experimental Setup

• Semi-Automatic Segmentation: o Automatic segmentation by multi-atlas label fusion o Manual correction by one clinician (PP) using Amira commercial software (FEI Corporate, Hillsboro, Oregon USA)

• Manual segmentation o produced by the same clinician one week after semi-automatic segmentation is finished using Amira with the interpolation technique Overview for Automatic Anatomy Segmentation

Training . . . Training Image 1 Image k Target Image

Registration Registration and Warping and Warping

Candidate Candidate Segmentation Segmentation

Joint Label Fusion

Initial Final Post Processing Segmentation Segmentation Overview for Automatic Anatomy Segmentation

Training . . . Training Image 1 Image k Target Image

Registration Registration Multi-Atlas Label Fusion and Warping and Warping

Candidate Candidate Segmentation Segmentation

Joint Label Fusion

Initial Final Post Processing Segmentation Segmentation Multi-Atlas Label Fusion

Atlases (Training anatomical volumes) registration Warped Atlases Given CT Scan

Image Registration Label Fusion

...... A t ti R lt Automatic segmentation: Post processing

• Remove small isolated segments • Ensure boundaries between right ventricle and pulmonary artery and boundaries between substructures of aorta do not cross axial planes.

image multi-atlas segmentation after post processing after manual correction

The colored anatomical structures are: sternum; right ventricle; pulmonary artery trunk; myocardium; aortic root; ascending aorta; descending aorta; left atrium; right atrium; vertebrae. Results: Leave-One-Out Performance of Multi-Atlas Segmentation

Inter-rater precision for non-valve structures

Automatic Segmentation Results: Overall Time Reduction

37% time reduction

statistically significant with p<0.0001 Conclusions and Discussion

• Multi-atlas anatomy segmentation is accurate enough to save time for manual segmentation, even with advanced interpolation tools

• Without post processing, manual correction does not save time! • Manually correcting small, isolated segments is time consuming! • Having smooth segmentation is important for efficient manual correction! Backup Material Results: Ratio of Corrected Voxels Results: Corrections are Heterogeneous with respect to Anatomical Structures