3D for Breast Cancer Diagnosis N. V. Ruiter, M. Zapf, T. Hopp and H. Gemmeke

INSTITUTE FOR DATA PROCESSING AND ELECTRONICS

1 KIT – University of the State of Baden-Wuerttemberg and 16.01.2017 National Laboratory of the Helmholtz Association www.kit.edu Karlsruhe Institute of Technology

Campus South Campus North Technical University Helmholtz Research Center

University of Karlsruhe and Helmholtz Research Center Karlsruhe

2 16.01.2017 Breast cancer

Most common cancer of women in western world (every 10th woman)

WHO cancer statistics 2012 (GLOBOCAN 2012) Challenge: Early diagnosis

…

X-ray Sonography MRI Screening Symptomatic patients

3 16.01.2017 What is USCT?

Basic idea Surround object with (unfocused) ultrasound transducers in a fixed setup Breast imaging in fixed setup Diagnostic value Reproducible 3D images with ultrasound Images three modalities concurrently: Reflection High quality “B-scans” Speed of sound and attenuation Quantitative information Example setup

Simplified from Greenleaf et al., 1981

4 16.01.2017 The beginnings

First attenuation imaging (Dussik, 1946): Not so First “USCT“ device (Holmes et al., 1954): Slice successful imaging of brain ventricles image of the neck, compounding device

5 16.01.2017 Image sources: ob-ultrasound.net, Szabo: Diagnostic Ultrasound: Inside out. Current state of the art

2D ring systems: Kamanos Cancer Institute (Delphinus Medical Technologies) University of Southern California

(MastoScopia) Sonovue 2,5D planar array system: University of Utah (TechniScan)

Anais CVUS

Problem: Anisotropic 3D point spread function

3D PSF 2D system

6 16.01.2017 KIT 3D USCT

3D aperture

Vision0 for 3D USCT … • as0.02 harmless as diagnostic

ultrasound0.04 Z / m / Z 0.06 • as affordable as X-ray 0.08

mammography0.1 KIT 3D USCT system 0.04 -0.04 0.02 • as -0.02good as MRI 0 0 -0.02 Y / m 3D point spreadX /function m 0.02 in breast volume (scaled by 10) 0.04 -0.04

7 16.01.2017 Schwarzenberg, Ruiter et al.: Aperture optimization for 3D ultrasound computer tomography, 2007. Challenges

Transducer Parallel data Clinical 3D Medical development acquisition applicability reconstruction analysis

Transducer design Parallel channels 3D aperture Ultrasound physics Image fusion

Electronics Electronics Technical saftey Algorithms Multimodality

Transducer Digital processing Biocompatibility Acceleration Analysis

8 16.01.2017 Patents: PCT/EP2012/000640 (pending), EP 2539870 A2 (2011), EP 2 056 124 (2009), EP 1 755 837 (2004), US.6.786.868 (2003).

Patents: Method for reducing ultrasonic data: PCT/EP2012/000640 (pending) Method for reconstruction of the internal structure of a sample body by means of reflection and scattering signal: EP 2539870 A2, 2011. Aperture Optimization for 3D Ultrasound Computer Tomography: EP 2 056 124, 2009. Ultrasound transducers: EP 1 755 837, 2004. Ultrasonic Tomograph: US.6.786.868, 2003. How does it work? Ultrasound and soft tissue

Wave equation for inhomogeneous water like materials: 2

2 2 퐾0 휌 푥 휇 푥 1 훻 푝 푥 + 푘0 + 푖 푝 푥 − 훻휌 푥 훻푝 푥 = 0 휌0 퐾 푥 푘0 휌 푥

Three physical properties influence wave propagation: Density 휌, compressibility 퐾 and absorption 휇 Typically reconstructed in USCT: qualitative acoustical impedance Z = 휌푐,

퐾 speed of sound c = , 휌 attenuation α = 휇 + 휇푠 2D simulation of interaction with point scatterers

9 16.01.2017

푝: ultrasound pressure, 푘: wave number, 휇푠: scatter How does it work? Data acquisition

Receiver A-scan

p(t)

Pressure

Transmission Reflection Time t

Emitter Acquired information Sound speed Attenuation Reflection

10 16.01.2017

Transducer array systems (TAS)

Center frequency (bandwidth) 2.5 MHz (1.5 MHz) Opening angle 38° at -6dB (± 1.5°) 3 vs. 6dB ??????

Structured Piezo composites 0.64 mm² 1.5 MHz at -3dB (± 0.15 MHz) Emitters / receivers per TAS 4 emitters and 9 receivers

Emitters (red) and receivers (blue) per TAS Frequency vs. opening angle TAS

Embedded TAS electronics

11 Kohout, Ruiter et al.: Simulation und Entwicklung von Ultraschallwandlersystemen für die 3D-USCT, 2012. 16.01.2017 Zapf, Ruiter et al.: Evaluation of chirp and binary codes as excitation pulses for 3D USCT transducers, 2009. Data acquisition system

Number channels 480 AD conversion 12 Bit @ 20 MHz Memory 80 GB Measurement time 10 s – 6 min

IPE-DAQ-V4 First Level Trigger Board Cabling

12 16.01.2017 Menshikov et al.: RTM-modules for waveform digitization, 2015. Kopmann et al.: FPGA-based DAQ system for multi-channel detectors, 2008. Listen to USCT data

13 N.V. Ruiter – 3D USCT 16.01.2017 Image reconstruction

Resolution/accuracy Example: N³ = 1000³ voxels from N² A-scans

Full-wave Paraxial inversion 휆 inversion

2 Born / Rytov inversion SAFT

Eikonal Iterative ray inversion Ray 퐷휆 based tomo- tomography Complexity graphy

O(N3log(N)) O(N5) O(N6) O(N7)

Althaus: On acoustic tomography using paraxial approximations, 2015. 14 Özmen: Ultrasound Imaging Methods for Breast Cancer Detection, 2015 (TU Delft). 16.01.2017 Dapp: Abbildungsmethoden für die Brust mit einem 3D-Ultraschall-Computertomographen, 2013. Hardt: Distributed Simulations for 3D USCT. Acoustic wave simulations for a new breast cancer imaging device, 2012. Intel® Core™ i7-2700K Processor (8M Cache, up to 3.90 GHz), 4 Kerne, 8 Acceleration of signal and image processing Threads Leistungsaufnahme 95 W

Signal processing ART 3D SAFT

21.1 CPU

25

14.5 to 20 7.2 10

15 relative relative 10 10.1 11.3 5 4.8 10.8 6.1 0 6.8 Acceleration Acceleration HW-Typen korrekt einführen FPGA Xilinx Virtex-6 FPGA Xilinx Virtex-7 GPU GTX 580 GPU GTX Titan • Hoch optimiert

• Nicht Matlab integriert

GPU server with 8 GPU GTX Titan: 3D SAFT in 16 min • Keine Interpolation • Nur Standard-Volumengröße von 1024 15 16.01.2017 Kretzek: Erweiterung der Synthetic Aperture Focusing Technique für die 3D-Ultraschall-Computertomographie, 2015. Birk: Effiziente Datenverarbeitung auf heterogenen Rechnerarchitekturen für die 3D-Ultraschall-Computertomographie, 2014.

Example: 444² x 266 voxels (0.9 mm)³ and 10 million A-scans Performance: 109 ∙ voxels ∙ A-scans/s (GVA/s) Clinical study Jena

Universitätsklinikum Jena (Prof. W. A. Kaiser)

Aim: Test device in clinical setting  prepare larger study

10 patients, all with suspicious lesions 2 implants Prof. Kaiser in Jena 4 cancers Papilloma, fibroadenoma, mastopathy, cyst

End of clinical trial: September 2014

16 16.01.2017 Ruiter et al.: First results of a clinical study with 3D Ultrasound Computer Tomography, 2013. Patient 1: Healthy

Coronal plane Transversal plane

(detail view) Registered MRI T1-weighted

USCT Reflectivity

17 16.01.2017 Patient 2: Inflammatory carcinoma

Coronal plane Sagittal plane Transversal plane

Registered MRI T1 contrast enhancement

USCT image fusion: Reflectivity + sound speed (color-coded)

Sound speed

1300 m/s 1600 m/s

18 16.01.2017 Patient 3: Multicenter carcinoma

Coronal plane Sagittal plane Transversal plane

Registered MRI T1 contrast enhancement

USCT image fusion: Reflectivity + sound speed (color-coded) threshold: 1500m/s

Sound speed

1300 m/s 1600 m/s

19 16.01.2017 3D data

20 16.01.2017 Summary pilot study in Jena

3D USCT was applicable in clinical setting (~ 1 patient/h) First images very were encouraging Speed of sound seem to give best cancer detection

Mean patient movement: 3 mm Breast positioning critical USCT in Jena

Major system updates: Data acquisition time: 8 min  6 min New patient interface: + 1 cm

Image analysis

21 16.01.2017 Ruiter, et al: First Results of a Clinical Study with 3D Ultrasound Computer Tomography, 2013. Clinical study Mannheim

Universitätsmedizin Mannheim (Prof. S. Schönberg)

Aims of study Does USCT give comparable diagnoses to MRI? Analyze different lesion types

200 patients

Start: October 2015

USCT in Mannheim

22 16.01.2017 Multimodal imaging and classification Supports comparability of USCT to MRI and X-ray mammography Enables automatic analyses

Example: USCT to X-ray registration

Projection of Mammogram FEM model Deformed FEM model deformed USCT-SOS

Data from Karmanos Cancer Institute

23 Hopp: Multimodal Registration of X-Ray Mammograms with 3D Volume Datasets, 2012. 16.01.2017 Ruiter: Registration of X-ray mammograms and MR-volumes of the female breast based on simulated mammographic deformation, 2004.

Multimodal imaging and classification

Can we reproduce Greenleaf’s results?

Mammogram

Segmented Mammogram Ground truth: segmented mammogram

1600 m/s

Sound Sound speed Registered 1300 m/s sound speed image Classification by Support Vector Machine Feature extraction after registration (9 patients, sound speed and attenuation averaged)

Data from Karmanos Cancer Institute

24 Hopp, Ruiter et al.: Image fusion of Ultrasound Computer Tomography volumes with X-Ray mammograms 16.01.2017 using a biomechanical model based 2D/3D registration, 2014. USCT tissue classification in X-ray mammogram

v < 1460 m/s 1460 m/s < v < 1490 m/s v > 1490 m/s 1550

]

1500

[m/s

1450

sound

of

1400 Speed

1350 Mammogram Fatty tissue Glandular tissue Lesion

Data from Karmanos Cancer Institute

25 Hopp, Ruiter et al: Evaluation of breast tissue characterization by ultrasound computer tomography 16.01.2017 using a 2D/3D image registration with mammograms, 2013. Summary

USCT is a new imaging method for at early Vision for 3D USCT … breast cancer diagnosis • as harmless as diagnostic ultrasound

• as affordable as X-ray KIT 3D USCT: mammography first clinically applicable full 3D USCT • as good as MRI 3D USCT during pilot study First pilot study successful, second study started

MRI subtraction USCT fused slice 3D USCT III: pattern #2 of TAS with 18 transducers, chi2=1730.2 21 18.9 pattern #2 of TAS with 13 transducers, chi2=179.2 0.03 16.8

Faster DAQ 0.027 14.7 12.6 0.024 10.5

0.021 8.4 6.3 0.018 Optimized image quality 4.2 0.015 2.1

m 0 m

0.012 /

y −2.1 0.009 −4.2 −6.3 0.006 −8.4 0.003 −10.5

0 −12.6 0 0.006 0.012 0.018 0.024 0.03 −14.7 −16.8 −18.9 −21 −21 −16.8 −12.6 −8.4 −4.2 0 4.2 8.4 12.6 16.8 21 x/mm 26 16.01.2017 Ruiter et al.: Optimization of the aperture and the transducer characteristics of a 3D Ultrasound Computer Tomography System, 2014.

Ausblick weg? Nochmal Vision für USCT? Thank you!

We acknowledge support of this project by Deutsche Forschungsgemeinschaft (DFG)

Algorithms / Imaging / Image Processing N. V. Ruiter, M. Zapf, T. Hopp, W.Y. Tan, H. Gemmeke, et al.

Hardware acceleration E. Kretzek, M. Balzer, et al.

Transducers M. Zapf, H. Bouquet, et al.

DAQ and Hardware D. Tscherniakhovski, A. Menshikov, et al.

Design and Mechanics L. Berger, B. Osswald, T. Piller, W. Frank, et al.

3D USCT in Mannheim Contact: [email protected]

27 N.V. Ruiter – Patient motion in 3D USCT 16.01.2017