Acknowledgements

Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Fully 3D Conference 2015

Reconstruction of Difference using Prior Images and a Penalized-Likelihood Framework

Amir Pourmorteza, Hao Dang, Jeffrey Siewerdsen, J. Webster Stayman

Department of Biomedical Engineering, Johns Hopkins University

Johns Hopkins University Schools of Medicine and Engineering

Acknowledgements

AIAI Laboratory Advanced Imaging Algorithms and Instrumentation Lab aiai.jhu.edu [email protected] I-STAR Laboratory Imaging for Surgery, Therapy, and Radiology istar.jhu.edu [email protected]

Faculty and Scientists Clinical Partners Students Industry Partners Tharindu De Silva Junghoon Lee Qian Cao Lyn Hibbard Grace Gang John Wong Hao Dang Xiao Han Aswin Mathews Sarah Ouadah Markus Eriksson Amir Pourmorteza Sureerat Reaungamornrat Himu Shukla Jeffrey Siewerdsen Steven Tilley II Alejandro Sisniega Ali Uneri J. Webster Stayman Jennifer Xu Shiyu Xu Thomas Yi Wojciech Zbijewski Funding This work was support, in part, by an academic-industry partnership grant from Elekta.

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 1 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Sequential Imaging: IGRT

Planning MDCT Subsequent Cone-Beam CTs

. . .

High-fidelity data Low-fidelity data High exposure Less radiation exposure per scan

Sequential Imaging: IGRT

Planning MDCT Subsequent Cone-Beam CTs

. . .

High-fidelity data Low-fidelity data High exposure Less radiation exposure per scan

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 2 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Sequential Imaging: Brain Perfusion and Cardiac CT

Low-fidelity High-fidelity Low exposure High exposure

Video from: J. C. Rios, M. Luttrull, E. G. Stein, L. N. Tanenbaum.Time resolved - 4D CT Angiography: Applications and Protocols. ECR 2011

Sequential Imaging and Prior Knowledge

• Prior-image-based reconstruction : – Prior image in regularization term • PICCS*: Prior Image Constrained Compressed Sensing • PIRPLE**: Prior Image Registration in Penalized Likelihood Estimation

– Prior image in data fit term • Reconstruction of Difference (RoD)

• The primary objective in some sequential imaging studies is to assess the difference in anatomy.

*: Chen, Guang-Hong, Jie Tang, and Shuai Leng. "Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets." Medical physics 35.2 (2008): 660-663. **: Stayman, J. Webster, et al. "PIRPLE: a penalized-likelihood framework for incorporation of prior images in CT reconstruction." Physics in medicine and biology 58.21 (2013): 7563.

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 3 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Difference Model for the Image Volume

= + ∆

Current Anatomy Prior Anatomy Change in Anatomy 0.03 ∆ 0.025

0.02

0.015

0.01

0.005

Difference Model for the Image Volume

= W + ∆

Current Anatomy Prior Anatomy Change in Anatomy 0.03 ∆ 0.025

0.02

0.015

0.01

0.005

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 4 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Image Reconstruction using Prior Images PIRPLE • Integrates prior image through a penalty term. {̂, } = argmin − ; + ∥Ψ∥1 + ∥Ψ − ∥1 ∈ℝ Data Fit Term Roughness Prior Image Registration Penalty term Penalty Term

Reconstruction of Difference (RoD) • Integrates prior image/projections through the forward model.

{̂∆, } = argmin − ∆, ; , + ∥Ψ∆∥1 + ∥∆∥1 ∆∈ℝ Data Fit Term Roughness Prior Magnitude Penalty term Penalty Term

Reconstruction of Difference (RoD) • Integrates prior image/projections through the forward model.

{̂∆, } = argmin − ∆, ; , + ∥Ψ∆∥1 + ∥∆∥1 ∆∈ℝ Data Fit Term Roughness Prior Magnitude Penalty term Penalty Term

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 5 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Objective Function Optimization

Forward Model: = − = W + ∆

= −∆ −W

Registration update Image update

= ℎ(∆) . −W = . −∆

= arg min Φ ; , , = arg min Φ ∆; , , ∈ℝ ∆∈ℝ

** find using 3D-2D find ∆ using OS-SPS registration, and BFGS *

Φ(∆, ; , ) = − ∆, ; , + ∥Ψ∆∥1 + ∥∆∥1

*: Stayman, J. Webster, et al. "Model-based tomographic reconstruction of objects containing known components." Medical Imaging, IEEE Transactions on 31.10 (2012): 1837-1848. **: Erdogan, Hakan, and Jeffrey A. Fessler. "Ordered subsets algorithms for transmission tomography." PMB, 44.11 (1999): 2835.

Objective Function Optimization

Forward Model: = − = W + ∆

= −∆ −W

Registration update Image update

= ℎ(∆) . −W = . −∆

= arg min Φ ; , , = arg min Φ ∆; , , ∈ℝ ∆∈ℝ

** find using 3D-2D find ∆ using OS-SPS registration, and BFGS *

Φ(∆, ; , ) = − ∆, ; , + ∥Ψ∆∥1 + ∥∆∥1

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 6 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Objective Function Optimization

Forward Model: = − = W + ∆

= −∆ −W

Registration update Image update

= ℎ(∆) . −W = . −∆

= arg min Φ ; , , = arg min Φ ∆; , , ∈ℝ ∆∈ℝ

** find using 3D-2D find ∆ using OS-SPS registration, and BFGS *

Φ(∆, ; , ) = − ∆, ; , + ∥Ψ∆∥1 + ∥∆∥1

Digital Phantom

• Digital Phantom: Derived from high-fidelity CBCT data • New measurements made from: Prior + tumor • spherical tumor – 10.5 mm diameter – Attenuation 0.020 mm-1 • C-Arm Geometry: SAD = 77.5, SDD = 118.3 cm, 720 projections over 360° • Poisson measurement noise: b = 102 : 101/2 : 105 (photons)

+ =

Prior Tumor New measurements

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 7 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Penalty coefficient optimization

ReconstructedI =10000 # projections:180 Difference I =10000 RMSE # projections:180 0 0 -3 x 10 0.02 4.5 4.5 2.5 4 0.015 4 Magnitude ∥ ∥ 3.5 3.5 2 ∆ 1 0.01 3 3 ) ) M M   1.5 ( ( 2.5 0.005 2.5 10 10 log log 2 2 0 1 1.5 1.5 *

1 -0.005 1 0.5

0.5 0.5

-0.01 0.5 1 1.5 2 2.5 0.5 1 1.5 2 2.5 log ( ) log ( ) 10 R Roughness ∥Ψ∆∥1 10 R

Local vs. Global Acquisition/Reconstruction

• If difference is spatially limited

– ∆ is negligible outside VOI

• Local Reconstruction – Usually not possible in Model-Based Reconstruction – Computational speedup ∆ • Local (truncated) Acquisition – Radiation dose reduction 1

= −W −∆

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 8 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Global vs. Local Acquisition

Global RoD Local RoD

0.045

0.04

0.035

0.03

0.025

0.02

0.015

0.01

0.005

-3 x 10 ̂∆ +

2 Fluence = 104 (photons) ̂∆ 0 180 projections RMSE: 4.19 4.02 ×10-4 mm-1

Performance: Sparse Acquisitions

-3 x 10 5 PL RoD 4 ) 1 - 3

2 RMSE RMSE (mm 1 Fluence= 104 (photons) 0 0 100 200 300 400 # of projections

PL 0.045 0.04 ̂ 0.035

0.03

0.025 RoD 0.02

0.015

̂∆ + 0.01

0.005

16 24 45 90 180 360

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 9 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Performance: Sparse Acquisitions

-3 x 10 5 PL RoD 4 ) 1 - 3

2 RMSE RMSE (mm 1 Fluence= 104 (photons) 0 0 100 200 300 400 # of projections 0.03 PL 0.045 0.04 0.02

0.035 ̂ − 0.010.03

0.025 RoD 0 0.02

-0.010.015

̂∆ 0.01 -0.02 0.005

16 24 45 90 180 360

Performance: Varying Fluence

-3 6 x 10 PL 5 RoD ) 1 - 4

(mm 3

2 RMSE

1 # of projections: 90 0 2 3 4 5 10 10 10 10 Incident Fluence

0.03 PL 0.045 0.020.04 0.035 0.01 0.03

00.025

RoD 0.02 -0.01 0.015 + ∆ -0.020.01

0.005 -0.03 102 105 (photons)

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 10 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Performance: Varying Fluence

-3 6 x 10 PL 5 RoD ) 1 - 4

2 RMSE RMSE (mm

1 # of projections: 90 0 2 3 4 5 10 10 10 10 Incident Fluence

0.03 PL

0.02 −

0.01

0 RoD -0.01 ∆ -0.02

-0.03 102 105 (photons)

Performance of Likelihood-based Registration

• Prior image transformed by a known rigid transform. – Single translation in x, y, z, or – Single rotation around x, y, z • Values are chosen randomly from a bimodal distribution. • RMSE of final reconstruction was calculated with respect to ground truth.

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 11 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Performance of Likelihood-based Registration

RMSE vs perturbations in Tx RMSE vs perturbations in Ty RMSE vs perturbations in Tz

0.012 0.012 0.012 ) ) ) 1 1

0.01 1 0.01 0.01 - - - 0.008 0.008 0.008 0.006 0.006 0.006 RMSE (mmRMSE (mmRMSE RMSE (mmRMSE 0.004 0.004 0.004 0.002 0.002 0.002 0 0 0 -60 -40 -20 0 20 40 60 -60 -40 -20 0 20 40 60 -60 -40 -20 0 20 40 60 Shift (mm) Shift (mm) Shift (mm)

RMSE vs perturbations in Rx RMSE vs perturbations in Ry RMSE vs perturbations in Rz

0.012 0.012 0.012 ) ) ) 1 0.01 0.01 1 0.01 1 - - - 0.008 0.008 0.008 0.006 0.006 0.006

RMSE (mmRMSE 0.004 (mmRMSE 0.004 (mmRMSE 0.004 0.002 0.002 0.002 0 0 0 -100 -50 0 50 100 -100 -50 0 50 100 -100 -50 0 50 100 Rotation () Rotation () Rotation ()

CBCT Test Bench Studies

X-ray Technique: 100 kVp, 453 mAs Projection Data: 720 angles over 360° Prior Image: PL reconstruction of “no tumor” data Current measurements: Phantom + 12.5 mm acrylic sphere “tumor” inside nasal cavity Additional noise added using Poisson model A random 3D rigid transform of the prior image within the capture range Reference “Truth” Data: High-fidelity, PL reconstructions of prior and current anatomy.

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 12 Amir Pourmorteza Fully 3D Recon 2015 [email protected] May 31 -June 4

Test-Bench Data Reconstructions FBP PL RoD 0.03

0.025

0.02

0.015

0.01

0.005

0 ̂ ̂ + ∆ -3 x 10 5

0 | − | |̂ − | |̂∆|

RMSE: 1.97 1.55 1.27 ×10-3 mm-1

Simulated Fluence = 5000 (photons) Original Fluence = ~25000 (photons)

Future Directions

̂∆ ≠ ̂ −

• Lower contrast and smaller changes • Prior images from different modalities • Non-rigid transforms • Other “difference-based” penalty terms

The I-STAR Laboratory (istar.jhu.edu) and The AIAI Laboratory (aiai.jhu.edu) Department of Biomedical Engineering Johns Hopkins University 13