MEDICAL IMAGING INFORMATICS: Lecture # 1
Estimation Theory
Norbert Schuff Professor of Radiology VA Medical Center and UCSF [email protected]
UC Medical Imaging Informatics 2012 Nschuff SF VA Course # 170.03 Department of Slide 1/31 Radiology & Biomedical Imaging Objectives Of Today’ s Lecture
• Understand basic conceppgts of data modeling – Deterministic – Probabilistic – Bayesian • Learn the form of the most common estimators – Least squares estimator – Maximum likelihood estimator – Maximum a-posteriori estimator • Learn how estimators are used in image processing
UCSF Department of VA Radiology & Biomedical Imaging What Is Medical Imaging Informatics?
• Signal Processing – Digital Image Acquisition – Image Processing and Enhancement • Data Mining – Computational anatomy – Statistics – Databases – Data-mining – Workflow and Process Modeling and Simulation • Data Management – Picture Archiving and Communication System (PACS) – Imaging Informatics for the Enterprise – Image-Enabled Electronic Medical Records – Radiology Information Systems (RIS) and Hospital Information Systems (HIS) – Quality Assurance – Archive Integrity and Security • Data Visualization – Image Data Compression – 3D, Visualization and Multi -media – DICOM, HL7 and other Standards • Teleradiology – Imaging Vocabularies and Ontologies – Transforming the Radiological Interpretation Process (TRIP)[2] – Computer pute -Aided D etectio n a nd Diag n osi s (CAD ). – Radiology Informatics Education •Etc. UCSF Department of VA Radiology & Biomedical Imaging What Is The Focus Of This Course? Learn using computational tools to maximize information for knowledge gain:
Pro-active
Drive the Let model Data Measurements knowledge Image Model
Collect data Compare Reactive with model
UCSF Department of VA Radiology & Biomedical Imaging Challenge: Maximize Information Gain
1. Q: How to estimate quantities from a given set of uncertitain ( noi se ) measuremen t?ts? A: Apply estimation theory (1st lecture today)
2. Q: How to quantify information? A: Apply information theory (2nd lecture next week)
UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 5/31 Radiology & Biomedical Imaging Motivation Example I: Tissue Classification
Gray/White Matter Segmentation Hypothetical Histogram
1.0
0.8
0.6
0.4
0.2
0.0
Intensity
GM/WM overlap 50:50; Can we do better than flipping a coin?
UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 6/31 Radiology & Biomedical Imaging Motivation Example II: Dynam ic Signa l Trac king Goal: Capture a dynamic signal on a Goal: Capture a periodic signal corrupted by noise static background
5
3
1
-1
-3
-5
0 1000 2000 3000 4000 5000 6000 Time
D. Feinberg Advanced MRI Technologies, Sebastopol, CA
UCSF Department of VA Radiology & Biomedical Imaging Motivation Example III: Signa l Decompos ition
Diffusion Tensor Imaging (DTI) Goal: •Sensitive to random motion of water Capture directions of fiber bundles •Probes structures on a microscopic scale
Microscopic tissue sample
Dr. Van Wedeen, MGH
Quantitative Diffusion Maps UCSF Department of VA Radiology & Biomedical Imaging Basic Concepts of Modeling
Θ: unknown state of interest
ρ: measurement Θ: Estimator - a good guess of Θ based on measurements
Cartoon adapted from: Rajesh P . N . Rao, Bruno A. Olshausen Probabilistic Models of the Brain. MIT Press 2002. UCSF Department of VA Radiology & Biomedical Imaging Deterministic Model
N = number of measurements M = number of states, M=1 is possible Usually N > M and ||noise||2 > 0
⎛⎞⎛ϕ11111ϕθh hm ⎞⎛⎞⎛θ noise 1 ⎞ ⎜⎟⎜ ⎟⎜⎟⎜ ⎟ ⎜⎟⎜...=+ ⎟⎜⎟⎜i ⎟ ⎜⎟⎜ ⎟⎜⎟⎜ ⎟ ⎝⎠⎝nnh1 h nmm ⎠⎝⎠⎝ noise n ⎠
φNN=+H NxMθ M noise Unknowns
The model is deterministic, because discrete values of Θ are solutions.
Note: 1) we make no assumption about the distribution of Θ 1) Each value is as likely as any another value
What is the best estimator under these circumstances? UCSF Department of VA Radiology & Biomedical Imaging Least-Squares Estimator (LSE)
The best what we can do is minimizing noise:
φ−Hθ = noise ˆ φ−=HθLSE 0 TTˆ H φ−(HH)θLSE = 0 TT−1 θLSE = (HHH ) ϕ
•LSE is popular choice for model fitting •Useful for obtaining a descriptive measure
But •LSE makes no assumptions about distributions of data or parameters •Has no basis for statistics Î “deterministic model” UCSF Department of VA Radiology & Biomedical Imaging Prominent Examples of LSE
1 N 150 ˆ Mean Value: θϕmean = ∑ ()j 100 N j=1
2 50 1 N Variance ˆˆ Intensities (Y) θϕθ=−()j 0 var iance∑() mean N −1 j=1
-50
100 300 500 700 900 Measurements (x)
200 Amplitude: ˆ θ1
100 Frequency: ˆ ity θ2
Intens 0 Phase: ˆ θ3 Decay: ˆ -100 θ4
100 300 500 700 900 Measurements UCSF Department of VA Radiology & Biomedical Imaging Likelihood Model
Likelihood Lpϕ (Φ)=Φ(ϕ | ) We believe θ is governed by chance, but we don’t know the governing probability.
We perform measurements for all possible values of θ.
We obtain the likelihood function of θ given our measurements ρ
Note: θ is random ϕ Is measurable The likelihood function is the joint probability of unknown θ and known ϕ
UCSF Department of VA Radiology & Biomedical Imaging Likelihood Model (cont’d)
Likelihood functionL
LN(θϕθ)= Pr( N | , )
New Goal: Find an estimator which gives the most likely ppyrobability of θ underlying L ( θ ) .
UCSF Department of VA Radiology & Biomedical Imaging Maximum Likelihood Estimator (MLE)
Goal: Find estimator which gives the most likely probability of θ underlying L θ () Higgphest probabilit y ΘMLE = maxp(ϕN |θ )
ΘMLE can be found by taking the derivative of the likelihood function d lnp(ϕθ |) = 0 dθ N θθ= MLE
UCSF Department of VA Radiology & Biomedical Imaging Example I: MLE Of Normal Distribution
Normal distribution Normal Distribution 1.0 N ⎡⎤1 2 2 0.8 Pr(ϕθσN | , ϕ) θ=− exp ⎢⎥( ( j) ) 2σ 2 ∑ ⎣⎦j=1 0.6 a2
0.4 log of the normal distribution (normD) 0.2 N 1 2 0.0 ln Pr()ϕ |θσ ϕ , θ2 =−()()j N 2 ∑ 100 300 500 700 900 2 j=1 a1 Log Normal Distribution σ st MLE of the mean (1 derivative): 0 d 1 N ln Pri =−=ϕ j θ 0 -5 ( ) 2 ∑( ( ) ) dNθ 4σˆ j=1
-10 1 N Θ = ϕ ( j) MLE ∑ -15 N j=1 100 300 500 700 900 a1 UCSF Department of VA Radiology & Biomedical Imaging Example II: MLE Of Binominal Distribution (()Coin Tosses) n= # of tosses y= # of heads in n tosses θ = probability of obtaining a head in a toss (unknown) Pr(y| n, θ) = probability of heads from n tosses given θ
Pr(y|n=10, θ =7) 070.7 0.2
010.1
Pr(y|n=10, θ =3) y 0.3 0.0
0.2
0.1
0.0 12345678910 UCSF VA Ny Department of Radiology & Biomedical Imaging MLE Of Coin Toss (cont’d)
Goal: Given Pr(y| n = 10, θ), estimate the most likely probability distribution Θ MLE that produced the data. #heads Intuitively: Θ = MLE #tosses