MEDICAL IMAGING INFORMATICS: Lecture # 1
Total Page:16
File Type:pdf, Size:1020Kb
MEDICAL IMAGING INFORMATICS: Lecture # 1 Basics of Medical Imaggging Informatics: Estimation Theory Norbert Schuff Professor of Radiology VA Medical Center and UCSF [email protected] UC Medical Imaging Informatics 2011 Nschuff SF VA Course # 170.03 Department of Slide 1/31 Radiology & Biomedical Imaging What Is Medical Imaging Informatics? • Signal Processing – Digital Image Acquisition – Image Processing and Enhancement • Data Mining – Computational anatomy – Statistics – Databases – Data-mining – Workflow and Process Modeling and Simulation • Data Management – Picture Archiving and Communication System (PACS) – Imaging Informatics for the Enterprise – Image-Enabled Electronic Medical Records – Radiology Information Systems (RIS) and Hospital Information Systems (HIS) – Quality Assurance – Archive Integrity and Security • Data Visualization – Image Data Compression – 3D, Visualization and Multi -media – DICOM, HL7 and other Standards • Teleradiology – Imaging Vocabularies and Ontologies – Transforming the Radiological Interpretation Process (TRIP)[2] – Computer pute -Aided D etectio n a nd Diag nos is (C AD ). – Radiology Informatics Education •Etc. UCSF Department of VA Radiology & Biomedical Imaging What Is The Focus Of This Course? Learn using computational tools to maximize information for knowledge gain: Pro-active Improve Data Refine collection Model Measurements knowledge Image Model Extract Compare information with Re-active model UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 3/31 Radiology & Biomedical Imaging Challenge: Maximize Information Gain 1. Q: How can we estimate quantities of interest from a gitfti(i)iven set of uncertain (noise) measuremen t?ts? A: Apply estimation theory (1st lecture today) 2. Q: How can we measure (quantify) information? A: Apply information theory (2nd lecture next week) UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 4/31 Radiology & Biomedical Imaging Estimation Theory: Motivation Example I Gray/White Matter Segmentation Hypothetical Histogram 1.0 0.8 0.6 0.4 0.2 0.0 Intensity GM/WM overlap 50:50; Can we do better than flipping a coin? UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 5/31 Radiology & Biomedical Imaging Estimation Theory: Motivation Example II Goal: Capture dynamic signal on a High signal to noise static background 1 0 -1 -2 0 1000 2000 3000 4000 5000 6000 Time PiltiPoor signal to noise 5 3 1 -1 D. Feinberg Advanced MRI Technologies, Sebastopol, CA -3 -5 0 1000 2000 3000 4000 5000 6000 Time UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 6/31 Radiology & Biomedical Imaging Estimation Theory: Motivation Example III Diffusion Imaging Goal: •Sensitive to random motion of water Capture directions •Probes structures on a microscopic scale of fiber bundles Microscopic tissue sample Dr. Van Wedeen, MGH Quantitative Diffusion Maps UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 7/31 Radiology & Biomedical Imaging Basic Concepts of Modeling : target of interest and unknown : measurement : Estimator - a good guess of bdbased on measurements Cartoon adapted from: Rajesh P . N . Rao, Bruno A. Olshausen Probabilistic Models of the Brain. MIT Press 2002. UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 8/31 Radiology & Biomedical Imaging Deterministic Model N = number of measurements M = number of states, M=1 is possible Usually N > M and |noise||2 > 0 NNHθM noise The model is deterministic, because discrete values of are solutions. Note: 1) we make no assumption about 2) Each value is as likely as any another value What is the best estimator under these circumstances? UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 9/31 Radiology & Biomedical Imaging Least-Squares Estimator (LSE) The best what we can do is minimizing noise: N HθMM noise ˆ N HθLSE 0 TTˆ HHHN θLSE 0 TT1 θ LSE HH Hn •LSE is popular choice for model fitting •UflfbiiUseful for obtaining a descr iiiptive measure But •LSE makes no assumptions about distributions of data or parameters •Has no basis for statistics “deterministic model” UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 10/31 Radiology & Biomedical Imaging Prominent Examples of LSE 1 N 150 ˆ Mean Value: mean j 100 N j1 2 50 1 N Variance ˆˆ Intensities (Y) j 0 var iance mean N 1 j1 -50 100 300 500 700 900 Measurements (x) 200 Amplitude: ˆ 1 100 Frequency: ˆ ity 2 Intens 0 Phase: ˆ 3 Decay: ˆ -100 4 100 300 500 700 900 Measurements UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 11/31 Radiology & Biomedical Imaging Likelihood Model Likelihood Lp | Pretend we know something about We perform measurements for all possible values of We obtain the likelihood function of given our measurements Note: is random is a fixed parameter Likelihood is a function of both the unknown and known UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 12/31 Radiology & Biomedical Imaging Likelihood Model (cont’d) Lp N | NGlNew Goal: Find an estimator which gives the most likely probability distribution underlying L UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 13/31 Radiology & Biomedical Imaging Maximum Likelihood Estimator (MLE) Goal: Find estimator which gives the most likely probability distribution underlying xN. Max like lihoo d func tion MLE maxpN | MLE can be found by taking the derivative of Likelihood F d ln p | 0 d N θθ MLE UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 14/31 Radiology & Biomedical Imaging Example I: MLE Of Normal Distribution Normal distribution Normal Distribution N 1.0 2 1 2 pj|, exp 0.8 N 2 2 j1 0.6 a2 0.4 log of the normal distribution (normD) 0.2 N 1 2 0.0 lnpj | , 2 N 2 100 300 500 700 900 2 j1 a1 Log Normal Distribution st MLE of the mean (1 derivative): 0 N d 1 -5 lnpj 0 2 MLE dMLE 4ˆ j1 -10 1 N j MLE -15 N j1 100 300 500 700 900 a1 UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 15/31 Radiology & Biomedical Imaging Example II: MLE Of Binominal Distribution (()Coin Toss) Distribution function f(y|n,w): n= number of tosses w= probability of success f(y|n=10, w=7) 0.7 0.2 010.1 y 0.0 f(y|n=10,w=3) 0.3 0.2 0.1 000.0 12345678910 UC Medical Imaging Informatics 2009, Nschuff N SF VA Course # 170.03 y Department of Slide 16/31 Radiology & Biomedical Imaging MLE Of Coin Toss (cont’d) Goal: Given the observed data f (y|w=0.7, n=10), find the parameter MLE that most likely produced the data. Lyn(MLE | 7, 10) For a fair coin MLE 0.5 UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 17/31 Radiology & Biomedical Imaging MLE Of Coin Toss (cont’d) Likelihood function of coin tosses 0.25 n! y ny 0.20 Ly|1 od yny!! oo 0.15 0.10 Likelih What is the likelihood of observing 7 heads 0.05 given that we tossed a fair coin 10 times 0000.00 10! 7 10 7 Ln| 10, w 7 0.5 1 0.5 0.12 0.1 0.3 0.5 0.7 0.9 7! 10 7 ! W unfair coin =0.6 10! 10 7 Ln| 10, w 7 0.67 1 0.6 0.21 7! 10 7 ! log likelihood function -1 lnLwy | -2 n! lnyw ln ny ln 1 w -3 0.1 0.3 0.5 0.7 0.9 yny!! UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 18/31 Radiology & Biomedical Imaging MLE Of Coin Toss Evaluate MLE equation (1st derivative) dLln y ny 0 dMLE MLE 1 MLE yn MLE y 0 MLE MLE(1 MLE ) n According to the MLE principle, the distribution f(y/n) for a given n is the most likely distribution to have generated the observed data of y. UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 19/31 Radiology & Biomedical Imaging Relationship between MLE and LSE Assume: is independent of noiseN MLE and noiseN have the same distribution pθ NN|| pnoise H noiseN is zero mean and gaussian p(|) is maximized when LSE is minimized UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 20/31 Radiology & Biomedical Imaging Bayesian Model Prior knowledge Now, the daemon comes into play, but we know The daemon’s preferences for (prior knowledge). prior p New Goal: Find the estimator which gives the most likely probability distribution of given everything we know. UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 21/31 Radiology & Biomedical Imaging Bayesian Model posterior C LN | p UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 22/31 Radiology & Biomedical Imaging Maximum A-Posteriori (()MAP) Estimator Goal: Find the most likely MAP (max. posterior density of ) given . Maximize joint density MAPmaxLp N | N MAP can be found by taken the partial derivative d lnLp | ln L | ln p 0 dNN UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 23/31 Radiology & Biomedical Imaging Example III: MAP Of Normal Distribution The sampl e mean o f MAP is: 2 N j MAP 22 T j1 If we do not have prior information on , inf or T inf μμμˆˆˆMAP ML , LSE UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 24/31 Radiology & Biomedical Imaging Posterior Distribution and Decision Rules p(|) MSEX MAP UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 25/31 Radiology & Biomedical Imaging Decision Rules Measurements Likelihood Posterior function Distribution Result Prior Gain Distribution Function UC Medical Imaging Informatics 2009, Nschuff SF VA Course # 170.03 Department of Slide 26/31 Radiology & Biomedical Imaging Some Desirable Properties of Estimators I: Unbiased: Mean value of the error should be zero E - 0 Consistent: Error estimator should decrease asymptotically as number of measurements increase.