Us 2017 / 0372471 A1
Total Page:16
File Type:pdf, Size:1020Kb
US 20170372471A1 ( 19) United States ( 12 ) Patent Application Publication (10 ) Pub. No. : US 2017 /0372471 A1 EURÈN (43 ) Pub . Date : Dec . 28 , 2017 ( 54 ) METHOD AND SYSTEM FOR DETECTING (52 ) U . S . CI. PATHOLOGICAL ANOMALIES IN A CPC .. .. .. G06T 7 /0012 (2013 . 01 ) ; G06K 9 /628 DIGITAL PATHOLOGY IMAGE AND ( 2013 .01 ) ; G06K 9 /6256 ( 2013 .01 ) ; G06T METHOD FOR ANNOTATING A TISSUE 2207 /20132 ( 2013 .01 ) ; G06K 2209 / 05 SLIDE ( 2013 .01 ) ; GO6T 2207 /30024 ( 2013 . 01 ) ; GO6T 2207 / 30096 ( 2013. 01 ) ; G06T 2200 / 24 ( 71 ) Applicant: ContextVision AB , LINKOPING (SE ) ( 2013 .01 ) (57 ) ABSTRACT ( 72 ) Inventor: Kristian EURÈN , VASTERAS (SE ) A method for annotating a tissue slide image , a system and a method performed by a computing system for detecting ( 73 ) Assignee : ContextVision AB , LINKOPING (SE ) pathological anomalies in a digital pathology image are disclosed . The method performed by a computing system for detecting pathological anomalies in a digital pathology (21 ) Appl . No .: 15 /195 ,526 image includes providing a digital pathology image to the computing system and analyzing the digital pathology ( 22 ) Filed : Jun . 28 , 2016 image using an identification module arranged on the com puting system . The identification module uses a machine learning module to execute recognizing an object containing Publication Classification an abnormal image pattern using an identification model (51 ) Int. CI. loaded in said identification module and identifying whether GOOT 7700 ( 2006 .01 ) the abnormal image pattern corresponds to a pathological G06K 9 /62 ( 2006 .01 ) anomaly using the identification model . wwwsnowowy Start 30 mm wwwwwwwww WWWWWProviding WWWWWWWWWWWWWWWWWWWWWWWW a user interface for inputting wwwww a digital pathology image and for outputting a display of detected pathological anomalies in the digital pathology 31 MWMWWWWWWWW imagenWWWWWwwwwwwwwww Providing an identification0911091101190 model using a machine learning algorithm trained on a plurality of annotated 32 digital pathology images Wwwwwwwwwwwwwwwwwwwwwwwwwwwwww UMUMM Providingi l WWWWWWWWWWWa digital pathology image 33 Analyzing the digital 11223 pathology image using the identification module which recognizes an 34 object containing an abnormal image pattern and identifies whether the abnormal image pattern corresponds to a l" ast pathological anomalyou UDWIKOVIWWIIIWV1IWILININIWINWWIIIIIIIIIII WHIKKWIKWIKWIKK Stop Patent Application Publication Dec . 28 , 2017 Sheet 1 of 7 US 2017 /0372471 A1 Fig.1 11 tittihatin pathologyimageusingan correspondstoa pathologicalanomaly.eu Start Providingadigital pathologyimage Analyzingthedigital identificationmodulewhich recognizesanobject containinganabnormal imagepatternand identifieswhetherthe abnormalimagepattern Stop WWWWWWWWWWWWWWWWWWWW IMWIMWWWWWWMWWW. MMMMMMMMMMMMMMMMMMMMMMM UmmmmmmmmMmd Hul WWWWWWWWWWWWWWWWWW WWWWWWWWWWINWINWINWINWINWINWINWINNINMI 10 Patent Application Publication Dec . 28 , 2017 Sheet 2 of 7 US 2017 /0372471 A1 Fig.2 |21 ATTI" itro H ww WWWWWWWWWWWWWWWWWWWWWWW/17 IKI patterncorrespondstoa pathologicalanomaly.com Wwwwhy, Providingadigital pathologyimage Analyzingthedigital pathologyimageusing whichrecognizesan objectcontainingan Stop Start Providinganidentification modelusingamachine learningalgorithmtrained onapluralityofannotated digitalpathologyimages theidentificationmodule abnormalimagepattern andidentifieswhether theabnormalimage WWWWWWWWWW wwww WwwwwwwW !! t WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW " AMMINIINIHINIHINHINNHHKHIINUMINUMINUMINUMIINU ANNUUSAASTALINIMAINANSINISILINISHINUVINIAYUMINHALAIMUHSIA 20mm Patent Application Publication Dec . 28 , 2017 Sheet 3 of 7 US 2017 /0372471 A1 Fig.3 31 32 33 34 77X 111111111 . WM WWW patterncorrespondstoa Providingauserinterfaceforinputtingdigital pathologyimageandforoutputtingadisplayof detectedpathologicalanomaliesinthedigitalpathology imagem Providingadigital pathologyimage Analyzingthedigital pathologyimageusing whichrecognizesan objectcontainingan Stop Start IIIIVOUW Providinganidentification modelusingamachine learningalgorithmtrained onapluralityofannotated digitalpathologyimagesWWWII abnormalimagepattern andidentifieswhether theabnormalimage pathologicalanomaly <IMA. wwwwwZIMININNI theidentificationmodule AWWWWWWWWWWW 211MM WWWWWWWWWWW MWWMWAMINIW4W MWWMWWW WWWWWWWWWWWWWWWWWWUWULUNA 3153553WWMWMwand wwwwwwwwWWWWWWWWWWWWWWWWWWWWWWWWWWWWWwwwwwww WWW WWWWWWWWWWWWWWWWWWWWWWWWWW wuwurl2/12error 30 Patent Application Publication Dec . 28 , 2017 Sheet 4 of 7 US 2017 /0372471 A1 42 it! Fig.4 41 43 44 46 47 IN W 2012 WISS histochemicaldyefromthe withslideatissueStainingthe wwwmmmmmm Annotatingtheatleastone StartStart Providingatissueslide WWWWWWWWWWWWWWWWWWWWWW Stainingthetissueslidewithatleastonebiomarkerand onefirsthistochemicaldyeoratleastbiomarkerand Generatingatleastone firstimageofthetissue slide firsttheRemoving tissueslide dyehistochemicalsecond Generatingatleastone secondimageofthetissueslidemmm secondimageofthetissue slideWWWWWWWWWW Stop MUUTUMN WWWWWWWWWWWWW onefluorescencedyeoratleastfluorescentbiomarker WAVUMI 176KW Www www WWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWMA WWWWWWWWWWWWWWWWWWW wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww WWWWWWWWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWWWWW . WOWOWOWwwwwwwwwwwww WWWWWWWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWW wwwwwwwwwwwwwwwwwwwwwwwwwww 40. WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW Patent Application Publication Dec . 28 , 2017 Sheet 5 of 7 US 2017 /0372471 A1 Fig.5 dlffiftititi dom FIFA112222 "SUNUMUWIMUNINNI CUIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII Asojoyzedjer! imagereceiving ?????? Digitalpathologyimage analyzingmodule Identificationmodule Machinelearning module Recognizing module Identifying module model Classifier Determining module www IIIIIIIIIIII NITS WWWWWWWWWWWWWWW Identification WINNIHINHINHA H WWWMWWWWWWWWW WWWWWWwwwwwwwwwwww ovwWWWWWWWWWWWWWWWWWWWWWWWWWWWWW wwWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW ittitWwwwwww WWWWWWWWWWWWWWWWWWWWWWWWWWW EFERAT WWWWWWWWWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW F IS ***** * WIwMTIwMTIWITIWWIIIIII .52 "521 5211“ 52111 52112 5212 52121 52122 50. Patent Application Publication Dec . 28 , 2017 Sheet 6 of 7 US 2017 /0372471 A1 tit a Identification modelproviding module HMMWWMWMTraining module WWWWWWWWWWWWWWWWWWWWA Fig.6 53 wwwmmwwwwwwwwwwwwETAT 531 I 512 Ni0111 ARTIR 111 D W10 NIKI * Digitalpathology imagereceiving module Digitalpathologyimage analyzingmodule wwwmmmWWW. model module 511 Identificationmodule Machinelearning module Recognizing module Identifying module Identification Classifierwwwwwwwwww Determining AITINIMWWWWWWWWWW W w WWWWWWWWWWWWWWWW FOLKKIKIRIKIMU VIIMKIIRWwWMWWWW WWWMMMMM 1811 JWOWWWWWWWWWWWWWWWWWW WIMININNIWINNINIWA VIVIUMENIWINIANVIVIUMWI WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW /11 51. MINNIRNIRIHIHNIINNIWIKWIKWITNIKHIKHIINIM fututett MINIMUMUIKUINIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKUN 1143381111 52. 521N 5211" 52111 52112 5212 52121 52122 50m Patent Application Publication Dec . 28 , 2017 Sheet 7 of 7 US 2017 /0372471 A1 531 TEETH Identification modelproviding module Training module Fig.7 53 WWWWWWWWWWWWWWWWWWWW Att"WWWWWWWWWWWWWWWWWWWWWW hittintit1113 withthe !LIGI " hile w Digitalpathology imagereceiving module Digitalpathologyimage analyzingmodule model WWWUWMMMMMMMMMMMMMMMMMMHHH! 511 Identificationmodule Machinelearning module nowwwwwwwwwwwwwwww Recognizing module Identifying module Identification Classifier Determining module Umum WWWWWWNININWIININIWWWNINIWIN "1111rttiVIINNITUWINUIKINININ ! UMUMMMMMWAUMUHIMU AMMINIWINNIN WWWWWWWWWWWWWWWWWWWWWWWWW stilitWWWWWWWWWWWWwwwwwww WWWWWWNINWWWWWWWWWWWWWWWWWWWWWWWWWW UMWWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWWWW WINIWWWWWWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWW 51. WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW INHIMMINIMIINIMUMIIHIIIIIIIIIIIIIIIIIIIII sama PAKOWWWWWWWWWWWWWWWWWWWUWW 52 5211" 5212" 514.wrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr 521N 52111 52112 52121 52122 513 154 uttfilm 50cm Input module 5411 User interface 542 Output module WWWWWWWWWWWWWWWW THWOWWWWWWWWWWWWWWW JAUNIIKIINNIKIWA US 2017 /0372471 A1 Dec . 28 , 2017 METHOD AND SYSTEM FOR DETECTING arranged on the computing system . The identification mod PATHOLOGICAL ANOMALIES IN A ule uses a machine learning module to execute recognizing DIGITAL PATHOLOGY IMAGE AND of an object containing an abnormal image pattern using an METHOD FOR ANNOTATING A TISSUE identification model loaded in said identification module and SLIDE identifying whether the abnormal image pattern corresponds to a pathological anomaly using the identification model . TECHNICAL FIELD Thereby, it is possible to automatically detect pathological [ 0001] The present invention relates to a method per anomalies in a digital pathology image . formed by a computing system for detecting pathological 10008 ] According to a further aspect of the invention , anomalies in a digital pathology image and a system for identifying whether the abnormal image pattern corresponds detecting pathological anomalies in a digital pathology to a pathological anomaly comprises classifying the abnor image . The present invention also relates to a method for mal image pattern using a classifier in the identification model to classify the abnormal image pattern in accordance annotating a tissue slide image . with at least two classes and determining whether the BACKGROUND OF THE INVENTION abnormal image pattern corresponds to a pathological anomaly based on the classification . [ 0002] A common method to detect pathological anoma 10009 ] According to another aspect of the