Automatic Classification of Digestive Organs in Wireless Capsule
Total Page:16
File Type:pdf, Size:1020Kb
Automatic Classification of Digestive Organs in Wireless Capsule Endoscopy Videos Jeongkyu Lee1, JungHwan Oh2, Subodh Kumar Shah1, Xiaohui Yuan2, Shou Jiang Tang3 1Dept. Comp. Sci. & Eng. 2Dept. Comp. Sci. & Eng. 3Division of Digestive Diseases University of Bridgeport University of North Texas UTSW Medical Center Bridgeport, CT 06604 Denton, TX 76203 Dallas, TX 75390 fjelee,[email protected] fjhoh,[email protected] [email protected] ABSTRACT A human digestive system consists of a series of several Wireless Capsule Endoscopy (WCE) allows a physician to di®erent organs including the esophagus, stomach, small in- examine the entire small intestine without any surgical op- testinal (i.e., duodenum, jejunum, and ileum) and colon. eration. With the miniaturization of wireless and camera Standard endoscopy has been playing a very important role technologies the ability comes to view the entire gestational as a diagnostic tool for the digestive tract. For example, track with little e®ort. Although WCE is a technical break- various endoscopies such as gastroscopy, push enteroscopy through that allows us to access the entire intestine without colonoscopy have been used for the visualization of digestive surgery, it is reported that a medical clinician spends one system. However, all methods mentioned above are limited or two hours to assess a WCE video. It limits the number in viewing small intestine. To address the problem, Wire- of examinations possible, and incur considerable amount of less Capsule Endoscopy (WCE) was ¯rst proposed in 2000, costs. To reduce the assessment time, it is critical to develop which integrates wireless transmission with image and video a technique to automatically discriminate digestive organs technology [13, 2, 1, 12, 14, 8]. After FDA approved in 2002, such as esophagus, stomach, small intestinal (i.e., duode- it is now used as one of the important tools to examine small num, jejunum, and ileum) and colon. In this paper, we intestine. propose a novel technique to segment a WCE video into WCE allows a physician to examine the entire small in- these anatomic parts based on color change pattern anal- testine non-invasively. WCE uses a small capsule, 11 mm in ysis. The basic idea is that the each digestive organ has diameter and 25 mm in length (see Figure 1 (a)). The front di®erent patterns of intestinal contractions that are quan- end of the capsule has an optical dome where white light ti¯ed as the features. We present the experimental results emitting diodes (LEDs) illuminate the luminal surface of that demonstrate the e®ectiveness of the proposed method. the gut, and a micro camera sends images via wireless trans- mission to a receiver worn by a patient (see Figure 1 (b)). Another part of the capsule contains a small battery that Categories and Subject Descriptors can last up to 8 hours. The patient swallows a small cap- I.2.10 [Arti¯cial Intelligence]: Vision and Scene Under- sule, usually after an overnight fast. As the capsule moves standing|Video analysis; I.4.8 [Image Processing and through the gastrointestinal tract, images are transmitted Computer Vision]: Scene Analysis|Time-varying imagery by the digital radio frequency communication channel to a data recorder (see Figure 1 (c)). This data are transferred to a computer for interpretation by the specialists. The cap- General Terms sule is then passed in the patient's stool without the need Algorithms, Experimentation, Software for retrieval. Keywords Wireless capsule endoscopy, event boundary detection, event hierarchy, energy function, high frequency content function 1. INTRODUCTION (a) (b) (c) Figure 1: Wireless Capsule Endoscopy Equipments: (a) the capsule, (b) the 8-lead antenna array, and Permission to make digital or hard copies of all or part of this work for (c) receiver/recorder unit and battery. personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to Although WCE is a technical breakthrough that allows republish, to post on servers or to redistribute to lists, requires prior specific to access the entire intestine without surgery, it is reported permission and/or a fee. SAC’07 March 11-15, 2007, Seoul, Korea that the interpretation time takes 1 or 2 hours. This is Copyright 2007 ACM 1-59593-480-4 /07/0003 ...$5.00. very time consuming for the gastroenterologist. It limits its general application and incur considerable amount of are di®erent since each digestive organ has di®erent types costs. To reduce the assessment time, Berens et al. proposed of movements and functionalities. We will characterize in- the technique for automatically discriminating stomach, in- testinal contractions to ¯nd events in WCE videos in the testine, and colon by using the Discrete Cosine Transform following subsections. (DCT) and Principal Component Analysis (PCA) classi- ¯ers [1]. However, the technique can detect only two bound- aries of stomach/intestine and intestine/colon, which is not enough to assist the specialist. In [14], ROC (Receiving Operating Characteristic) curves analysis are used for the classi¯cation of contraction and non-contraction images in (a) WCE videos. However, it is not based on the contents of WCE video. In this paper, we propose a novel algorithm for event boundary detection in Wireless Capsule Endoscopy videos based on an energy function. The basic idea is that each di- gestive organ such as esophagus, stomach, duodenum, small (b) intestine, and colon, has di®erent patterns of intestinal con- tractions. These patterns have been widely used in the anal- Figure 2: Sequence of Frames of WCE Video from: ysis of biogenic signals, such as Electrogastrogram [7] or (a) stomach and (b) small bowel. Electrocardiogram [10]. We ¯rst characterize the contrac- tions of WCE video using Energy function in a frequency domain. Then, we segment WCE video into events by us- 2.2 Feature Extraction ing a high frequency content (HFC) function. The detected To characterize the contractions in WCE videos, we tested event boundaries indicate either entrance of the next organ several di®erent features. Many works have tackled the or unusual events in the same organ, such as intestinal juices, problem of the characterization of intestinal contractions in bleedings, and unusual capsule movements. We classify the capsule endoscopy using various features [1, 12, 14]. In [14], segmented events into higher level events that represent di- 34 features are extracted to describe the contractions along gestive organs. The classi¯cation result is represented by a 9 frames: 9 mean intensities, 9 hole sizes, 9 global contrasts, tree structure, which is called an event hierarchy of WCE. 6 correlations among frame sequences, and 1 variance of in- The remainder of this paper is organized as follows. Fea- tensities. However, the approach requires high computations ture extraction and event boundary detection in WCE video to extract many features from videos, and pre-processing to are discussed in Section 2. Section 3 presents the proposed test the data, which prevent on-line processing. technique for building event hierarchy of WCE video. In We mainly focus on color features in order to pursue on- Section 4, we discuss our experimental results. Finally, Sec- line processing. Since colors are the only feature values cap- tion 5 presents some concluding remarks. tured by a camera directly, it can reflect the activities of di- gestive system e®ectively such as contractions and juices. In other words, intestinal movements, i.e. contractions, could 2. EVENT BOUNDARY DETECTION IN WCE change the color values. Among the various color domains, VIDEOS we select HSI color space. The reason we choose HSI is that In this section, we ¯rst discuss the intestinal contractions hue, saturation and intensity are most robust components of which patterns are the most important discriminator of for video and image processing [3]. digestive organs. In order to characterize the contractions, First, we convert the RGB color space into the HSI color we extract energy-based feature in frequency domain from space for every frame in WCE video. Since the intestinal WCE images, and then detect event boundaries by using a contractions that are periodic movements a®ect all compo- high frequency content (HFC) function. nents in the color space, we can use either any components among hue, saturation and intensity, or any combinations 2.1 Intestinal Contractions of components in HSI. In this paper, we use mean of entire pixel values in a frame. Figure 3 shows the average values Peristalses or contractions are intrinsic motility patterns of HSI for the ¯rst 5000 frames of a sample WCE video. in bowel movements, which is also the basic activity through- out entire gastrointestinal tract. Since the intestinal con- ABC D traction is a very good pathological indicator, it is widely used for the diagnosis of many gastrointestinal diseases. One Intensity Saturation Hue of examples using contractions is Electrogastrogram [7] (EGG). Avg. EGG is a non-invasive recording of the electrical activity Value of stomach. Slow waves are generated from the activity of the gastrointestinal wall surface, or gastrointestinal contrac- tions. Frame # WCE videos can record continuous activities in digestive system such as intestinal contractions, and visualize them Figure 3: Average values of HSI colors for the ¯rst easily. Figure 2 shows a number of examples of intestinal 5000 frames of a sample WCE video. contractions taken by WCE. Figure 2 (a) and (b) are se- quences of frames captured from stomach and small bowel, The blue, pink, and gray lines in the ¯gure indicate the respectively. As seen in the ¯gure, the motility patterns average values of hue, saturation, and intensity of each frame in a video, respectively.