Direct Comparison of MEDCIN ® and SNOMED CT ® for Representation of a General Medical Evaluation Template Steven H. Brown MS MD 1,2 , S. Trent Rosenbloom MD MPH 2 , Brent A. Bauer MD 3, Dietli nd Wahner -Roedler MD 3, David A. Froehling, MD, Kent R, Bailey PhD, M ichael J Lincoln MD, Diane Montella MD 1, Elliot M. Fielstein PhD 1,2 Peter L. Elkin MD 3 1. Department of Veterans Affairs 2. Vanderbilt University, Nashville TN 3. Mayo Clinic, Rochester MN Background : Two candidate terminologies to efforts. Usable and functionally complete support entry of ge neral medical data are standard terminologies need to be available to SNOMED CT and MEDCIN . W e compare the systems designers and architects. Two candidate ability of SNOMED CT and MEDCIN to terminologies to support entry of general medical represent concepts and interface terms from a data are SNOMED CT and MED CIN . VA gener al medical examination template. Methods : We parsed the VA general medical SNOMED CT is a reference terminology that evaluation template and mapped the resulting has been recommended for various components expressions into SNOMED CT and MEDCIN . of patient medical record information by the Internists conducted d ouble independent reviews Consolidated Health Informatics Council and the on 864 expressions . Exact concept level matches National Committee on Vital and Health were used to evaluate reference coverage. Exact Statistics. (12) SNOMED CT, licensed for US - term level matches were required for interface wide use by the National Library of Medicine in terms. 2003, was evaluated in 15 M edline indexed Resul ts : Sensitivity of SNOMED CT as a studie s in 2006 . According to the College of reference terminology was 83% vs. 25% for American Pathologists , SNOMED CT is “the MEDCIN (p<0.0 01). The sensitivity of universal health care terminology that makes SNOMED CT as an interface terminology was health care knowledge usable and accessible 53% vs. 7% for MEDCIN (P< 0.001) . wherever and whenever it is needed”. (15) Discussion : The content coverage of SNOMED SNOMED CT includes approximately 370,000 CT as a reference termin ology and as an concep ts and over 1 million synonyms. Despite interface terminology outperformed MEDCIN. SNOMED CT 's relatively good performance in We did not evaluate other aspects of interface content coverage studies, the Department of terminologies such as richness of clinical Defense elected to use MEDCIN as its point of linkages . care terminology for CHCS II (now AHLTA) . Background MEDCIN is a clinical terminology designe d to suppor t medical documentation entry into Computerized structured data entry systems have electronic health record systems. MEDCIN was been used in a variety of environmen ts to initially developed as "an intelligent clinical improve clinical documentation quality and database for documentation at the time of timeliness , quality of care , practice guideline care." (16) MEDCIN 's producer, Medicomp, compliance, research data collection and other states that their software "makes capture of the aspects of patient care .(1 -9) Structured data encounter information fast enough, sufficiently entry permits the imple mentation of reminders, comprehensive and rewarding to overcome alerts, data -driven monitors, and other types of physician reluctance." MEDCIN has evolved to decision support in addition to monitoring of include more than 2 50 ,000 concepts since 1978, documentation completeness and has been installed in several EHR systems as an interface terminology for clinical To realize the potential of structured data entry, documentation including AHLTA, the EHR systems should be built using standard system developed for the US Department of terminologies for u nderlying knowledge Defense. MEDCIN covers concepts commonly representation. (10 -13) This strat egy facilitates used in medical histories, physical examination, sharing of patient -specific data and decision tests, and some diagnoses and therapies. Unlike support rules and content. (14) Without standards SNOMED CT, MEDCIN evaluations are for knowledge representation, sharing requires few. (17) In general, MEDCIN concepts are tim e consuming and often challenging mapping designed to be pre -coordinated to a level that AMIA 2007 Symposium Proceedings Page - 75 allows them to contain adequate clinical meaning We extracted a ll objects from the three PNCS to be useful for documentation. MEDCIN has template definition files including captions, been mapped to other te rminologies, including labels, object names and object contents (e.g. list CPT -4, ICD -9, ICD -10 and DSM -IV. box choices). We mapped the extracted data to MEDCIN (3rd quarter 2005 R2 ) via automated We previously evaluated SNOMED CT's ability and manual processes. Two independent to represent concepts and interface terms (18) reviewers from our previous study of SNOMED - needed for the US Department of Veterans CT (released July 2006) examined a convenience Affairs ( VA ) general medical evaluation sample of half of the template objects based on template for compensation and pension their previous review of SNOMED CT . The two (disabil ity) examinations .(19) In the current internist reviewe rs assessed MEDCIN as a study we evaluate MEDCIN 's ability to represent reference terminology (i.e., concepts present or the VA general medical examination template absent) and as an interface terminology (i.e., and compared its performance to SNOMED CT's normalized terms present or absent). The performance on the same task. reviewers were required to search MEDCIN manually using the Multi -threaded Clinical Methods Vocabulary Server’s Browser (21) before categorizing a term as a “non match”. We used We utilized a portion of the same data set, and Bo olean operators to construct post -coordinated the same review methodology, reviewers, and compositions of MEDCIN terms when needed to review assignments to assess MEDCIN that we model complex input data from the template . used previously to assess SNOMED CT. Mapping examples are given in table 1. Linking semantics were not required to be present for The general medical eval uation template post -coordinations to be cons idered a match . A examined in the current study of MEDCIN and third independent review was conducted when SNOMED CT was created using the Progress needed to establish a consensus rating for Note Construction Set (PNCS) template design subsequent analysis . A greement statistics (e .g. environment .(1) Three definition files fully kappa) are not required for consensus ratings and define a PNCS template : a form definition file, a are not reported . script definition file, and a report d efinition file . The form def inition file details each data object Results were classified according to the schema on the template . The script definition file in figures 1 and 2. We report sensitivity and controls form behavior at run -time. Report positive predictive value (PPV) as a reference definitions specify how captions are merged with terminology and as an interface terminology. We patient data elements to create a free -text report compared the sensitivity and PPV of MEDCIN for upload into the Veterans H ealth Information and SNOMED using uncorrected Pearson Chi - Systems and Technology Architecture (VistA), square test for comparing two independent an integrated outpatient and inpatient clinical binomial proportions. information system. (20) All All Template Template Terms Terms 864 864 Valid Term All Valid Term All for MEDCIN Others & Concept Others Coding Present 848 16 212 652 Concept Concept Concept Concept Interface Interface Interface In terface Present in Absent in Present in Absent in Term Term Term Term MEDCIN MEDCIN MEDCIN MEDCIN Present Absent Present Absent 212 636 0 16 15 19 7 1 651 Figure 1. Reference t erminology classification Figure 2. Interface t erminology clas sification AMIA 2007 Symposium Proceedings Page - 76 Template Term CT Concept Medcin Concept Hx of Congestive Heart - Congestive heart failure (disorder) [42343007] [K] History [5141] [K] Failure - [is Qua lified By] Congestive Heart Failure . History of (present illness) (contextual qualifier) (qualifier value) [33268] [K] [51042001] [Q] Neurologic Exam Deep - Neurological assessment (procedure) [225398001] [K] Physical Examination Tendon Reflexes - [is M odified By] [6000] [K] . Deep (qualifier value) [795002] [M] - [is Modified By] Deep Tendon Reflexes . Tendon reflex (observable entity) [38299000] [M] (DTR) [9051] [K] ] Location of enlarged lymph . Location (attribute) [24 6267002] [M] Lymph Nodes Enlarged nodes . Enlarged (qualifier value) [260376009] [M] [9325] [K] . Entire lymph node (body structure) [181756000] [M] Calf Tenderness - Tenderness (finding) [247348008] [K] (Lower) Leg Tenderness On - [has Finding Site] Palpation Gastrocnemius . Entire calf of leg (body struct ure) [244015008] [M] [8118] [K] Table 1 . Examples of terms and MCVS mappings. Reference Terminology Interface Terminology Sensitivity (Recall) PPV (Precision) Sensitivity (Recall) PPV (Precisio n) Group n MED CT p MED CT p MED CT p MED CT P Full group 864 0.25 0.83 <0.001 1.00 0.98 0.034 0.07 0.53 <0.001 0.94 0.96 0.634 Excluding disability 853 0.25 0.83 <0.001 1.00 0.99 0.100 0.07 0.53 <0.001 0.94 0.98 0.331 Only disability 11 0.00 0.20 0.29 2 n/a 0.14 n/a 0.00 n/a 0.00 Excluding navigation 829 0.03 0.83 <0.001 1.00 0.98 0.044 0.07 0.52 <0.001 1.00 0.96 0.456 Navigation only 35 0.26 0.82 <0.001 1.00 0.93 0.786 0.00 0.78 0.018 0.00 0.91 0.007 Excluding labels only 437 0.09 0.86 <0.001 1.00 0.98 0.385 0.10 0.34 0.002 1.00 0.95 0.635 Labels only 427 0.41 0.79 <0.001 1.00 0.98 0.043 0.06 0.75 <0.001 0.92 0.97 0.328 no branch points 738 0.28 0.81 <0.001 1.00 0.98 0.033 0.07 0.59 <0.001 0.94 0.96 0.585 branch points only 126 0.05 0.93 <0.