COMLEX-USA FOR RESIDENCY PROGRAM DIRECTORS COMLEX-USA Evidence–based assessment designed specifically for osteopathic medical students and residents that measures competencies required for the provision of safe and effective osteopathic medical care to patients.
It is recommended but not required that COMLEX-USA Level 3 be taken after a minimum of six months in residency. The attestation process for COMLEX-USA Level 3 helps to fulfill the NBOME mission to DO candidates are not required to pass the United States protect the public, and adds value and entrustability to state licensing Medical Licensing Examination (USMLE®) to be eligible to boards and patients. Additionally, attestation provides COMLEX-USA apply to ACGME-accredited residency programs. The score reports to residency program directors and faculty. ACGME does not specify which licensing board exam(s) (i.e.,
COMLEX-USA, USMLE) applicants must take to be eligible COMPETENCY AND EVIDENCE-BASED DESIGN for appointment in ACGME-accredited residency programs. In 2019, COMLEX-USA completed a transition to a contemporary, two Frequently Asked Questions: Single Accreditation System decision-point, competency-based exam blueprint and evidence- Accreditation Council for Graduate Medical Education, 20191 based design informed by extensive research on osteopathic physician practice, expert consensus and stakeholder surveys.3 The enhanced COMLEX-USA blueprint4 assesses measurable outcomes PATHWAY TO LICENSURE of seven Fundamental Osteopathic Medical Competency Domains5 COMLEX-USA, the Comprehensive Osteopathic Medical Licensing and focuses on high-frequency, high-impact health issues and clinical Examination of the United States, is the exam series used by all presentations that affect patients. medical licensing authorities to make licensing decisions for osteopathic physicians. COMLEX-USA is accepted in all 50 United States and recognized by a number of international jurisdictions.2 COMLEX-USA is designed to assess osteopathic medical knowledge, The COMLEX-USA Level 3 exam blueprint, designed for fundamental clinical skills, and other foundational competencies osteopathic physicians, provides a good example of how considered essential for the practice of osteopathic medicine. The medical examiners can update their assessment systems to primary and intended use of COMLEX-USA is for licensure. stay ahead of the curve of change… focuses on ‘knowledge, skills, experiences, attitudes, values, and/or behaviors that are RECOGNIZED EVIDENCE FOR VALIDITY observable and measurable and can be directly assessed in a A rigorous scientific process is used to produce exams of the highest reliable manner’— all factors that can help medical regulators quality with evidence-based validity and reliability. Following a make sound, balanced decisions as they continue their work comprehensive review of COMLEX-USA and USMLE, the FSMB in protecting the public and ensuring quality in medical (Federation of State Medical Boards) endorsed both exams as valid practice, keeping our patients safe.” and reliable for their intended respective purposes, and concluded —Heidi M. Koenig, MD that support for the validity of COMLEX-USA is exemplary. A recently Editor-In-Chief, Journal of Medical Regulation, October 2018 published collaborative study with the Federation of State Medical Boards (Roberts et al) on the predictive validity of COMLEX-USA demonstrates a strong correlation between successful completion of COMLEX-USA and a lower likelihood of state licensing board EQUALITY FOR COMLEX-USA AND disciplinary action. USMLE IN ACGME PROGRAMS ELIGIBILITY AND ATTESTATION The American Medical Association6, the Eligibility criteria for COMLEX-USA exams require attestation by a Accreditation Council for Graduate Medical dean or residency program director to confirm that the candidate is in Education, and the Federation of State Medical good academic and professional standing and is approved to take the Boards recognize equivalent uses for COMLEX- exam. Good standing denotes that the osteopathic medical student or USA and USMLE. resident meets the academic and professional requirements of the college of osteopathic medicine or residency program and is eligible to continue in the program.
RESIDENT SELECTION AND SECONDARY USES COMLEX-USA and USMLE scores are used increasingly to screen and evaluate DO residency applicants. In addition to use by residency Students and their abilities as a physician are not defined program directors, other secondary uses of COMLEX-USA scores solely by a number. Program directors need to get comfortable include student and resident promotion and assessment. with DO applicants and using COMLEX-USA as just one tool to review which applicants have the skills necessary to thrive in a GME program — just like they learn about other things in All osteopathic medical students must pass their program, they need to learn to adapt to this change.” COMLEX-USA Level 1, Level 2-CE and Level 2-PE —Kenneth B. Simons, MD, Senior Associate Dean for GME to graduate from a college of osteopathic medicine and Accreditation, Medical College of Wisconsin with the DO degree.7
INTERPRETING COMLEX-USA SCORES NBOME recommends residency and fellowship program directors COMLEX-USA exams use a criterion-referenced standard setting develop a strong understanding of what COMLEX-USA exams methodology. The minimum passing score for COMLEX-USA Level 1 or measure, how standards are set, what the scores mean, and how Level 2-CE is 400, and for Level 3 a score of 350 is passing. COMLEX-USA scores correlate to performance in residency. Passing Levels 1 and 2 means a candidate has demonstrated EQUIVALENT USES OF LICENSURE EXAMS competence to enter into supervised clinical practice settings, enter COMLEX-USA is recognized by the ACGME for acceptance into graduate medical education, and prepare for lifelong learning. Passing graduate medical education (GME). The ACGME Single Accreditation Level 3 means the candidate has demonstrated competence in System for GME aligns accreditation standards for all residency foundational competency domains required for generalist physicians to programs in the U.S., providing graduates of DO- and MD-granting deliver safe and effective osteopathic medical care of patients as required medical schools the ability to complete residency and fellowship for entry into the unsupervised practice of osteopathic medicine and to training in all programs. A single accreditation system for GME is continue lifelong learning and practice-based learning and improvement. based on mutual recognition and understanding of the important and unique qualifications and philosophies of both DO and MD graduates.
The ACGME publically communicates its policy that COMLEX-USA and NBOME cautions residency program USMLE are both acceptable for ACGME residency programs and has directors to avoid the sole use of any clarified that DO candidates are not required to pass USMLE exams in examination score, or the overuse or order to be eligible to apply to ACGME-accredited residency programs. sole use of any examination program, An individual who has completed an ACGME-accredited residency in screening or hiring residents. program or an AOA-approved residency program in any given specialty is eligible for appointment to any ACGME-accredited fellowship program that allows prerequisite education and training in that specialty, regardless of which licensing exam(s) the individual has pursued.1 PERCENTILE RANK SCORE CONVERSION Acceptance of COMLEX-USA by all ACGME programs may reduce A percentile score conversion tool is accessible via the NBOME stress and support wellness in residency applicants, and diminish website as well as the ERAS® Program Director Workstation. This tool barriers for DO students applying to programs and improve holistic converts an applicant’s 3-digit COMLEX-USA score to a percentile rank resident screening processes. to help programs compare an applicant’s relative performance to other applicants within a testing cycle, and correctly interpret assessment OVER 82% OF ACGME- scores when filtering applications. ACCREDITED RESIDENCY COMLEX-USA Level 1, May 2018-April 2019 Cohort PROGRAM DIRECTORS 3-DIGIT SCORE/PERCENTILE RANK USE COMLEX-USA AS PART OF THE 750 700 650 600 550 500 450 400 APPLICATION PROCESS 99 97 91 80 62 39 20 7 FOR DO APPLICANTS CONVERSION VARIES BY TESTING CYCLE RESEARCH AND HOLISTIC ADMISSION REVIEW • Hudson K, Tsai T-H, Finch C, Dickerman J, Liu S, and Shen L. A Holistic admission review processes in medical education consider the validity study of COMLEX-USA Level 2-CE and COMAT clinical “whole” applicant to avoid disproportionate focus on a single factor in subjects: Concurrent and predictive evidence. Journal of Graduate order to achieve diversity in a physician population that is prepared to Medical Education: 2019;11(5):521-526. address varied needs across health care populations. As part of its • Hudson KM, Feinberg G, Hempstead L, Zipp C, Gimpel JR, Wang Y. research program, NBOME conducts studies designed to help Association between performance on COMLEX-USA and the residency program directors understand COMLEX-USA exams and the American College of Osteopathic Family Physicians in-service validity support for using these scores in holistic residency application examination. Journal of Graduate Medical Education. 2018;10(5): decisions. A recent study demonstrates a strong association between 543-547. COMLEX-USA Level 1 and USMLE Step 1 performance of osteopathic • O’Neill TR, Peabody MR, Song H. The predictive validity of medical students who took both exams.8 With growing support for NBOME’s COMLEX-USA with regard to outcomes on ABFM equivalent uses of COMLEX-USA and USMLE for residency examinations. Academic Medicine. 2016;91(11):1568-1575. applications, NBOME provides program directors with insight into the • Li F, Gimpel JR, Arenson E, Song H, Bates BP, Ludwin F. The uses of COMLEX-USA and how COMLEX-USA scores can be an asset relationship between COMLEX-USA scores and performance on the in holistic resident screening and selection processes. American Osteopathic Board of Emergency Medicine Part I Certifying Examination. Journal of the American Osteopathic SCORE CONCORDANCE STUDIES Association. 2014;114(4):260-266. • Sandella JM, Gimpel JR, Smith LL, Boulet JR. The use of COMLEX- • Pierce DL, Mirre-Gonzalez MA, Carter MA, Nug D, Salamanca Y. USA and USMLE for residency applicant selection. Journal of Performance on COMLEX-USA exams predicts performance on EM Graduate Medical Education. 2016; 8(3):358-363. residency in-training exams. Academic Emergency Medicine. • Lee AS, Chang L, Feng E, Helf S. Reliability and validity of 2013;20(5) Suppl. 1. conversion formulas between Comprehensive Osteopathic Medical • Langenau EE, Pugliano G, Roberts W. Relationship between high- Licensing Examination of the United States Level 1 and United stakes clinical skills exam scores and program director global States Medical Licensing Examination Step 1. Journal of Graduate competency ratings of first-year pediatric residents.Medical Medical Education. 2014;6(2):280-283. Education. 2011;16:7362. • Schenarts PJ, Termuhlen PM, Pasley J, Rose JS, Friedell ML. A • Langenau EE, Pugliano G, Roberts W, Hostoffer R. Summary of primer on how to select osteopathic applicants to an allopathic ACOP program directors’ annual reports for first-year residents and general surgery residency. Journal of Surgical Education. relationships between resident competency performance ratings and 2011;68(3):239-245. COMLEX-USA test scores. Electronic Journal of the American • Chick DA, Friedman HP, Young VB, Solomon D. Relationship College of Osteopathic Pediatricians. 2010;2(7). between COMLEX and USMLE scores among osteopathic medical students who take both examinations. Teach Learn Medicine. 2010;22(1):3-7. FOR MORE INFORMATION ABOUT IMPORTANT COMLEX-USA PREDICTIVE VALIDITY PERFORMANCE IN IN-TRAINING RESEARCH ON VALIDITY AND NBOME AND CERTIFICATION EXAMS RELIABILITY, VISIT THE WEBSITE. • Roberts WL, Gross GA, Gimpel JR, Smith LL, Arnhart K, Pei X, Young A. An investigation of the relationship between COMLEX-USA licensure examination performance and state licensing board disciplinary actions. Academic Medicine. 2019; Volume Publish Ahead of Print.
REFERENCES 1. Frequently Asked Questions: Single Accreditation System. Accreditation Council for Graduate Medical Education (ACGME). 2. Federation of State Medical Boards of the United States. U.S. Medical Regulatory Trends and Actions. 3. Horber D, Gimpel JR. Enhancing COMLEX-USA: Evidence-based redesign of the osteopathic medical licensure examination program. Journal of Medical Regulation. October 2018, 104(3):11-18. 4. National Board of Osteopathic Medical Examiners. COMLEX-USA Master Blueprint effective beginning September 2018. 5. National Board of Osteopathic Medical Examiners. Fundamental Osteopathic Medical Competency Domains 2016: Guidelines for Assessment for Osteopathic Medical Licensure and the Practice of Osteopathic Medicine, Chicago, IL, 2016. 6. American Medical Association press release, COMLEX, DO matching and GME accreditation: What you need to know. January 23, 2019. 7. American Osteopathic Association. Commission on Osteopathic College Accreditation, Accreditation of Colleges of Osteopathic Medicine: COM Continuing Accreditation Standards. 8. Sandella JM, Gimpel JR, Smith LL, Boulet JR. The use of COMLEX-USA and USMLE for residency applicant selection. Journal of Graduate Medical Education. 2016;8(3):358- 363.
March 2020