Computer Technology in Medical Education and Assessment September 1979 NTIS order #PB80-102528 Library of Congress Catalog Card Number 79-600146 For sale by the Superintendent of Documents, U.S. Government Printing Office Washington, D.C. 20402 Stock No. 052-003-00704-5 PREFACE This report reviews the state-of-the-art of the use of computer technology in medical education and assessment. It both describes the technologies and strategies for computer- based education and assessment, the current state of medical education and assessment activities, and computer applications in medicine. It also provides examples of systems for the development and dissemination of computer-based educational materials. A sum- mary and analysis of such computer uses are also presented. Although the focus of the re- port is on physician education and assessment, the implications of the activities described are applicable to other health professions. The Federal Government’s role in computer-based education and assessment has been primarily in the research and development phases. However, the medical areas in which such computer-based methods have been applied have been of long-standing in- terest to the U.S. Congress. Such areas include health professions education, monitoring and evaluating physician services through Professional Standards Review Organizations (PSROs), and efforts to contain the costs of such federally financed programs as Medi- care and Medicaid. In all of these areas the use of computer technology is becoming the rule rather than the exception. This report examines these computer applications in various stages of the life-long process of medical education from entry into medical school through active practice. It therefore provides an overview that may be of value to two audiences: policymakers at the Federal, State, and local levels, and those in the public and private sectors who are directly involved in the use of computer technology in medical education and assessment. The study was conducted by the OTA Health Group staff with assistance from an advisory panel and contractors. The report was also reviewed by the OTA Health Ad- visory Committee. This report is a synthesis and does not necessarily represent the views of any of the individuals involved in its preparation. JOHN H. GIBBONS Director . 111 OTA HEALTH GROUP STAFF Joyce C. Lashof, Assistant Director Health and Life Sciences Division H. David Banta, Health Group Manager Ruann Ernst Pengov, Study Director Lawrence Miike, Study Director Page S. Gardner, Editor Lynne Pepi Pietz, Congressional Fellow Administrative Staff William S. Burnett Shirley A. Gayheart Contractors Association of American Medical Colleges Washington, D.C. Health Care Management Systems, Inc. Encinitas, Calif. Human Resources Research Organization Alexandria, Va. OTA PUBLISHING STAFF John C. Holmes, Publishing Officer Kathie S. Boss Joanne Heming OTA HEALTH GROUP ADVISORY COMMITTEE Frederick C. Robbins, Chairman Dean, School of Medicine, Case Western Reserve University Stuart H. Altman Sidney S. Lee Dean Associate Dean Florence Heller School Community Medicine Brandeis University McGill University Robert M. Ball Mark Lepper Senior Scholar Vice President for Evaluation Institute of Medicine Rush-Presbyterian Medical School National Academy of Sciences C. Frederick Mosteller Bernard Barber Professor and Chairman Professor Department of Biostatistics Department of Sociology School of Public Health Barnard College Harvard University Columbia University Beverlee Myers Lewis H. Buttler Director Health Policy Program Department of Health Services University of California State of California Kurt Deuschle Helen Ewing Nelson Professor of Community Medicine President Mount Sinai School of Medicine Consumer Research Foundation Mill Valley, Calif. Rashi Fein Professor of the Economics of Medicine Charles A. Sanders Center for Community Health and General Director Medical Care Massachusetts General Hospital Harvard Medical School Kerr L. White Melvin A. Glasser Deputy Director Director Health Sciences Social Security Department Rockefeller Foundation United Auto Workers vi ADVISORY PANEL ON COMPUTER TECHNOLOGY IN MEDICAL EDUCATION AND ASSESSMENT Richard B. Friedman, Chairman Associate Professor of Medicine and Human Oncology University of Wisconsin—Madison Edward N. Brandt, Jr. Edithe J. Levit Vice Chancellor for Health President and Director University of Texas System National Board of Medical Examiners Laurel A. Cappa Frederick T. Merchant Resident, Program in Social Medicine Permanent Secretary of the Federation Montefiore Hospital Licensing Examination (FLEX) Board Bronx, N. Y. Lawrence Miike * * Lloyd R. Evans Consultant on Health Policy Chairman, Federation Licensing Berkeley Springs, W. Va. Examination (FLEX) Committee Harold F. O’Neill Jr. Michael Goran* Research Psychologist Director, Health Standards and U.S. Army Research Institute for the Quality Bureau Behavioral & Social Sciences Office of Professional Standards Review Organizations Richard Spohn Department of Health, Education, Director and Welfare California Department of Consumer Affairs William G. Harless August Swanson Union Graduate School/ West Director San Francisco, Calif. Department of Academic Affairs Association of American Medical Colleges ● Resigned position in October 1978; currently serves as Medical Director, Rockridge Health Care Plan, Oakland, Calif. * *Assumed co-directorship of the study in November 1978. vii CONTENTS Chapter Page 1. SUMMARY AND ANALYSIS . 3 Introduction . 3 Organization of the Report . 4 Conclusions. 4 Implications. , . 6 2. COMPUTER TECHNOLOGY IN EDUCATION AND ASSESSMENT. 13 The Technology. 13 Description . 13 Trends. 18 Computer-Based Education . 20 Methods. 20 Strategies. 21 Funding . 23 Limitations and Capabilities. 24 3. MEDICAL EDUCATION AND ASSESSMENT . 29 Introduction . 29 Undergraduate Medical Education. 30 Admissions . 30 Curriculum . 32 Assessment of Student Performance. 33 Graduate Medical Education . 34 Licensure and Specialty Certification. 35 I.icensure . 35 Specialty Certification . , . 37 Emerging Developments. 38 Clinical Practice. 39 4. COMPUTER APPLICATIONS IN MEDICAL EDUCATION AND ASSESSMENT. 47 Introduction . 47 Computer-Supptortecl Independent Study Program in the Basic Medical Sciences 48 The Use of Computer-Based Simulations in Education and Assessment . 55 Introduction. 55 Pediatric Fellowship Examination of the Royal College of Physicians and Surgeons of Canada . 57 The Computer-Based Examination (CBX) Project of the American Board of Internal Medicine and the National Board of Medical Examiners . 60 A Model for Evaluation andl Recertification Through Individualized Testing (MERIT) of the American Board of Internal Medicine. 60 Computer-Based Medical Consultation Systems . 63 Computer Uses in Quality Assurance Data Management Systems. 65 Medicaid Management Information System. 65 Professional Standards Review Organization . 65 Computer-Stored Ambulatory Record System . 66 Problem Oriented Medical Information System . 66 Summary. 67 ix — .———. 5. DEVELOPMENT AND DISSEMINATION OF COMPUTER-BASED EDUCATION . 71 Overview . 71 Institutional Development and Use . 71 The Ohio State University College of Medicine . 73 The University of Illinois School of Basic Medical Sciences. 76 Massachusetts General Hospital. 79 Networks, Consortia, and Other Dissemination Mechanisms . 80 The Health Education Network . 81 The Computer-Assisted Teaching Systems Consortium . , . 84 Milliken Communications Corporation , . 85 APPENDIX I–EXAMPLES OF SIMULATIONS OF THE PATIENT-PHYSICIAN ENCOUNTER. 91 REFERENCES . 125 LIST OF TABLES Table Number Page 1. Computer-Based Education Course Development . 19 2. A Sample Spread of Computer-Based Education Funding and Development Sources as of 1974 . 24 3. Thirty-two Studies of Computerized Simulations and Testing . 25 4. Number of Medical Schools, Graduates, and Physicians in Practice, Selected Years . 32 5. Jurisdictions With Authority to Require Continuing Medical Education for Reregistration of the License to Practice Medicine . 38 6. Implementation of Recertification by the Specialty Boards, . 39 7. The Independent Study Program Curriculum Outline . 494$) 8. Sample Practice Profile to Select Appropriate CASEs. 62 9. Institutions Producing Computer-Based Materials . 73 10. Computer-Assisted Instruction Regional Education Network (CAIREN) Membership Profile. 74 11. Accreditation of Computer-Based Continuing Education Courses by Professional Societies . 75 12. The Ohio State University College of Medicine Computer-Based Education Programs Indexed by Audience . 75 13. College of Medicine Computer-Based Education User Group Summary. 76 14. Computer-Based Education Utilization Summary for the College of Medicine . 77 15. Massachusetts General Hospital: Computer-Assisted Continuing Medical Education Program (1979) . ..