Innovation Report Why Medical Schools Should Embrace Wikipedia: Final-Year Medical Student Contributions to Wikipedia Articles for Academic Credit at One School Amin Azzam, MD, MA, David Bresler, MD, MA, Armando Leon, MD, Lauren Maggio, PhD, Evans Whitaker, MD, MLIS, James Heilman, MD, Jake Orlowitz, Valerie Swisher, Lane Rasberry, Kingsley Otoide, Fred Trotter, Will Ross, and Jack D. McCue, MD Abstract Problem course on student participants, and improved their articles, enjoyed giving Most medical students use Wikipedia readership of students’ chosen articles. back “specifically to Wikipedia,” and as an information source, yet medical broadened their sense of physician schools do not train students to improve Outcomes responsibilities in the socially networked Wikipedia or use it critically. Forty-three enrolled students made information era. During only the “active 1,528 edits (average 36/student), editing months,” Wikipedia traffic Approach contributing 493,994 content bytes statistics indicate that the 43 articles Between November 2013 and November (average 11,488/student). They added were collectively viewed 1,116,065 2015, the authors offered fourth-year higher-quality and removed lower- times. Subsequent to students’ efforts, medical students a credit-bearing course quality sources for a net addition of these articles have been viewed nearly to edit Wikipedia. The course was 274 references (average 6/student). As 22 million times. designed, delivered, and evaluated by of July 2016, none of the contributions faculty, medical librarians, and personnel of the first 28 students (2013, 2014) Next Steps from WikiProject Medicine, Wikipedia have been reversed or vandalized. If other schools replicate and improve Education Foundation, and Translators Students discovered a tension between on this initiative, future multi-institution Without Borders. The authors assessed comprehensiveness and readability/ studies could more accurately measure the effect of the students’ edits on translatability, yet readability of most the effect of medical students on Wikipedia’s content, the effect of the articles increased. Students felt they Wikipedia, and vice versa. Problem they have read about on the Internet. traditional sources of medical knowledge The rise and spread of Internet accessibility Although medical educators typically train (e.g., textbooks and journals), the idea students to address patient misperceptions of benefiting from active contribution has created an unprecedented resource for in clinical practice, they are not leveraging to crowd-sourced resources such as the dissemination of medical information, clinician and medical student knowledge Wikipedia remains the perspective of as well as an invaluable tool for health to improve the quality of the information a minority or fringe group within the care providers and the general public patients and others find online. academic medicine community. alike. At the same time, opportunities for the rapid spread of misinformation Wikipedia is a free, online, multilingual We believe that not contributing to or the misinterpretation of medical facts encyclopedia that is continually and crowd-sourced resources represents a lost have never been greater. Clinicians must collaboratively created. Anyone with an opportunity for enriching medical students’ sometimes gently redirect patients who are Internet connection can edit its articles. learning and for disseminating more convinced that they have some rare disease One of the most frequently visited Web sites accurate, up-to-date medical information worldwide, it is among the leading sources of to Wikipedia’s readers worldwide. Please see the end of this article for information health information for medical professionals about the authors. and health care consumers alike.1 At the Approach Correspondence should be addressed to Amin end of 2013, Wikipedia’s medical content Azzam, UCSF Department of Psychiatry, Box 0984- included over 155,000 articles written in 255 We created what is, to our knowledge, MSE, 401 Parnassus Ave., San Francisco, CA 94143- 0984; telephone: (415) 476-7836; e-mail: Amin. languages, supported by more than 950,000 the first formal medical school course [email protected]; Twitter: @AminMDMA. references.2 Despite its increasingly prevalent worldwide through which medical This is an open access article distributed under the use as a medical information resource in students actively work to improve Creative Commons Attribution License 4.0 (CCBY), clinical practice, clinical instructors and Wikipedia’s health-related articles. We which permits unrestricted use, distribution, and faculty members often dissuade medical expected enrolled students to hone their reproduction in any medium, provided the original work is properly cited. students from using Wikipedia, citing information retrieval and assessment concern for its perceived inaccuracies and skills,4 practice communicating medical Acad Med. XXXX;XX:00–00. lack of traditional editorial controls.3 knowledge to an exceptionally broad First published online 2 doi: 10.1097/ACM.0000000000001381 global audience, and expand their sense Supplemental digital content for this article is Although physicians and medical of health care providers’ roles in the available at http://links.lww.com/ACADMED/A386. students are encouraged to contribute to Internet age. We designed our course with Academic Medicine, Vol. XX, No. X / XX XXXX 1 Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited. Innovation Report assistance from WikiProject Medicine, of the course. WikiProject Medicine not always certain, a priori, how much a volunteer group of experienced maintains a list of the most frequently students can accomplish in a month, we Wikipedia editors who seek to ensure accessed health-related Wikipedia articles, encourage them to improve their selected that the general public and health care ranked by importance, and graded articles “as much as feasible.” Thus far, professionals have access to free, current, according to an article quality scheme that students have completed the majority accurate, and understandable medical is applied to most Wikipedia Projects.5 We of their editing work independently. information in their own language. encourage (but do not require) students Instructors (A.A. and J.H.) and medical to select a “top-” or “high-importance” librarians (E.W. and L.M.) provide Thus far, we have run the course four article that has also been tagged as intermittent encouragement via e-mail, times: November 2013, April 2014, needing quality improvement. At the hold weekly office hours, and schedule November 2014, and November 2015. beginning of the course, students receive consultation as needed. At the end of the Each cycle has begun with a two-day custom reports for their chosen articles course, faculty provide students access to orientation (Table 1) during which we that contain both a list of grammar and custom-built tools that allow comparison introduce students to Wikipedia’s editorial style errors and a quantitative readability of summary statistics of each article on tools, style, and standards. Additional score. These Acrolinx (San Jose, the first versus last day of the course. didactic activities include reviewing California) reports use specialized natural While presenting the final versions of guidelines for writing simplified English language processing software, which their Wikipedia pages, all students share and strategies for locating and evaluating considers sentence length and over 100 highlights of their accomplishments and source material quality. grammar and style rules. challenges, describe lessons learned, and provide feedback for future iterations of All students establish their own Wikipedia Detailed course structure and materials the course. user accounts and choose a single health- are publicly available under free license related article to edit over the remainder on our course Wikipedia page.6 As we are This report summarizes the outcomes of our pilot initiative at the University of California, San Francisco (UCSF). The Table 1 UCSF institutional review board reviewed Instructional Content During Initial Two-Day Orientation Session of UCSF Wikipedia- this research and deemed it exempt from Editing Course formal review. Day Content Content provider Effect on Wikipedia articles 1 Course structure Course faculty and guest We compared the state of the Wikipedia • Objectives instructors articles before and after student editing • Logistics using several approaches. Two UCSF- • Expectations affiliated physicians (E.W. and J.D.M.) • Defined deliverables provided independent subjective ratings WikiProject Medicine Members of the WikiProject of article changes and reconciled their Medicine community • Why Wikipedia matters and how it works impressions by consensus. Wikipedians • Managing your first edits with experience rating medical articles Global reach of Wikipedia Chief executive officer but unaffiliated with the elective received • Principles of simple writing of Content Rules (text blinded unedited and edited versions of simplification company) • Principles of writing for translation each article to evaluate using the general • Translators Without Borders and WikiProject WikiProject article-grading rubric. We Medicine collaboration obtained changes in word, paragraph, and 2 Wikipedia editing Experienced Wikipedian and citation counts using Wikipedia’s analysis • Core policies and the