The most suitable person to establish quality assurance guidelines for the generation and use of noncommercial clinical software is a medical physicist Diane Kelly, Alan Wassyng, and Colin G. Orton Citation: Medical Physics 41, 090601 (2014); doi: 10.1118/1.4883877 View online: http://dx.doi.org/10.1118/1.4883877 View Table of Contents: http://scitation.aip.org/content/aapm/journal/medphys/41/9?ver=pdfcov Published by the American Association of Physicists in Medicine Articles you may be interested in Quality assurance and training procedures for computer-aided detection and diagnosis systems in clinical usea) Med. Phys. 40, 077001 (2013); 10.1118/1.4807642 Report of AAPM TG 135: Quality assurance for robotic radiosurgery Med. Phys. 38, 2914 (2011); 10.1118/1.3579139 Patient-specific quality assurance method for VMAT treatment delivery Med. Phys. 36, 4530 (2009); 10.1118/1.3213085 IAEA Technical Reports Series No. 430: Commissioning And Quality Assurance Of Computerized Planning Systems For Radiation Treatment Of Cancer Med. Phys. 33, 561 (2006); 10.1118/1.2167371 Dosimetric evaluation of the clinical implementation of the first commercial IMRT Monte Carlo treatment planning system at 6 MV Med. Phys. 31, 2771 (2004); 10.1118/1.1786172 POINT/COUNTERPOINT Suggestions for topics suitable for these Point/Counterpoint debates should be addressed to Colin G. Orton, Professor Emeritus, Wayne State University, Detroit: [email protected]. Persons participating in Point/Counterpoint discussions are selected for their knowledge and communicative skill. Their positions for or against a proposition may or may not reflect their personal opinions or the positions of their employers. The most suitable person to establish quality assurance guidelines for the generation and use of noncommercial clinical software is a medical physicist Diane Kelly, Ph.D. Department of Mathematics and Computer Science, Royal Military College, Kingston, Ontario K7K 7B4, Canada (Tel: 613-541-6000 ext. 6171; E-mail: [email protected]) Alan Wassyng, Ph.D. Department of Computing and Software, McMaster University, Faculty of Engineering, Hamilton, Ontario L8S 4L8, Canada (Tel: 905-525-9140 ext. 2607, E-mail: [email protected]) Colin G. Orton, Ph.D., Moderator (Received 14 May 2014; accepted for publication 16 May 2014; published 5 August 2014) [http://dx.doi.org/10.1118/1.4883877] OVERVIEW crease the trustworthiness of scientific software. She teaches a graduate seminar course to both RMC and Queen’s students Noncommercial software is widely used in radiation ther- that critiques software development and quality assurance ap- apy, especially in academic centers. Such software can take proaches popular in software engineering when specifically many forms, ranging from MU-check spreadsheets; to in- applied to scientific software. Dr. Kelly has a Ph.D. and house built treatment planning and plan-evaluation systems; M.Eng. in Software Engineering, both from RMC, and a B.Sc. or to public domain codes such as EGSnrc or MNCP that in Pure Mathematics and B.Ed. in Mathematics and Computer are interfaced to IMRT planning systems. This noncommer- Science, both from the University of Toronto. She worked in cial software is usually written by in-house medical physi- the nuclear industry for over 20 years as a scientific software cists and, thus, the question arises as to how to establish the developer, technical trainer, and QA advisor. She is a senior integrity of the software they have written. One view has been member of IEEE. that only software professionals are properly equipped to pro- vide quality-assurance guidance on the generation and use of such software. An opposing view is that the medical physi- Arguing against the Proposi- cists themselves are in the best position to determine effective tion is Alan Wassyng, Ph.D. practices to address software quality in their community, and Dr. Wassyng earned his Ph.D. this is the premise debated in this month’s Point/Counterpoint. in Applied Mathematics from the University of the Witwa- tersrand, Johannesburg, South Arguing for the Proposition Africa in 1979. After spend- is Diane Kelly, Ph.D. Dr. ing 14 years as an academic, Kelly is an Associate Profes- first at the University of Wit- sor in the Department of Math- watersrand and then at the Uni- ematics and Computer Science versity of Minnesota, he incor- at the Royal Military Col- porated a computer consulting lege of Canada (RMC). She company in Toronto, Canada. is cross appointed to RMC’s He returned to academia in 2002, joining the Department Department of Electrical and of Computing and Software at McMaster University, Hamil- Computer Engineering and to ton, ON, Canada, where he is currently Associate Profes- the School of Computing at sor and Director of the Centre for Software Certification. Queen’s University. Dr. Kelly’s Dr. Wassyng has published widely on software certifica- research focuses on ways to in- tion and the development of dependable embedded systems. 090601-1 Med. Phys. 41 (9), September 20140094-2405/2014/41(9)/090601/4/$30.00 © 2014 Am. Assoc. Phys. Med. 090601-1 090601-2 Kelly and Wassyng: Point/Counterpoint 090601-2 He is cofounder of the Software Certification Consortium, Kendall and Post, after studying decades of development and is Co-PI on the highly funded “Certification of Safety- of nuclear arsenal simulation software at the Los Alamos Na- Critical Software Intensive Systems” project led by McMaster tional Laboratory, concluded that the best people to draw up a University. list of “best practices” for software development and main- tenance are the members of the code project teams them- selves, and that these practices are those “. that the teams FOR THE PROPOSITION: Diane Kelly, Ph.D. have judged useful for improving the way they do business”.5 It is the same for medical physicists. They know the best Opening Statement ways to assess their software in order to safely do their Presentations at the Canadian Organization of Medical business. Physicists (COMP) Winter School1 showed that medical physicists are deeply imbued in a safety culture. They re- AGAINST THE PROPOSITION: Alan Wassyng, Ph.D. act instinctively within this culture, pay attention to human- technology interaction, and exhibit due process in the light of Opening Statement safety concerns. I compare this to the environment of a soft- This proposition may make some sense if the guide- ware engineering colleague who specializes in testing: she lines are solely for the use of the software. However, med- lives in a volatile, market-driven, and cost-minimizing envi- ical physicists (MPs) are experts in medical physics, not ronment. Even though she has years of experience in testing the software fundamentals necessary for establishing soft- different products, her instincts and her quality goals are dif- ware quality assurance (SQA) guidelines for software de- ferent from those of a medical physicist. velopment. A history of creating software is not necessar- The software engineering literature does not acknowledge ily a plus. There are lots of people writing software, many the need for the conjunction of computational software de- of whom are not really qualified to do so, and the depend- sign processes with a deep safety culture, which is required ability of software, in general, is dismal.6 Software quality for deployment of software used to support clinical decision in the medical domain should be evaluated on the basis of making. Instead, such software is confused with either con- safety, security, and dependability, and paraphrasing “official” trol software, which directly operates a medical device, or definitions:7 commercial products where patient wellbeing is not directly r affected by the correctness of the output. As a result, there Software safety – under defined conditions, software are no guidelines in the software engineering literature that should not contribute to unsafe behavior or generate re- address the specific characteristics and needs of clinical soft- r sults that can lead directly to harm; ware. Software security – protection afforded the software to When advising on software quality guidelines, a typi- keep it from harm, and from causing harm through users cal software engineer takes a broad-spectrum approach. This maliciously bypassing the software’s designed-in safety approach suffers from two serious flaws. First, it encour- r and dependability; ages the perception that software is correct unless proven Software dependability – in its intended environment, otherwise. This dangerous assumption has been a contrib- the software can be trusted to produce the outputs for utor to several fatal accidents in the safety-critical world.2 which it was designed, with no adverse effects. A recent article3 talks about problems “when a computer Anyone establishing SQA guidelines for the generation of lulls us into a false sense of security”. Second, this broad- software must fully understand fundamental principles of spectrum approach does not use the knowledge of the peo- software that relate to safety, security, and dependability. ple associated with the software, and does not acknowledge Examples are: the specifics of the operational environment that these people r understand. Lack of continuity8 (Moderator: in software engineer- Vessey and Glass4 criticize the software engineering com- ing, continuity refers to a continuous function for which, munity for their broad, generic solutions, calling them weak intuitively, small changes in the input result in small solutions. They contend that the most effective and strong so- changes in the output) – If a bridge is built to withstand lutions are those that target the specific environment or situ- a force of 100 tonnes, because of the mostly continu- ation. Medical physicists, with their knowledge, can provide ous behavior of physical objects and our mathematical this strong solution. models, the bridge is likely to be safe for loads less than In a 2012 survey of software development and mainte- or equal to 100 tonnes. We do not expect it to collapse nance practices sent to medical physicists across Canada, the if a bicycle goes across it.