Communications Protocol Testing: Balancing Technology, Planning, and Compliance by David A. Vogel, Ph. D. and David S. Bernazzani both from Intertech Engineering Associates, Inc. as published in Medical Electronics Manufacturing, Fall 2004 The design and validation of communications • Facilitating parallel development and testing protocols for intra-device communications activities. and for external communications must be But, decentralized design philosophies also introduce handled with the same care as other device new challenges to instrument designers, implementers, software. This article addresses some of the and testers. Challenges are especially apparent when design, validation, and regulatory compliance the microprocessors in a device are to communicate challenges related to communications directly with each other. When the integration of microprocessor-based subsystems are inadequately protocols. planned, implemented, and tested, it often leads to esting the increasing number of microprocessors discovery of defects after instruments are released to that need to communicate reliably with each COMPANY PROFILE Tother within medical devices poses not one but three different testing challenges: technical, planning Intertech Engineering Associates, Inc. (resources, schedule, budgets, etc.), and compliance. It’s a rare electronic medical device that does not make use Address: 100 Lowder Brook Avenue of microprocessors in its design. Increasingly, designs Suite 2500 are taking advantage of low-cost microprocessors to Westwood, MA 02090 partition functionality. This is a change from earlier www.inea.com - (781) 801-1100 designs in which more functionality was centralized Industry: (Electro)Medical Devices into a more powerful general-purpose processor that Services: Assessments Training relied on an array of external devices. Consulting Hands-on Engineering The benefits of a partitioned design philosophy Skills: Product Design include: Risk Management • Keeping hardware and software simpler and Requirements Engineering Electronics Development easier to develop and test. Software Development • Keeping functionality separate (fire-walling) Software Verification and Validation Production/Quality System Software Validation as a protective measure from random software defects. the field. The integration testing of large multiprocessor It is important to understand and address FDA’s systems should focus on interfaces, many of which are suggestions for the validation of communications communications interfaces. interfaces. But, is that the only reason to undertake a serious validation test effort? Defects in communications Compliance Challenges software are notorious for escaping detection under FDA’s guidance, “General Principles of Software laboratory test conditions only to surface shortly after Validation,” mentions communications or interfaces release of a device to the marketplace. several times under the section relating to “Activities Some field defects are benign and often are not even and Tasks – Typical Tasks Supporting Validation.”1 identified as defects. Others, however, can become a The guidance recommends that “typical software nuisance to users, and still others can create safety issues. requirements specify … the definition of all external The effort expended on validation should be proportionate and user interfaces, as well as any internal software- to the complexity of the communications software (i.e., to-system interfaces.” It points out that “external likelihood of failure) and the severity of the consequences interfaces are those with other software (including of its failure. If the consequences include a risk of harm operating system software), system hardware, and the to a user, the validation effort should be substantial. A users.” This definition covers quite a bit of territory. substantial communications software validation effort This article focuses on communications between might also be justified if a communications software microprocessor-control subsystems within a device failure could trigger a device recall, resulting in financial and digital communications with external devices or harm to the device manufacturer. computers. Planning Challenges As device development progresses from requirements Beyond the quality planning suggested by FDA’s definitions to design, the guidance recommends that guidance, considerable effort should be put into the “the software design specification should include … technical planning for development and testing of device communication links (links among internal modules of communications. the software, links with the supporting software, links with the hardware, and links with the user).” Communications software is unique in that the software for both sides of the communication needs to be working Furthermore, as designs are evaluated by peer team to fully exercise the software on either side (unless some members and reviewed with management, the guidance type of simulation is used). Planning must establish the recommends that “an analysis of communication links order in which the communicating pieces of software are should be conducted to evaluate the proposed design to be brought up and whether communications simulators with respect to hardware, user, and related software are needed to help debug the software. requirements.” For similar reasons, testing communications software The guidance also covers testing of communications should also be planned carefully. It is quite common to within a device under integration-level testing. It design special connections and switches into hardware to suggests that “integration-level testing focuses on the facilitate development and testing. In some cases, special transfer of data and control across a program’s internal equipment must be purchased, rented, or custom-designed and external interfaces.” Further, the guidance says to fully test a communications interface. Unexpected that “these tests should provide a thorough and rigorous costs and delays in the development and test schedules examination of the software product’s compliance with may result if this is not planned adequately in advance. its functional, performance, and interface definitions and requirements.” Communications testing can also require hardware modifications to allow access to the communications “Communications Protocol Testing: Balancing Technology, Planning and Compliance” page as published in: Medical Electronics Manufacturing - Fall, 004 Copyright 004 - Intertech Engineering Associates, Inc. signals. Part of the planning for communications testing Finally, a good communications specification also should consider this to ensure that required switches, includes use of case-based data-flow diagrams that show connections, and other monitoring and control access typical communications sequences that are anticipated points are factored into the hardware and software in a normally functioning system, as well as in adverse development plans. system conditions such as dropped communications, Perhaps most important, communications testing is like nonsense message receipt, alarm conditions, system all other forms of verification and validation testing power up and down, and restart. in the sense that software cannot really be tested if Technology Challenges the test engineer doesn’t know how it is supposed to Communications via Ethernet, RS-232, or RS-422, etc., work. As with most validation efforts, detailed, testable with external devices or computers can be monitored requirements are perhaps the most important ingredient and sometimes tested with commercial off-the-shelf for success. Useful communications testing requires that test equipment. More often than not, these devices a detailed protocol requirements specification be in place are communications monitors, and, while useful before initiating testing activities. These specifications for examining the communications traffic on the are also called interface requirements specifications, communications medium, they are typically limited communication protocol specifications,or interprocessor in their ability to insert errors to test the handling on communications specifications. These specifications both sides of the communications. should include high-level descriptions of: Development systems, specifically debuggers and in- • Communication timing requirements circuit emulators, are often used to test communications • Message initiation and acknowledgement software. The procedure is to set breakpoints in the sequencing communications software to stop the system so that the • Details on message structure (envelope, test engineer can view the contents of a buffer that is synchronization patterns, error checking and full of information received or about to be transmitted. correction, addressing, sequence numbers, etc.) This approach is useful for some testing; however, it • Handling of unexpected or out-of-sequence is labor-intensive and time-consuming, and it requires messages a skilled tester. This approach also is limited in its • Command groups ability to test a system in full operation at its stressed • Error detection, reporting, and recovery timing limits. • Resynchronization details. Most communications protocols require rapid Additionally, a command dictionary should include sequencing of messages and timely responses that details on: make testing difficult when using debugger breakpoints. • The structure of each command Often the testing is confined not only by the technical • Data elements