Using Quality System Regulations and Fda Design Control Guidance As a Basis for Capstone Senior Design

AC 2010-409: USING QUALITY SYSTEM REGULATIONS AND FDA DESIGN CONTROL GUIDANCE AS A BASIS FOR CAPSTONE SENIOR DESIGN

Robert Gettens, Western New England College Michael Rust, Western New Engalnd College Assistant Professor of Biomedical Engineering

Diane Testa, Western New England College Judy Cezeaux, Western New England College Page 15.1335.1

Abstract Medical device development in the industrial setting follows the tenets of Quality System Regulations (QSR) and the design control guidance of the U.S. Food and Drug Administration (FDA). Many biomedical engineers learn the language and practices of QSR and design controls on the job. Experiential learning in these areas gives biomedical engineering graduates a valuable skill set coveted by medical device companies. This skill set will position biomedical engineers apart from other engineering disciplines and will help more completely define the biomedical engineer.

The Biomedical Engineering Department at Western New England College has developed an approach to the capstone senior design course which integrates QSR and design controls into the curriculum. This integration uses an experiential method in which students follow the guidelines for design control and QSR, closely mimicking best practices seen in the medical device industry.

The idea to incorporate QSR and FDA design control guidance was generated largely through the Department’s industrial advisory board. Members of our board from the medical device industry see a knowledge gap in QSR and design control in recent hires from the general pool of engineering graduates. The incorporation of these elements into our capstone design course, not just in theory, but in practice, seeks to alleviate this gap.

Introduction According to the 2009 AIMBE biomedical engineering placement survey, 49% of bachelor-level graduates obtained employment in industry. 1 The U.S. Department of Labor projects an employment growth rate of 72% for biomedical engineers in the decade 2008-2018. This growth rate is much faster than for other engineering disciplines. 2 Reasons for this projected rapid increase include the demand for more technically sophisticated medical devices due to an aging population, and concern for the development of more cost effective medical procedures. 2 This increased demand coupled with an existing trend of engineers going to the medical device industry necessitates a change in the academic setting to better prepare and train Page 15.1335.2 these engineers for careers in biomedical device and related industries. The objective of this paper is to present an experientially-based pedagogical method using the senior capstone design course to train engineers directly in the procedures of the Quality System Regulation (QSR), thus better preparing graduates for careers in the biomedical device workplace.

A pilot survey of faculty, students and industry sources concerning engineering design courses across disciplines demonstrated an emerging theme of learning and development of professional skills in these courses. 3 Indeed in recent years the importance of preparing biomedical engineers professionally through the use of the capstone design course has been stressed by a number of programs. 4-6 Pedagogical techniques being used in biomedical engineering curricula to introduce students to “real-world problem-solving”, which was presented by Ropella, Kelso and Enderle, include the use of computer simulation, internships and cooperative education, guest speakers, guest instructors, field trips, bioethics instruction and problem-centered instruction. 5 At Bucknell, a four course sequence over the Junior and Senior Years was implemented in order to introduce students to such skills as regulatory issues, teamwork, environmental impacts, formal decision making, computer-aided design, machining, rapid prototyping, cell culture and statistical analysis. 4 Importantly these skills are taught and practiced prior to embarking on the senior capstone design project. 4 At the University of Virginia professional skills such as job searching, interviewing, written and oral communication, ethics, negotiation skills, leadership, intellectual property and entrepreneurship have been integrated into the senior capstone design course. 6 Our capstone design course offers an experiential method that builds upon these professional skills.

For engineers to be effective in the medical device industry they must be familiar with and be able to adhere to Food and Drug Administration (FDA) regulations as outlined in Title 21 of the U.S. Code of Federal Regulations. Section 820 of Title 21 governs QSR. The design controls put forth in Subsection 820.30 of the QSR are of particular importance to engineers involved in the design process. A summary of 21CFR820.30 from a user perspective is outlined in the FDA design control guidance document. 7

The importance of design over research projects is firmly established for senior capstone design courses, particularly as directed by guidelines of the ABET, Inc.8 Therefore, since accredited biomedical engineering programs must offer design-based projects and design in the biomedical device industry must follow the design controls put forth by 21CFR820.30, it is

logical that academic programs should attempt to incorporate these regulations into the capstone Page 15.1335.3 design course to some extent. In previous biomedical engineering education conferences hints of merging these two concepts were presented. At the 2009 BME-IDEA Biennial meeting the incorporation of 21CFR820.30 in the Case Western Summer Design Experience was presented. 9 A discussion of the need for and current resistance to incorporating design controls into the capstone design course was discussed by Jay Goldberg in the IEEE Engineering in Medicine and Biology Magazine. 10

Prior to employing this method of delivering the capstone project we followed a more traditional academic structure. At that time, the course structure was a two semester sequence of senior capstone design. A fall written and oral proposal was followed by spring project execution and final oral defense and written report. The emphasis of the projects was engineering design even though an academic structure was in place.

The impetus behind our endeavor to integrate 21CFR820.30 into our senior capstone course came from our industrial advisory board. Members of the board, and specifically those from the biomedical industry, indicated to our department that the new hire engineers they were employing had only a cursory knowledge of FDA regulations, the quality function and design control. We were advised to better incorporate 21CFR820.30 into our senior capstone course. It was pointed out that knowledge of the FDA design control process could be one of the major skill sets separating biomedical engineers from other engineers. This would make the undergraduate biomedical engineer an attractive asset for a medical device employer.

This paper outlines a method to incorporate 21CFR820.30 into a capstone design course. It should be noted that the method attempts only to simulate working in the biomedical device industry. The method does not and could not replace the massive workforce and procedural documentation required to obtain FDA approval for a biomedical device.

General Course Structure

The general course structure used in this work incorporates many of the tenets put forth in Jay Goldberg’s book on biomedical engineering capstone design courses. 11 Similar to many programs, the senior capstone design project is delivered in a series of two courses. A 3-credit fall course covers the initial phases of the design process. A 4-credit spring course builds on the fall course and incorporates the majority of the prototype fabrication process and device testing. During both semesters students meet with faculty advisors for weekly status update reports. Page 15.1335.4 These updates last roughly one hour. Meetings with clinical and industrial advisors are also encouraged. The fall course includes a weekly lecture followed by a working laboratory section later in the week. The lecture typically introduces the topic to be covered in the working section. Lecture topics cover areas of professionalism focused around the FDA design control guidance. Written deliverable documents based on working sessions are scheduled to document the design process as well as guide the students toward successful completion of their project. A summary of the presented lectures, working sessions and project deliverables (due dates are for the draft forms) is shown (Table 1).

Table 1: General course design for the fall section of the capstone design course. Lecture is for 1 hour. Lab activities range from 3-4 hours. All deliverable due dates are for draft documents to guide student project planning.

Page 15.1335.5

Ideas from several other programs were incorporated in this work. An example is the two week introductory design experience used at Bucknell University and presented at the 2009 BME- IDEA Biennial conference. 12 Rather than offer the activity at the start of the semester, as Bucknell did, we offered it midway through the course (Table 1: week 7). Initial feedback from students indicated that this timing was ideal, since at that point in the course they were familiar enough with the design process to effectively engage the exercise.

Incorporation of Design Control

Design Reviews

The course structure outlined above was built around the FDA design control guidance. Design controls were built into the design process using the traditional waterfall model presented in the FDA Design Control Guidance (Fig. 1). 7 For the fall course, design reviews are held at the first three phases of the design process, that is, after “user needs” solicitation, creation of design input and finally after the design process. The final design review constitutes a design freeze, and is held with a large community of clinical and industrial experts outside of the institution as well as engineering faculty members. Ideally this would be the case for all of the design reviews, but has not been implemented due to practical considerations. In the spring semester design reviews focus on the design output and ideally design verification and validation.

Figure 1: Traditional waterfall process reproduced from the FDA design control guidance (a federal document). 7 The fall capstone design course focuses on the first three phases of the design process (orange oval), while the spring semester focuses on the final two Page 15.1335.6 phases of the design process (green oval). Design History Files

In addition to focusing design reviews on the FDA design control model, the capstone program mimics the project documentation required for the development of a biomedical device. It should be stressed that the capstone project only mimics the documentation required for a biomedical device. The major portion of this documentation is the maintenance of a design history file (DHF) (Fig. 2). In an industrial setting the file would more appropriately be called a project folder since students log additional information in the folders than would typically be required for an FDA audited DHF. Examples of this additional information are inclusion of project planning and financial aspects of the project. It would not be practical to include an entire design history file here due to size limitations. An example table of contents is included (Fig. 3) to give a feel for the types of documents included in a design history file using our method.

Figure 2: Examples of typical student design history files. These files are maintained by each student and assessed at the end of each term. The opened file shows a completed change control form for one deliverable. Page 15.1335.7

Figure 3: An example table of contents from a design history file. Due to size limitations posting an entire file is not practical. This table of contents gives an idea of the types of documents included in a design history file using our method.

The majority of capstone projects in our program are medical device projects and most students also move on to the medical device industry. A similar program, however, could be tailored for pharmaceutical design history files if such projects become available.

Change Control

Students are instructed that the DHF is a living document thus changes to deliverables are expected and welcomed. These changes are maintained using a formal change control process which mimics that seen in industry. As aspects of deliverables change during the course of the project the most recent version of the deliverable is placed at the top of that deliverable section (Fig. 2). On top of the deliverable document a change control form is placed. The change control form indicates the project, deliverable, revision number, contents, reason for the change and approvals of the project leader (student), technical director and a quality reviewer (Fig. 4). This process mimics the quality system practiced in the device industry. Typically, the technical director is the project advisor and the quality reviewer is another biomedical engineering faculty member. In the industrial setting the quality function would follow a separate hierarchy of supervision. For the purposes of introducing students to design controls, using a second faculty Page 15.1335.8 member as a reviewer was deemed an appropriate model. Note that when all signatures are obtained students receive an approval stamp from the department chair for that deliverable. This does not, of course, preclude follow-on changes to the deliverable.

Figure 3: Example of a change control document for a project deliverable. The student is the project leader while the technical director and quality reviewer are faculty members.

Student Assessment and ABET

Students are assessed by the faculty on professionalism, maintenance of the DHF and performance on design reviews. The professionalism portion of the assessment is based on maintenance of a laboratory notebook, project leadership and preparedness for meetings. The

DHF is assessed based on a grading matrix and rubric for each deliverable (Table 2). Page 15.1335.9 Performance on design reviews are similarly assessed using a grading rubric focusing on 1) the aesthetics of the performance and 2) the technical content of the review.

Table 2: Grading matrix used for the DHF portion of the capstone design course.

Since the DHF is a living document a certain amount of liberty must be given in the assessment

of the deliverables. Faculty must be able to assess the grey area of design, eloquently described Page 15.1335.10 by Gassert et al. in their paper concerning research vs. design in capstone courses. 8 This freedom is particularly needed in the fall semester when certain deliverables based on individual projects may be largely incomplete. Examples of note are the design verification and validation plan as well as global considerations of the design. It should also be stressed that projects evolve at different rates, and this must be taken into consideration. All of these factors are of particular concern when incorporating a design control process such as that described in this paper.

The incorporation of QSR and design controls into the capstone design course is only in its second year with fourteen students having been through the program. Therefore, the data needed for a critical assessment on the impact for graduates in the industrial setting is not yet available. Also we did not yet receive IRB approval to use quantitative information on student performance for research purposes so we are not able to report those data. Initial reports do indicate that the process does indeed better prepare students for the language and requirements of design control and QSR. Additionally, we received very positive feedback from our Industrial Advisory Board on the incorporation of this program. John Kirwan, President of Incite Innovation, LLC gave the following incite in response to the program, "As a biomedical industry veteran, I frequently evaluate skill sets of potential new hires. Having a solid education in the engineering fundamentals coupled with a firm grasp of design controls and quality systems regulation provides recent graduates with the definite advantage of being able to join a R&D group and hit the ground running."

While many of the ABET assessment criteria could be assessed in the capstone design courses our program chooses to specifically assess criteria 3c, 3e and 3h. The criteria definitions are 3c: an ability to apply to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, societal, political, ethical, health and safety, manufacturability, and sustainability, 3e: an ability to identify, formulate, and solve engineering problems and 3h: the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context. A summary of these criteria, the delivery strategy and assessment methods are shown in Table 3.

Page 15.1335.11

Table 3: Summary of the ABET criteria assessed by the two semester senior capstone design courses using design control as a basis for instruction.

Lessons Learned

The program described in this paper has been implemented for one and one half years incorporating two fall semesters and one spring semester. In addition to standard student surveys, formal after action reviews were held at the end of each fall semester. Several valuable lessons were learned from these sessions:

≠ Students felt that the change control process should be streamlined. Page 15.1335.12 ≠ Students felt that the entire faculty, not just the capstone course director, should be better educated on 21CFR820-30 and the implemented program. It should be pointed out that we have full buy-in on this program from our faculty. Our faculty is learning the process along with the students and becoming more and more knowledgeable with QSR as we move forward.

≠ The requirements needed to elicit user needs should be communicated early in the semester so that ample time is given to complete the necessary paperwork. This pertains specifically to institutional review board (IRB) approval to conduct user surveys.

≠ Students felt that underclassmen, freshman-junior level should be invited to and take part in design reviews as outside observers.

≠ Design projects stemming from research have difficulty fitting into the design model presented. These projects should be avoided or the structure should be altered to make allowances.

One area of difficulty for students was taking the lead as the project manager. Students are, of course, used to a unilateral approach in the faculty-student relationship, in which information is given by the faculty member to the student. It may be challenging for students to break this cycle and begin generating knowledge on their own, but this effort is ultimately necessary for their development. It was also found that Gantt charts were an underutilized resource. Students suggested that as part of weekly project meetings they should update and bring project Gantt charts. It was felt that this would help guide them in leading projects and more efficiently use the Gantt chart tool.

Conclusions Knowledge of the requirements to develop a medical device, specifically QSR and design control is one key facet that sets biomedical engineers apart from the other engineering disciplines. Practicing the tenets of design control, rather than simply having those tenets dictated, better prepares biomedical engineers for the medical device workplace. The program described here is an easy to implement system that mimics the design control process in a medical device company. The method provides a means for students to practice being design engineers in the “real-world”. The attainment of this skill set will be a key asset for the biomedical engineering community, setting us apart from our engineering colleagues and making Page 15.1335.13 our students employment exceedingly desirable by the medical device community.

Bibliography 1. Thurston, P., "Academic Council Graduation and Job Placement Rates Survey," American Institute for Medical and Biological Engineering, November 2009. 2. Bureau of Labor Statistics, "Engineers", U.S. Department of Labor, www.bls.gov/oco/ocos027.htm (accessed 1/7/2010). 3. Howe, S., Lasser, R., Su, K., Pedicini, S., "Content in capstone design courses: Pilot survey results from faculty, students, and industry," Proceedings for the 2009 ASEE Annual Conference and Exposition, Austin, TX, June 14-17, 2009. 4. Ebenstein, D., Tranquillo, J., Cavanagh, D., "Developing student design and professional skills in an undergraduate biomedical engineering curriculum," Proceedins for the 114th Annual ASEE Conference and Exposition, Honolulu, HI, June 24-27, 2007. 5. Ropella, K.M., Kelso, D.M., Enderle, J.D.,"Preparing biomedical engineers for real-world problem- solving," Proceedings for the 2001 ASEE Annual Conference and Exposition, Albuquerque, NM, June 24-27, 2001. 6. Allen, T., Peirce-Cottler, S., "Career development and professionalism within a biomedical engineering capstone course," Proceedings for the 2008 ASEE Annual Conference and Exposition, Pittsburg, PA, June 22- 24, 2008. 7. FDA: Center for Devices and Radiological Health, "Design Control Guidance for Medical Device Manufacturers" 2007. 8. Gassert, J., Enderle, J.D., Lerner, A., Richerson, S., Katona, P., "Design versus research; ABET requirements for design and why research cannot substitute for design," Proceedings for the 113th Annual ASEE Conference and Exposition, Chicago, IL, June 18-21, 2006. 9. Tyler D., "Innovations in Preparing for the Undergraduate Design Experience," 2009 BME-IDEA Biennial Meeting, Pittsburgh, PA October 7, 2009. 10. Goldberg, J.R., "Incorporating design controls into capstone design courses," IEEE Eng Med Biol Mag , 27:105-106, 2008. 11. Goldberg, J.R., "Capstone Design Courses: Producing Industry-Ready Biomedical Engineers," Morgan & Claypool Publishers, 2007. 12. Tranquillo, J., "Innovations in Preparing for the Undergraduate Design Experience," 2009 BME-IDEA Biennial Meeting, Pittsburgh, PA October 7, 2009.

Page 15.1335.14