Engineering the Future Workforce Required by a Global Engineering Indus- Try

Paper ID #13811

Engineering the Future Workforce Required by a Global Engineering Indus- try

Dr. Michael Richey, The Boeing Company Michael Richey is an Associate Technical Fellow currently assigned to support workforce development and engineering education research. Michael is responsible for leading learning science research, which focuses on learning ecologies, complex adaptive social systems and learning curves. Michael pursues this research agenda with the goal of understanding the interplay between innovation, knowledge transfer and economies of scale as they are manifested in questions of growth, evolvability, adaptability and sustainability. Additional responsibilities include providing business leadership for engineering technical and professional educational programs. This includes topics in advanced aircraft construction, composites structures and product lifecycle management. Michael is responsible for leading cross-organizational teams from academic, government focusing on how engineering education must acknowledge and incorporate this new information and knowledge to build new methodologies and paradigms that engage these develop- ments in practice. The objective of this research is focused on achieving continuous improvement and sustainable excellence in engineering education. Mr. Fabian Zender, The Boeing Company Fabian Zender is an Engineering Performance Coach at The Boeing Company where he participates in research in the Technical and Professional Learning Solutions group. He obtained his undergraduate and graduate degree in Aerospace Engineering from the Georgia Institute of Technology. In his research Fabian focuses on learning as a sociotechnical system, utilizing data analytics and learning science and combining them with traditional engineering approaches to advance personalized learning and optimize organizational performance. Dr. Charles J. Camarda, NASA Dr. Charles Camarda Biography (Long) Dr. Camarda graduated from Archbishop Molloy High School, Jamaica, New York, in 1970. He received a bachelor of science degree in aerospace engineering from Polytechnic Institute of Brooklyn in 1974 and a master of science degree in engineering science from George Washington University in 1980. In 1990, he received a doctorate in aerospace engineering from Virginia Polytechnic Institute and State University. Upon completing his B.S. degree from the Polytechnic Institute of Brooklyn, Camarda began work for NASA’s Langley Research Center, Hampton, Virginia, in 1974. He was a research scientist in the Ther- mal Structures Branch of the Structures and Materials Division and was responsible for demonstrating the feasibility of a heat-pipe-cooled leading edge for Space Shuttle by analysis, laboratory experiments, and aerothermal testing in Langley’s 8-foot High Temperature Tunnel. He conducted analytical and experimental research in heat pipes, structural mechanics and dynamics, heat transfer, and numerical optimiza- tion for aircraft, spacecraft, and space launch vehicles. While at Langley, Camarda earned his masters’ degree from George Washington University in Engineering Science with emphasis on mechanics of composite structures at elevated temperature and his doctorate degree from Virginia Polytechnic Institute and State University with emphasis on the development of advanced modal methods for efﬁciently predict- ing transient thermal and structural performance. In 1989, Camarda was selected to lead the Structures and Materials Technology Maturation Team for the National Aero-Space Plane (NASP) program, which was responsible for maturing materials and structures technologies necessary to enable the development of an airbreathing hypersonic vehicle capable of horizontal take-off to orbit. Camarda was selected to

head the Thermal Structures Branch (TSB) in 1994 with responsibility for a research engineering staff, Page 26.646.1 two major focused programs (the high-speed research (HSR) and reusable launch vehicle (RLV) programs), and several structural test facilities including the Thermal Structures Laboratory. Some of the primary responsibilities of the TSB are the development of durable, lightweight metallic thermal protec- tion systems (TPS), advanced leading edges for hypersonic vehicles using carbon-carbon material and

c American Society for Engineering Education, 2015 Paper ID #13811 heat pipes, reusable cryogenic tank systems, and graphite-composite primary structure for RLV. Camarda has received over 21 NASA awards for technical innovations and accomplishments. He also received a Research and Development 100 award from Industrial Research Magazine for one of the top 100 technical innovations of 1983 entitled ”Heat-Pipe-Cooled Sandwich Panel.” He holds 9 patents. Selected as an astronaut candidate by NASA in April 1996, Dr. Camarda reported to the NASA Johnson Space Center in August 1996. He completed two years of training and evaluation that qualified him for flight assignment as a mission specialist. Dr. Camarda has been assigned technical duties in the Astronaut Office Spacecraft Systems/Operations Branch, was on the Expedition-8 back-up crew, served as Director, Engineering, Johnson Space Center, and was assigned to the NASA Engineering and Safety Center (NESC). Through the NESC, Dr. Camarda used his technical expertise to evaluate problems and supplement safety and engineering activities for Agency programs. Dr. Camarda flew as MS-5 on the Return to Flight mission STS-114 Discovery (July 26-August 9, 2005), and has logged over 333 hours in space. He currently serves as Senior Advisor for Engineering Development to the Center Director at NASA’s Langley Research Center. Biography (Short) Charles Camarda was born in Queens, New York and received his undergraduate degree in Aerospace Engineering from the Polytechnic Institute of Brooklyn in 1974. Upon graduation, he began work at NASA’s Langley Research Center (LaRC), received his M.S. from GW in Mechanical Engineering in 1980 and a Ph.D. in Aerospace Engineering from VPI in 1990. He was Head of the Thermal Structures Branch at LaRC and led the structures and materials efforts of two programs: The National Aero-Space Plane (NASP) and the Single-Stage-to Orbit Program. He was selected to be an Astronaut in 1996 and flew on the return-to-flight mission of Space Shuttle following the Columbia Accident, STS-114, in 2005. He was selected Director of Engineering at JSC in December 2005 and is now the Sr. Advisor for Engineering Development at NASA’s Langley Research Center. Page 26.646.2

c American Society for Engineering Education, 2015 Engineering the Future Workforce required by a Global Engineering Industry

Introduction The nation’s aerospace workforce is undergoing systemic and disruptive changes including, age distribution, technological advances and transformations as well as global business pressures. Traditional undergraduate programs are not equipping graduates with the skills needed for the complex challenges of the 21st century. 1 These pressures are leading industry to ask the questions; a) how can we partner with academia and the government to advance personalized learning and b) how can we leverage our investment and intellectual capital to increase the quantity/quality and knowledge transfer of the current STEM workforce, education pipeline and labor supply?

Disruptive changes:  Ageing: Roughly a quarter of the nation's 637,000 aerospace workers could be eligible for retirement in 2015. 2  Globalization: Engineers work through global multidisciplinary and distributive teams to optimize business solutions.  Technology: Convergence of competition is connecting IT, infrastructure, automation and economies. These mega trends will converge and future Internet of Things (IOT) interconnectivity and Big Data will reshape the marketplace and drive new innovation into industries and products.

For the U.S. to remain competitive in advanced manufacturing, our students must have access to education opportunities that prepare them for this transformation. This “complex adaptive social system” requires us to rethink the traditional boundaries of engineering and manufacturing education within the broader ecosystem of a sociotechnical framework. The systemic and disruptive changes described above have exposed the skills required by the continuous application of innovative technologies. The dynamics of this complex system, coupled with challenges in the workforce demographics, advances in technology and social connectivity have created an environment requiring dramatic changes in the way we educate students, from primary and secondary to post-secondary education to ensure their future career success. 3

While individual teachers have made great strides in improving the learning of their individual students to accommodate the requirements of a global workforce in the 21st century overall engineering companies and governmental agencies are challenged by the scarcity and quality of graduates produced by the education system at all levels. 1 To better understand this complex sociotechnical system and counter the visible phenomena 4, The Boeing Company and the National Aeronautics and Space Administration (NASA) both engineered capstone programs in partnership with leading educational institutions which prepares students with skills in the science, technology, engineering and mathematics (STEM) areas and do so collaboratively across the United States. This paper will map these new capstone programs detailed below against existing accreditation criteria including The Accreditation Board for Engineering and Technology – ABET 5 and detailed criteria for Knowledge, Skills, and Abilities (KSA’s) described in a recent report titled Transforming Undergraduate Engineering Education, funded Page 26.646.3 by the National Science Foundation (NSF) and published by the American Society for Engineering Education (ASEE). 6

Problem Statement Expanding on the issues described in the introduction, the problems faced by science and engineering (S&E) employers, whether in industry or governmental agencies, are multifaceted and combinatorial. The supply and demand of graduates currently is not in an equilibrium stage, and despite the efforts to expand STEM opportunities, the number of college students pursuing science and engineering is stagnating. 7 Stagnation continues when unemployment is at record lows for S&E graduates, this dynamic defies the “invisible hand” logic and persist after years of investment and countless new programs. 8 Similar trends are widely reported by research centers, professional societies, and consultants among others. 4,9,10

The world is intertwined with the advancing of distributive business processes, i.e., additive manufacturing, big data, massive multiplayer online role playing (MMORPG) technology, and social networking all converging and accelerating the skill gap between engineering education and the workforce. This disruptive landscape presents a significant challenge to future workforce and advanced manufacturing leadership in the United States. This skills gap manifests itself in the unfamiliarity that recent hires often face when working on projects where they are required to collaborate across space and time in an environment with non-optimal data availability requiring them to make decisions that fall outside the narrowly prescribed theoretical scenarios encountered in school. 9 In addition the dialogue between universities and industry has not yet yielded a balance between the academic foundational requirements and industry required application to real world problems. The skills gap is constituted of lacks in both “hard” and “soft” skills. 11,12

In this paper the focus is on identifying opportunities to further develop these professional (“soft”) skills required by small and large companies alike. Almost all graduates in (STEM) fields will have interactions in a social web comprised of colleagues, suppliers, and customers located outside their home state and likely distributed globally. This reality has prompted various professional societies to reevaluate their educational objectives after thorough review with their industry sponsors. 13,14,15 Most notably ASEE recently provided a draft document valid for all engineering disciplines which was developed through various workshops with representatives of both industry and academia. 6

Industry has reacted to this reality by increasingly reaching out to academia and providing input through external advisory boards, research collaborations, or other initiatives. Boeing and NASA partnered with universities to not only provide feedback but actively engage in curriculum development and delivery around actual problems faced in their respective organizations. Universities in their often siloed departmental structure frequently have difficulty to bring real world, multidisciplinary challenges in the context of the education they provide. 16

Capstone course are typically regarded as the pinnacle of the undergraduate engineering experience, not only is each university very particular about its design but even different schools

(e.g. Electrical Engineering vs. Aerospace Engineering) within the same College of Engineering Page 26.646.4 may have very different capstone programs. There are some examples of departments working to bridge these gaps 17 18, but these differences, the associated history within each program, and the fact that capstone courses are often the only vehicle by which some ABET criteria can be fulfilled result in strong protectionist stances within many academic departments when approached with the opportunity utilizing a different approach.

Table 1 shows how a typical capstone, the AerosPACE capstone (Boeing sponsored), and ICED capstone (NASA sponsored) align to the ABET criteria. Both courses have already and still are being delivered. 19 20 While there are of course variations between capstone programs, as previously discussed, there are some general trends that can be observed. In a survey 83% of academic programs responded that there capstone course consisted of department teams, 21 this means that for a large majority of programs the capstone cannot be multidisciplinary as required of the programs in ABET criteria (d). Similarly can criterion (g) effective communication really be met if the only communication required is with people that you can always talk face to face with? Both the AerosPACE and ICED program meet and surpass all ABET program requirements in a single course and because they are steeped in research and utilizing modern interaction platforms allow for true evaluation of student success in each. More details for this will be provided later on.

Page 26.646.5 Table 1. Comparison of Traditional Capstone and Keystone Programs

ABET Criteria Traditional AerosPACE ICED Capstone (a) an ability to apply knowledge of mathematics, science, and engineering (b) an ability to design and conduct experiments,

as well as to analyze and interpret data (c) an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability (d) an ability to function on multidisciplinary

teams (e) an ability to identify, formulate, and solve engineering problems (f) an understanding of professional and ethical

responsibility (g) an ability to communicate effectively (h) the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context (i) a recognition of the need for, and an ability to

engage in life-long learning (j) a knowledge of contemporary issues (k) an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice

Solution Approach The natural alignment of the AerosPACE and ICED programs quickly let to a close collaboration with joint development of future iterations between Boeing and NASA. AerosPACE is a partnership between The Boeing Company, Brigham Young University with a NSF funded Center for e-Design, Georgia Institute of Technology, Purdue University, a Embry-Riddle Aeronautical University, and a Tuskegee University a Historically Black College carrying out a collaborative design, build, fly experience where student teams distributed across the universities over the course of two semesters solve a real life engineering challenge provided by industry. Design requirements are based on national and international needs, e.g. support of first responders on dangerous situations or precision agriculture to increase yield for a growing global population.

NASA partnered with multiple universities (including Massachusetts Institute of Technology, Georgia Institute of Technology, and Penn State) engaging both high school, undergraduate, and graduate students in an epic challenge of global concern, e.g. capturing and retrieving an Page 26.646.6 asteroid. Students meet during the summer for an intense one week workshop to begin their collaboration and use it as a starting point for the two semester endeavor into an epic space problem. 20

Both programs require the students to apply their theoretical knowledge in a new collaborative environment where they do not fight for their personal award (grade) but rather have to rely on the distributed cognition of team members in various domains to solve challenging tasks. Focus switches from competition to collaboration, from disciplinary to multi-disciplinary thinking, from theory to application. Students are mentored and must execute many of the tasks that are part of everyday work experiences for so many, like managing a budget or creating and delivering on project schedules. Students are encouraged to fail fast, early, often, cheap, and smart to develop truly innovative solutions that fulfill customer requirements. 22 Graduates of the program see a clear alignment of these objectives and tasks to their careers following graduation.23

The programs are developed as a partnership requiring both industry representatives and faculty to overcome the multifaceted challenges of such an endeavor. The structure for such a collaboration relies on mutual agreement and understanding of purposes and objectives. 24 Student outcomes have been documented and proven to be successful 19,25,23, but must now be linked back to the realities of academic degrees, mainly the accreditation process.

Accreditation Criteria While many of the above mentioned guidance by professional associations is very detailed, the reality of engineering education today is that programs are accredited based on ABET requirements not desires of professional societies or industry. ABET evaluates programs based on multiple criteria, one of the important ones being student outcomes, criterion 3. 5 Appendix A shows how these criteria are met and measured in the heretofore described programs.

In addition to current ABET criteria, efforts by ASEE have also yielded a list of Knowledge, Skills and Abilities (KSA’s) that are desired of future engineers. 6 These KSA’s were deemed critical by industry and academia. An extended list was provided, but Appendix B below shows the fifteen most valuable KSA’s. These KSA’s are similar, but more detailed than Criterion 3 currently in use by ABET (see Appendix A). Some of the KSA’s are likely to find their way into any new accreditation standards to be developed.

Program sponsors and faculty of both the AerosPACE and ICED programs collaborated to not only identify how ABET criteria and KSA’s are applied in their programs, but more importantly how they can be measured. Many of the ABET criteria are professional skills that are difficult to evaluate, if not architected appropriately, a capstone program will be unable to evaluate them. A group of program leaders thus identified for each ABET criterion (see Table 2 for sample, and Appendix A for full table) and each KSA (see Appendix B) how AerosPACE and ICED align to the regulatory requirements. For this purpose criteria were evaluated at the capstone course level, considering both semesters of each project (AerosPACE and ICED) as one. Table 2 shows just one example how both programs not only provide an opportunity to work in multidisciplinary teams, but also provide robust evaluation thereof through the means of an online interaction platform. Page 26.646.7 Table 2. Sample of ABET criterion application and measurement

ABET Criterion Application in Program Measurement (d) an ability to function on Problem statements are Students are evaluated multidisciplinary teams designed such that individuals through multiple surveys to from a single discipline establish their motivation and would be unable to complete collaboration. Data is it. Students are on teams that supplemented by clickstream are not only multidisciplinary data from the learning (multiple majors) but also management system (LMS) have team members from utilized for team discussions. various universities in Faculty and industry coaches different time zones. assigned to work with teams on a daily/weekly basis and provide guidance and informal evaluation of collaboration.

The full tables in Appendix A and Appendix B show that both AerosPACE and ICED provide an opportunity to have students practice all the skills required by ABET accreditation or outlined as critical KSA’s by industry and academia. Not only are these metrics addressed in these programs, but in fact they are measured and allow student evaluation on both an individual and a team basis. For ABET accreditation (or reaccreditation) this is a very important aspect as it gives additional merit to the application.

The heretofore described correlation of the program to ABET requirements, exists not only at the course level, but can in fact be reduced to individual lectures. Table 3 shows last year’s lecture schedule for the AerosPACE program and how each lecture aligns to the ABET criteria. Alignment was measured using a three point scale (low -1, medium – 5, high – 10), a three point scale rather than the more common five-point Likert scale was used due to the lack of definition in the ABET requirements. It was determined that a five-point scale did not yield consistent results between evaluators. Weightings of 1, 5, and 10 were used simply to more clearly distinguish between the three choices. Evaluators were selected from the program leadership and faculty, results shown indicate consensus between all evaluators; no individual tallies were obtained. The authors are aware that bias may be introduced by utilizing evaluators that are participants in the program, but given the constraints (availability of neutral evaluator with sufficient knowledge of the detailed lectures) it was determined that this was an acceptable risk.

Out of 71 total lecture instances 27 (38%) meet all ABET criteria, 61 (86%) touch on 8 or more of the 11 ABET criteria. Criterion (j) knowledge of contemporary issues is covered least often, but still touched upon in more than half of the lecture instances. Four criteria (e, f, i, k) are touched upon more than 90% of the time, and additional five more than 80% of the time, see Table 4.

Page 26.646.8

Table 3. Lecture Alignment to ABET Criteria (Partial)

Topic (a) an ability tomathematics, apply knowledge science, of and engineering(b) an ability toexperiments, design and interpretasconduct well as data toanalyze (c) an and ability toor designprocess a system,to realisticmeet desiredcomponent, constraints environmental,needs suchwithin ashealth economic, social, and political,safety, sustainability manufacturability,ethical, (d) an ability and tomultidisciplinary function on teams (e) an ability engineeringto identify, formulate,problems and solve(f) an understandingethical responsibilityof professional and (g) an ability to communicate effectively (h) the broad understandeducation necessary thesolutions impact to inof aengineeringenvironmental, global, economic, and(i) a societal recognition contextability of the toengage need for, in life-longand an learning (j) a knowledge of contemporary issues (k) an ability toand use modernthe techniques, engineering skills, toolspractice. necessary for Count Count % Average Boeing Introduction 1 1 10 5 4 36.4% 4.3 Introduction to AerosPACE 1 5 2 18.2% 3.0 Introduction to the RFP 1 5 10 5 4 36.4% 5.3 Teamwork & Collaboration 10 1 5 10 1 1 1 7 63.6% 4.1 Teamwork & Collaboration 10 1 5 10 1 1 1 7 63.6% 4.1 Aircraft Design & Requirements 10 5 10 10 1 1 1 10 8 72.7% 6.0 Project Planning & Management 5 1 5 1 10 5 45.5% 4.4 Systems Engineering & Critical Thinking 10 5 5 10 1 5 5 10 8 72.7% 6.4 Configuration Selection & Vehicle Performance 5 1 10 10 10 1 1 1 5 1 5 11 100.0% 4.5 OpenVSP Demonstration 1 1 5 5 5 10 6 54.5% 4.5 Spreadsheet Based Sizing Tool 10 10 10 10 10 1 1 1 10 9 81.8% 7.0 Background on Sensor Information 10 10 5 5 5 1 1 5 1 5 10 11 100.0% 5.3 Performance Based Sizing 10 5 10 10 1 1 1 10 8 72.7% 6.0 Spreadsheet Based Sizing Tool - Constraint Sizing 10 5 10 10 1 1 1 10 8 72.7% 6.0 Wing and Airfoil Analysis using XFLR5 10 10 5 1 5 1 5 1 10 9 81.8% 5.3 Stability & Control Guidelines 10 5 5 5 5 5 5 1 10 9 81.8% 5.7 Introduction to XFLR5 10 10 5 5 10 1 5 1 10 9 81.8% 6.3 Intorduction to AVL 10 10 5 5 10 1 5 1 10 9 81.8% 6.3 Propulsion Considerations 10 5 10 10 10 5 5 1 1 5 10 90.9% 6.2 Electric Motor and Fan/Prop Analysis 10 10 10 5 10 1 5 1 10 9 81.8% 6.9 Application of XFLR5 and AVL 5 10 10 10 10 5 5 1 1 10 10 90.9% 6.7 Structural Arrangement, Weight & Balance 10 5 10 10 10 5 5 5 1 10 10 90.9% 7.1 Spreadsheet Based Sizing Tool - Weight & Balance 10 5 10 10 1 1 1 10 8 72.7% 6.0 Collaborative Technical Writing and Reporting 10 1 5 10 5 5 5 10 8 72.7% 6.4 Structural Drawings 5 5 1 5 5 10 1 10 8 72.7% 5.3 Prepapration for Conceptual Design Review 5 10 10 10 5 5 45.5% 8.0 Conceptual Design Review 10 10 10 10 10 10 10 10 5 10 10 11 100.0% 9.5 Introduction of Boeing Coaches 1 10 1 5 10 1 10 5 8 72.7% 5.4 Page 26.646.9 Table 4. Summary of Lecture Alignment to ABET Criteria

ABET Criteria %of Average Lectures Alignment (a) an ability to apply knowledge of mathematics, science, and 83.1% 8.9 engineering (b) an ability to design and conduct experiments, as well as to 83.1% 7.9 analyze and interpret data (c) an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, 88.7% 8.0 environmental, social, political, ethical, health and safety, manufacturability, and sustainability (d) an ability to function on multidisciplinary teams 88.7% 8.1 (e) an ability to identify, formulate, and solve engineering 94.4% 8.6 problems (f) an understanding of professional and ethical responsibility 94.4% 4.9 (g) an ability to communicate effectively 87.3% 7.1 (h) the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and 71.8% 4.5 societal context (i) a recognition of the need for, and an ability to engage in life- 93.0% 2.9 long learning (j) a knowledge of contemporary issues 50.7% 3.9 (k) an ability to use the techniques, skills, and modern 93.0% 9.1 engineering tools necessary for engineering practice

Conclusion The skills gap both in the technical (“hard”) and professional (“soft”) skills is a reality of the modern science and engineering workforce, but collaboration of employers (industry or governmental agencies) with universities can lead to successful partnerships to design and develop curriculum that brings together the theoretical foundation with real-life problems and exposes students to the realities of work life with its associated tools and processes. Such a partnership offers unique opportunities to meet and exceed current accreditation standards (ABET) and future goals for Knowledge, Skills, and Abilities (KSA’s). When designed properly and given the proper tools, as outlined here, it allows for the evaluation of these multi-modal requirements for individual students, thus advancing the National Academy of Engineering’s grand challenge to advance personalized learning. In partnership industry, governmental agencies, and academia can work together to create a brighter future for all.

Page 26.646.10 References 1. Stephens, R. & Richey, M., Accelerating STEM Capacity: A Complex Adaptive System Perspective. Journal of Engineering Education 100 (3), 417-423 (2011). 2. Montgomery, D., Retiree flood waits in aerospace wings (Seattle, WA, 2008). 3. Carnevale, A. P., Smith, N. & Strohl, J., 2013. 4. Carnevale, A. P., Smith, N. & Strohl, J., Recovery Job Growth and Education Requirements Through 2020 (2013). 5. Engineering Accreditation Comission, Criteria for Accrediting Engineering Programs (Accreditation Bureau for Engineering and Technology, Baltimore, MA, 2013). 6. American Society for Engineering Education, Transforming Undergraduate Education in Engineering (American Society for Engineering Education, Arlington, VA, 2013). 7. Korn, M., Number of College Students Pursuing Science, Engineering Stagnates. Wall Street Journal (2015). 8. National Science Board, Science & Engineering Indicators (National Science Foundation, Arlington, VA, 2014). 9. Society of Manufacturing Engineers, Workforce Imperative: A Manufacturing Education Strategy (Society of Manufacutring Engineers, Dearborn, MI, 2012). 10. Adachi, B., Gretczko, M. & Pelster, B., Human Capital Trends in Manufacturing Challenges and Opportunities (Deloitte Consulting LLP, 2014). 11. Zender, F., An IPPD Approach Providing a Modular Framework to Closing the Capability Gap and Preparing a 21st Century Workforce (Georgia Institute of Technology, Atlanta, GA, 2014). 12. Morrison, T. et al., 2013. 13. Wepfer, W. & Warrignton, R., Vision 2030Creating the Future of Mechanical Engineering Education (American Society of Mechanical Engineers, Pittsburgh, 2010, 2010). 14. American Society of Civil Engineers, Civil Engineering Body of Knowledge for the 21st Century (American Society of Civil Engineers, Reston, VA, 2008). 15. American Institute for Aeronautics and Astronautics, Building our Competitive Foundation: Supporting K-12 STEM Education (American Institute for Aeronautics and Astronautics, Reston, VA, 2014). 16. Hotaling, N., Burks Fasse, B., Bost, L. F., Hermann, C. D. & Forest, C. R., A Quantitative Analysis of the Effects of a Multidisciplinary Engineering Capstone Design Course. Journal of Engineering Education 101 (4), 630-656 (2012). 17. Stanfill, K., Wiens, G., Eisenstadt, W. & Crisalle, O., Lessons Learned in Integrated Product and Process Design Education, presented at ASEE Southeast Section Conference, , 2002 (unpublished). 18. Hotaling, N., Burks Fasse, B., Bost, L., Hermann, C. & Forest, C., A Quantitative Anlysis of the Effects of a Multidisciplinary Engineering Capstone Design Course. Journal of Engineering Education 101 (4), 630-656 (2012). 19. Gorrell, S. et al., Aerospace Partners for the Advancement of Collaborative Engineering, presented at ASEE Annual Conference & Exposition, Indianapolis, IN, 2014. Page 26.646.11 20. Camarda, C., de Weck, O. & Do, S., Innovative Conceptual Engineering Design (ICED): Creativity, and Innovation in a CDIO Like Curriculum. Proceedings of the 9th International CDIO Conference (2013). 21. Todd, H., Magleby, S. P., Sorensen, C. D., Swan, B. R. & Anthony, D. K., A Survey of Capstone Engineering Courses. Journal of Engineering Education, 165-174 (1995). 22. Camarda, C., Failure is Not an Option. It's a Requirement, presented at 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Palm Springs, CA, 2009. 23. Cannon, L., Zender, F., Stone, B., Innouye, A. & Cunningham, C., Looking back: A Student Review and History of AerosPACE – a Multi-University, Multi-Disciplinary, Distributed, Industry-University Capstone Project , presented at ASEE Annual Forum, Seattle, WA, 2015. 24. Zender, F. et al., Aerospace Partners for the Advancement of Collaborative Engineering (AerosPACE) - Connecting Industry and Academia through a Novel Capstone Course, presented at International Conference for e-Learning in the Workplace, New York, NY, 2014. 25. Zender, F. et al., Wing Design as a Symphony of Geographically Dispersed, Multi- disciplinary, Undergraduate Students, presented at 54th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Boston, MA, 2013 .

Page 26.646.12 Appendix A: ABET Accreditation Criterion 3 Alignment ABET Criterion Application in Program Measurement (a) an ability to apply Students are required to solve Formative assessments during knowledge of mathematics, real life multi-disciplinary lecture and lab, summative science, and engineering engineering challenges. Team pre and post assessments members represent various aligned to learning objectives, educational backgrounds, the and project reporting collaboration resulting in requirements are an integral increased transfer of part of these collaborative knowledge and enhanced projects. understanding across domains. (b) an ability to design and Students are required to build Test plans, experimental conduct experiments, as well multiple prototypes and design, and analysis have to as to analyze and interpret validate design assumptions be documented and are data by means of component and presented to both faculty and systems testing. Final industry advisory board. reporting must include Feedback is provided analysis of captured flight test informally from team coaches data. Students at various as well as formally after points have to present test reviews. A final report is plans, experimental design, required which shows final and results before approval to flight test data as well as prior proceed is granted. Students testing to arrive at the final conduct aerodynamic, design. electrical, propulsion, and flight tests among others as required by vehicle design. (c) an ability to design a Students are engaged in real Key performance indicators system, component, or life design challenge, e.g. (KPI’s) are established for process to meet desired needs designing a vehicle to assist various constraints as part of within realistic constraints first responders. Students the evaluation portion of the such as economic, have to design and build a request for proposal (RFP). environmental, social, systems, while facing Through the advisory board political, ethical, health and budgetary constraints, students are consistently safety, manufacturability, and regulatory challenges, ethical informed of new challenges sustainability concerns (e.g., delivering a discovered by expert in the defibrillator to non- workplace. paramedics), and manufacturing concerns (they have to build what they design). (d) an ability to function on Problem statements are Students are evaluated multidisciplinary teams designed such that individuals through multiple surveys to Page 26.646.13 from a single discipline establish their motivation and would be unable to complete collaboration. Data is ABET Criterion Application in Program Measurement it. Students are on teams that supplemented by clickstream are not only multidisciplinary data from the learning (multiple majors) but also management system (LMS) have team members from utilized for team discussions. various universities in Faculty and industry coaches different time zones. assigned to work with teams on a daily/weekly basis and provide guidance and informal evaluation of collaboration. (e) an ability to identify, Design challenges are created While the course is structured formulate, and solve by engineers based on real around problem-based engineering problems world problems, students are learning and has an responsible for fully defining overarching engineering the problem and providing problem - sub-problems exist possible solutions. for the various systems and interfaces. Students are given formative feedback during weekly review sessions and summative feedback following design reviews. (f) an understanding of Students operate as a team Students submit weekly professional and ethical with professional timecards and project responsibility expectations regarding schedules. Teams must report planning and execution of on team organization and work statements. Students collaboration as part of interact with customers (e.g., formal reports. Student first responders) to derive surveys evaluate design requirements. responsibilities taken on by each student. (g) an ability to communicate Students collaborate with Clickstream data from the effectively team members and faculty LMS enables mining of from multiple schools as well communication and allows as industry coaches. Effective for analysis by means of communication is paramount network graphs of student to success and thus communication. Contextual emphasized in various recognition is utilized to lectures. evaluate the topics of communication. Students are separately surveyed particularly on communication with their team members. Page 26.646.14 (h) the broad education Students experience the Student teams have to define necessary to understand the impact of engineering directly their own mission and report ABET Criterion Application in Program Measurement impact of engineering as they are participating in to faculty and advisory board solutions in a global, this project. They are what they are designing their economic, environmental, immersed in the context as vehicle for. This mission and societal context they are working with definition directly highlights customers and suppliers. the team’s view of the impact of engineering. (i) a recognition of the need Students are provided with Clickstream data captured as for, and an ability to engage formal learning opportunities, part of the LMS allows the in life-long learning but are also provided with evaluation of resources by additional resources (e.g., each individual student. In EdX course on composites, or combination with survey reference documents). Life- responses, this data can long learning is exemplified inform a model on the level by faculty and coaches who of effort to engage in learning everyday learn something outside the required by the new from the students or their students, which likely is an colleagues and make a indicator for life-long particular effort to highlight learning. such learning to the students. (j) a knowledge of Students monitor publications Unobtrusive mining of the contemporary issues on issues related to their LMS allows for the analysis project and share with their of contemporary issues team and course members as shared amongst students. appropriate. Additionally Similarly feedback brought faculty and advisory board by faculty or advisory board members provide feedback. members is captured and student responses are required. (k) an ability to use the Students use a variety of Student skills are assessed via techniques, skills, and engineering processes and pre/post assessments for some modern engineering tools tools. Students are instructed tools, or via informal necessary for engineering by faculty and industry assessments as part of labs. practice representatives on the tools Outputs of the engineering and processes most applicable practices are reviewed during to their project. the design reviews.

Page 26.646.15 Appendix B: ASEE KSA Alignment KSA Application in Program Measurement Good communication skills See g) above See g) above Physical sciences and See a) above See a) above engineering science fundamentals Ability to identify, formulate, See e) above See e) above and solve engineering problems Systems integration Students have to design a Students self-assign into system that consists of Integrated Product Teams various sub-systems that need (IPT’s) and have to manage to be integrated (e.g. their interfaces and electrical, propulsion) but in integration. Unobtrusive itself is part of systems of mining allows an evaluation systems (e.g. national air of how interfaces were space) managed during the design process while design reviews allow for an analysis of the success of the integration which is ultimately visible in a flying vehicle. Curiosity and persistent Students are presented with Curiosity of the students is desire for continuous learning an open-ended design evaluated by means of challenge, where they have to student surveys, their desire be curious to properly define for continuous learning can their mission alongside their be observed via the customers. Solving the clickstream data which makes various challenges associated visible the informal resources with the design of a vehicle accessed by the students. calls for various formal and informal learning activities, not all part of the formal curriculum. Self-drive and motivation Students operate as self- Student surveys evaluate each contained teams with team individual student’s members required to motivation throughout the complete tasks assigned to project. In addition an them on a tight schedule, integrated computing motivation is paramount to environment combined with a success. self-reported hours log enables a more thorough analysis of the motivation and

its translation into work. Page 26.646.16 Cultural awareness in the Teams are distributed, student Faculty and industry coaches broad sense (nationality, diversity is vast within and observe and guide team KSA Application in Program Measurement ethnicity, linguistic, gender, across the various interactions where necessary sexual orientation) universities. Successful teams to further project goals. take advantage of their Longitudinal student surveys unique characteristics to observe attitudes and changes further the team’s success. therein. Economics and business Teams define their own Budgets are checked at every acumen mission and thus their own design review. The business business strategy. Teams are case is evaluated as part of provided a budget and must initial mission definition and manage logistics, purchases, continuously monitored. and travel. High ethical standards, Students are held to the Design reviews evaluate the integrity, and global, social, highest ethical standards and global, social, intellectual, intellectual, and technological are advised by faculty and and technological impact. As responsibility advisory board members on required, feedback is concerns regarding their provided to students and their project and integration into a progress towards is tracked. larger system of systems. Critical thinking Students have to solve an Critical thinking is evaluated open ended engineering by analyzing the checklists challenge that require them to students create for design, critically think about their tests, and flights. Critical customer and a design to thinking is required to meet the needs. properly create these. Willingness to take calculated Students have to take a Each design reviews includes risk variety of calculated risk to an analysis of the systems design their airplane. In engineering processes utilized addition students are given and risks are established and instructions by industry on (re)evaluated at each stage. risk, issues, and opportunities (RIO). Ability to prioritize Students receive credit for Prioritization of tasks is efficiently this course and are typically evaluated on a weekly basis involved in other classes as by faculty and industry well. Students have to coaches. Team meetings with prioritize the work within individual report outs provide their team and for themselves an opportunity to work with in order to succeed. each student. Mentoring is available to all students. Project management Students have to manage the Students submit updated (supervising, planning, team budget, including all budgets at each design

scheduling, budgeting, etc.) raw material and testing costs review, faculty may review Page 26.646.17 as well as required orders. them as needed. Faculty Students are developing their approves all purchases. KSA Application in Program Measurement own project plan, lectures on Students submit updated project management are given project plans weekly for by industry. review. Teamwork skills and ability See d) above to function on multidisciplinary teams Entrepreneurship and Students are given The mission definition review intrapreneurship opportunities to work directly includes an analysis of the with customers and need to business case. The design is develop a business plan to continuously monitored justify their design choice. against the original proposal.

Page 26.646.18