AC 2008-2966: A FRAMEWORK FOR SUBSTANTIAL ABET ACCREDITATION OF AN INTERNATIONAL CIVIL PROGRAM

Suleiman Ashur, Indiana University - Purdue University Fort Wayne Khaled El-Sawy, University Essam Zaneldin, Indiana University Purdue University Fort Wayne Page 13.41.1

© American Society for Engineering , 2008 A Framework for Substantial ABET Accreditation of an International Civil Engineering Program

Abstract

The United Arab Emirates University (UAEU) is the national university of the United Arab Emirates. The UAEU is the largest public university in the country, serving about 14,000 students. The College of Engineering offers B.S. degrees in civil, computer, electrical, , chemical, and petroleum engineering.

The paper presents the framework developed for preparing the Civil Engineering program for substantial ABET accreditation. The paper discusses all direct and indirect program outcomes and a summary of the process to achieve these outcomes. In addition, the paper focuses on the major steps toward developing and achieving the curriculum assessment of the program. The paper will present the major step in integrating the program assessment tools by a comprehensive and sophisticated spreadsheet that lists all the curricula and course outcomes of each course in the program. It automatically imports the individual course outcome into the spreadsheet and provides outcome indices on whether program objectives and outcomes achieved or not. Discussion on challenges for developing and achieving the program outcomes will present. Recommendations and lessons learned from this process to help other Civil Engineering programs in general and international program in particular in achieving accreditation will be presented.

Page 13.41.2 Introduction

The United Arab Emirates University (UAEU) like other international universities in developing countries is striving to improve the quality of higher education by several means. One approach is being used successfully is international accreditation of their academic institutions. In , for example, the universities purse accreditation by either the English institutional accreditation system, FEANI or the American Board for Engineering and Technology (ABET)1. The choice in the UAE was to pursue ABET accreditation as a vehicle to meet its standards and improve the engineering higher education in the country.

However, the new ABET 2000 criteria for accreditation made the process more demanding by shifting the focus of accreditation from “teaching” to “learning.” 2 In addition, achieving an effective outcome assessment plan and meeting ABET’s Engineering Criteria 2000 requires flexibility from the faculty member to learn and apply the new process and adjust to continues nature of the new criteria 3. One factor of concern to faculty members is that working load at international universities is cumbersome. The faculty members are required to teach a minimum of 12 credit hours, conduct scholarly research and do services. Research and good teaching are two critical elements that influence faculty annual evaluation and promotion. The work requirement without any major incentives for faculty members makes it very hard to actively participate in the assessment process. Therefore, it is very important to make the process as simple and systematic as possible to encourage faculty to actively participating in the assessment plan. In addition, the process should be efficient and effective for assessing course and program outcomes.

The UAEU is the national university of the United Arab Emirates. The UAEU is the largest public university in the country, serving about 14,000 students. The College of Engineering offers B.S. degrees in six engineering programs (civil, computer, electrical, mechanical engineering, chemical, and petroleum) with ABET substantial equivalency recognition 4. The Civil Engineering programs as well as other programs were scheduled for ABET review in 2004.

At the beginning, the assessment process is centralized and developed by the college. Therefore, each department shall adapt the process. However, faculty members found the process to be overly complicated and time-consuming. It was decided to change the process to be effective and not to increase the workload of the faculty 2 and conduct the process at the department level while reporting the process to the college committee for quality control purposes.

Framework for the Program Assessment

The department educational outcomes were adopted to be the same as ABET outcomes. Table 1 maps the relationship between the program outcomes and program educational objectives for the department of Civil and Environmental Engineering at the UAEU. The Department of Civil and Environmental Engineering has established a well defined process for outcomes assessment in order to ensure that its graduates achieve the program educational objectives. Figure 1 presents the framework of the process established for the program outcomes assessment.

Page 13.41.3 Selecting the tools for assessment is very critical. Some argue for the use of direct tools like exams and homework; others like the indirect tools of conducting surveys. However a balance of effective direct and indirect tools should be used. A recent study to evaluate assessment tools for computer science found that exit surveys, external advisory panels and alumni surveys are used the most in this area 5. Probably programs in computer science prefer this method because of the time and effort needed by other assessment tools.

The process used in the civil engineering program at UAEU is to adopt eight assessment tools to assess the program outcomes. These eight assessment tools are divided into direct and indirect tools. The direct tools include Curriculum Assessment, Exit Exam, and Capstone Course. The indirect tools include Internship Advisor Survey, Industrial Advisory Board Survey, Students Exit Interview, Alumni Survey, and Employer Survey. The objective of this paper is to present in details the framework developed for the curriculum assessment.

Figure 1: CE Program Outcome Assessment Framework

Page 13.41.4 Table 1: Relationship between Program Outcomes and Program Educational Objectives

CE Program CE Program Educational Objectives Outcomes* 1. Graduate students with knowledge of engineering principles and theories necessary for application in civil engineering projects. A, J

2. Develop students’ capabilities towards innovation and creativity in C, E engineering design. 3. Develop students’ computer skills to a highly competent level. K 4. Enhance students’ ability to communicate effectively. G 5. Enable students to conduct experimental work effectively. B 6. Enable students to improve their team-working skills, and to achieve life-long learning habits. D, I 7. Help students to develop a positive attitude towards ethical, social, and environmental issues relevant to the engineering profession. F, H

* ABET 6 outcomes are: Engineering programs must demonstrate that their students attain the following outcomes: (a) an ability to apply knowledge of mathematics, science, and engineering (b) an ability to design and conduct experiments, as well as to analyze and interpret data (c) an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability (d) an ability to function on multidisciplinary teams (e) an ability to identify, formulate, and solve engineering problems (f) an understanding of professional and ethical responsibility (g) an ability to communicate effectively (h) the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context (i) a recognition of the need for, and an ability to engage in life-long learning (j) a knowledge of contemporary issues (k) an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice. Curriculum Assessment The curriculum assessment process cumulates the individual contribution from all courses in the CE program to the program outcomes in order to assess the contribution of the entire curriculum to the program outcomes. The flowchart of this process is outlined in Figure 2. By mapping the program outcomes to the program objectives, as shown in Table 1, recommendations could be made at the end of the process to improve the program outcomes as well as the program educational objectives. The process then starts all over again, and continues on. The following Page 13.41.5 sections describe the assessment process in more details and present illustrative examples.

Figure 2: CE Curriculum Assessment Framework

Course Assessment Forms The process starts by developing the Course Syllabus (Form 1). The course syllabus contains the course outcomes mapped to the program outcomes. The measure for assessment is the Course Assessment Form (CAF) (Form 2-a) where the course objectives and outcomes are listed and mapped to the program outcomes, as shown in Figure 3. The form lists three performance measures: students (S), Faculty (F), and Quantitative (Q). A) Measuring Course Outcomes The course outcomes are measured using three indices: Student Assessment, Faculty

Assessment, and Quantitative Assessment. Page 13.41.6 Student Assessment of the Course: By the end of each semester, students evaluate the course using the Students Assessment of the Course Survey and the results are documented in a folder available in the Department. The survey lists the course outcomes as presented in the course syllabus. Students are asked to evaluate each course outcome based on a 1-5 scale, where 1 is very low and 5 is very high. The evaluation is based on student's feeling of how the class has helped him or her to achieve the abilities, attributes, and skills as described in the outcomes over the course of the semester. The results of the survey are collected and analyzed. The averages for all students are then entered on the CAF. In order to ensure impartiality and for fair evaluation by students, the survey is conducted at the end of the semester by a Lab without the presence of the faculty member teaching the course. The process could use as well another faculty member who is not teaching the course. Students are not required to write their names on the survey form, and are informed that the faculty will not see these surveys until the final grades of the course are turned in. Faculty Assessment of the Course: Each faculty evaluates each of the course intended outcomes on a scale similar to that used for the Student Assessment of the Course Form. The evaluation corresponds to the extent the faculty feels the class has helped the students to achieve through the course of the semester. The result of the survey is added to the CAF by the end of the semester. The faculty is requested to complete the survey based on his feeling and observations, without considering the performance of the students in exams and homework. Quantitative Assessment of the Course: This index is based on the actual performance of students in class activities and exams. The faculty chooses a course activity or activities (e.g., homework, exam problem, quiz, etc) to evaluate each course outcome. The selected activity or activities should be highly correlated to the course outcome. The index is calculated using the average grade, out of five, for all students in the class for the selected activities. Notice that the value 1 for the scaled average, or performance level 1, indicates that the outcome is poorly met, while the value 5 indicates that the outcome is exceptionally-well met. Also, if the performance level for a course outcome is below 3, the instructor should provide reasons for this shortcoming and recommend a specific action to be taken to remedy the problem. It is worth mentioning that the quality assurance and improvement of the course is based on grades of students, student participation in the class, students’ survey for achievements of the course outcome, exist exam and the comments and evaluation by the faculty focus group. At this point, the Students, Faculty, and Quantitative performance levels, or values, have been measured for all course outcomes. These values are graphed for comparison, evaluation, and to draw recommendations for further improvement of the course. Part of the faculty and focus group recommendations for course improvement (Form 2-b) is shown in Figure 4. In order to simplify the process on faculty members, a spreadsheet was developed to do all calculations. The faculty members need only to enter the course objectives and assessment tools and the spreadsheet will report the results. At the beginning, the process will take an average of 2-3 hours per course. After the first time, the course will take an average of one hour. Figure 4 shows the results of the assessment using the spreadsheet program. Page 13.41.7 Quality Control and Follow up on the Course Assessment: In order to assure quality control and consistency of the assessment process, the Department has established 4 focus groups to cover all specialties in the program. These focus groups are: 1) Structures and Materials, 2) Environmental and Water Resources, 3) Transportation and Surveying, and 4) Geotechnical and Construction Management. Each group meets at the end of each semester to discuss the results of course assessment and provides the final recommendations to be implemented next time the course will be offered. The discussion is focused on the recommendations the faculty has implemented from the previous assessment cycle, the reason(s) for not implementing a specific recommendation, if any; and his recommendations for next semester. The focus group then makes the final recommendations that should be implemented during the next semester the course will be offered. The same process is applied to all courses in the Department. The full documentation of the course assessment forms, results and changes to improve the course performance is included in the course file of each course. The course files are available and accessible by all faculty members in the Department for review and use.

B) Evaluating Program Outcomes from Course Outcomes The CAF establishes the relationship between the course outcomes and the program outcomes. First, the course outcomes are mapped to program outcomes and then the results are aggregated to measure the contribution of the course to the program outcome. The link is established through the Relevance Level which measure, according the faculty and the focus group, how much this course is relevant to the program outcomes on a scale 1-5. These values are summarized in the Curriculum Analysis Matrix (CAM), as will be explained in the following section.

Curriculum Analysis Matrix (CAM). The curriculum evaluation is measured through the courses listed in the degree plan. The courses are classified into three types: University, College, and Major. The assessment includes most of the program core courses and the elective courses taught in the past three semesters. Some courses which are taught by other departments were not included because the data were not available to us. The CAM captures several input data including the course title and number, the instructor name and title, the semester and the academic year, the credit hours of the course, course relevance level to the program outcomes, and the average values of the course actual performance from student, faculty, and quantitative assessment of the course. Figure 5 shows part of the input sheet of CAM. It should be noted that the following summary of the analysis is based on 65 credit-hour courses (56 credit hours of core courses and 9 credit hours of elective courses) taught in the consecutive three semesters. The results of the assessment are shown in Figure 5.

Page 13.41.8 Figure 3. Sample of Course Assessment Form (Form 2-a)

Page 13.41.9

Figure 4. Part of CAF Spreadsheet Reporting Results and Faculty and Focus Group Recommendations for Course Improvement (Form 2-b) (Closing the Cycle)

Macros were developed to automate the process of importing data from any CAF in order to reduce the labor work of copying and pasting information from each CAF course to CAM. Faculty members were instructed to name their CAF course with a specific format that contains the course number and semester according to specific format. Once the file is received, the macro in the CAM is activated the all required information from the file is imported to the CAM. As shown in Figure 6, students’ impression that the part of the curriculum taught last semester has helped them achieving the program outcomes (S-score) is relatively high. Students' perception was higher than the faculty's perception and the quantitative analysis results of the students' actual performance in the class. In addition, students' actual performances (Q-score) were higher than what faculty members felt students had achieved. In addition, the actual performance of students in all program outcomes has achieved or superseded the target value of

3.5. Page 13.41.10 Figure 5 Part of the Curriculum Analysis Matrix (Form 3)

Active / Course Credit Unit Course Title Instructor Semester/Year Course Relevance Level from Individual CAF [Sacle 1-5] Inactive Number Hours ABCDEFGHIJK

Active and the Environment CIVL 205 Munjed Maraqa, Associate Professor 1st 2002/2003 2 5 0 3 0 3 1 1 0 0 1 1

Active Statics CIVL 215 Aly Nazmy, Associate Professor 1st 2002/2003 3 5 0 0 0 3 0 1 0 0 0 1

Active Dynamics CIVL 210 Khaled El-Sawy, Assistant professor 2nd 2002/2003 3 5 0 0 0 5 0 1 0 0 0 0

Active Mechanics of Materials I GENG 305 1st 2003/2004 3 3.9 4 4 3 4 0 3 0 4 0 0

Active Structural Analysis I CIVL 3103 Bilal El-Ariss, Assistant Professor 1st 2003/2004 3 5 0 0 0 5 0 1 0 0 0 5

Active Computer Aided Drawing MECH 315 3 3 0 2.52.5 0 0 2.5 0 0 0 0

Inactive Concrete Technology CIVL 3162

Active Structural Analysis II CIVL 318 Khaled El-Sawy, Assistant professor 1st 2002/2003 3 5 0 2 0 5 0 1 0 0 0 1

Active Environmental Engineering I CIVL 320 Walid Elshorbagy, Associate Professor 1st 2002/2003 3 4 4 3 2 3 0 2 0 0 1 2

Inactive Water Resources CIVL 325 0

Active Surveying CIVL 335 Ahmed El-Mowafy, Assistant Professor 2nd 2002/2003 3 4 3 3 3 4 1 1 0 0 0 3

Active Fluid Mechanics I GENG 340 Mohsen Sherif, Professor 1st 2003/2004 3 5 3 2 2 0 0 2 0 0 0 0

Active Engineering Materials MECH 390 1st 2003/2004 3 4 4 0 0 0 0 3 0 0 3 4

Program Compulsory Courses Active Structural Design I CIVL 411 Bilal El-Ariss, Assistant Professor 2nd 2002/2003 3 5 0 5 0 0 0 1 0 0 0 5

Active Highway Engineering CIVL 433 Yaser Hawas, Assistant Professor 1st 2002/2003 3 3 1 4 0 0 1 1 1 0 0 3

Active Construction Management CIVL 445 Essam Zaneldin, Assistant Professor 1st 2002/2003 3 4 0 0 0 3 3 3 0 0 0 4

Active Soil Mechanics CIVL 449 Abdel-Mohsen Mohamed, Associate Professor1st 2002/2003 3 4 5 0 0 4 0 2 4 0 0 4

Active Structural Design II CIVL 513 Ashraf Biddah, Assistant Professor 1st 2002/2003 3 4 3 5 1 4 0 2 0 0 0 4

Active Structural Design III CIVL 514 Aly Nazmy, Associate Professor 2nd 2002/2003 3 5 0 4 0 5 0 1 0 0 0 2

Active Graduation Project I CIVL 585 Dr. Yaser Hawas 1st 2003/2004 3 4 0 3.75 3.75 4 3.75 4 3.5 3.5 3.5 3.5

Active Graduation Project II CIVL 590 Dr. Suleiman Ashur 1st 2003/2004 3 4 4 3.75 3.75 3.75 4 4.25 3.5 3.5 3.5 3.75

Active Special Topics in Structural Engineering CIVL 510 Ashraf Biddah, Assistant Professor 1st 2002/2003 3 5 0 4 0 5 0 3 0 0 0 4

Active Special Topics in Transportation Eng. CIVL 530 Suleiman Ashur, Assistant Professor 1st 2002/2003 3 4 0 5 0 4 5 0 0 0 5 3 Electives Program Active Advanced Construction Management CIVL 547 Essam Zaneldin, Assistant Professor 1st 2002/2003 3 4 0 0 0 3 3 3 0 0 0 3

Figure 6 Curriculum Assessment Analysis (Form 3)

Curriculum Assessment

5.0

4.0 S 3.0 F

2.0 Q Target 1.0 Weighted Average

0.0 ABCDEFGHIJK Program Outcomes

Page 13.41.11 Summary and Recommendations The process of the curriculum assessment is complicated by nature and requires involvement of all faculty members in the program and faculty members from other departments such as math and physics. The assessment committee shall make the process simple and efficient in order to present a solid case for accreditation. The current developed process was successful because it is based on detailed information from students’ performance in the course. The developed CAF spreadsheet made the process very easy for the faculty to use without spending too much time in the process. In addition, the developed CAM made the process very fast and effective without wasting time on manually moving data between sheets.

It is recommended pursuing other options that can automate the process and make assessment more powerful at the program, department, college, and university level. The current system was successful but has its limitation. For example, the program requires continuous maintains and it is limited to assess at the program level. It cannot carry the assessment to a department or college level. Therefore, one option should be consider is adapting an assessment management systems such as TrueOutcomes (www.trueoutcomes.com ), eLumen Achievement (www.elumen.info), or Blackboard Outcomes System (www.blackboard.com ). However, the financial burden and faculty training becomes an issue in this case and something to consider before adapting such system.

Acknowledgment The authors would like to thank all faculty members and all in the department of civil engineering for their contribution and input in the development of the assessment tools and collection of the data used in this paper. In particular, the authors would like to acknowledge the effort of the faculty members who served in the Assessment and ABET reaccreditation committees. Finally, thanks are due to the College of Engineering Assessment Committee and the University Administration for their support during the development of the original manuscript.

Bibliography

1. Ibrahim Akduman, Lerzan Özkale, and Ekrem Ekinci. “Accreditation in Turkish universities.” European Journal of , Vol. 26, No. 3. 2001, pp. 231–239. 2. Theis, T.T., “Trends in Engineering: Education and Practice,” Civil Engineering, ASCE, Vol. 66, no. 11, 1996, p. 6. 3. Jack D. Bakos Jr. “Outcomes Assessment: Sharing Responsibilities.” Journal of Professional Issues in Engineering Education and Practice, Vol. 125, No. 3, 1999, pp. 108-111. 4. http://www.engg.uaeu.ac.ae/ . Accessed in February 2008. 5. Kathryn E. Sanders and Robert McCartney, “Collected Wisdom: Assessment Tools for Computer Science Programs.” Computer Science Education, Vol. 14, No. 3, 2004, pp. 183–203. 6. Criteria For Accrediting Engineering Programs, Effective for Evaluations During the 2008-2009 Accreditation Cycle, ABET Board of Directors, November 3, 2007. ABET, Inc, , MD. Page 13.41.12