Paper ID #30515

Implementation and Evaluation of Active Learning Techniques: Adaptable Activities for A Variety of Engineering Courses

Dr. Jillian Schmidt, Missouri University of Science and Technology

Dr. Jillian Schmidt is an Assistant Teaching Professor in the Department of Mechanical and Aerospace Engineering at Missouri University of Science and Technology. She teaches primarily first and second- year engineering design courses, and her research interests include technology incorporation and team dynamics in project based courses. Dr. Nicolas Ali Libre, Missouri University of Science and Technology

Nicolas Ali Libre, PhD, is an assistant teaching professor of Civil Engineering at Missouri University of Science and Technology. He received his BS (2001), MS (2003) and PhD (2009) in civil engineer- ing with emphasis in structural engineering, from University of Tehran, Iran. His research interests and experiences are in the field of computational mechanics, cement-based composite materials as well as in- novative teaching techniques. Dr. Libre is the manager of Materials Testing lab at Missouri S&T, teaches mechanics of materials and develops digital educational resources for the engineering students. He had the opportunity of leading several scientific and industrial research projects and mentoring graduate and undergraduate students. Over the span of his career, Dr. Libre authored and co-authored 3 chapter books, 17 peer-reviewed journal articles and over 60 conference papers. He has advised and co-advised 8 gradu- ate students and mentored over 30 undergraduate students. He has collaborated with scholars from several countries, including Iran, China, Slovenia, Canada, and the US. He also served as a reviewer for 6 journals and a committee member of 5 conferences. He is the recipient of the University of Missouri President Award for Innovative Teaching (2018), the Teaching with Technology Award in the Focus on Teaching and technology conference (2018), Joseph H Senne Jr. Academy of Civil Engineering Faculty Teaching and Service Achievement Award (2017) and the Excellence in Teaching Award from the National Society of Leadership and Success (2016).

c American Society for Engineering Education, 2020 Implementation and Evaluation of Active Learning Techniques: Adaptable Activities for A Variety of Engineering Courses

Abstract Active learning is a form of learning approach in which students are actively involved in the learning process through specially designed activities, often in groups, followed by reflection upon what they have done. The effectiveness of active learning techniques that are intended to improve students’ learning outcomes is highly dependent on the extent to which students are participating in the activity and also on the feedback they receive throughout or at the end of activity. While many studies have shown the positive impact of such approaches on students’ performance, many instructors still hesitate to implement active learning in the classroom. Typical concerns and obstacles include the difficulty of designing an effective activity, the required time for conducting the activity, the tools needed for facilitating the activity, and the willingness of students to engage in the activity.

This paper is aimed at providing a library of active learning strategies that could be used in teaching various engineering courses, discussing strategies to ensure students are effectively engaged in the activity, and evaluating the impact of those approaches on students’ performance. The implemented techniques include pop-up quizzes followed with a group discussion and small group challenge problems with timed release of hints throughout the activity; also algorithmically coded questions with randomly generated numbers were used in which students needed to collaborate in formulating the solution but the final answer would be unique for every member. An online learning platform was used that enables the instructors to measure the performance of students individually and as a part of a team. The studied teaching techniques have been implemented in various engineering classes at different levels (sophomore, junior) and different class sizes (40 to 110 students in each class). The efficacy of active learning on students’ performance was evaluated by comparing the grades on summative assessments with and without applying the active strategies. In addition, surveys were conducted to understand the students' perception of active learning and to identify the learning strategies they considered the most effective. Evaluation of student responses indicates broad agreement on the effectiveness of high-impact learning activities.

Key words: Intro to Engineering, Problem based learning, summer camp, High school students

1- Introduction Active learning is a form of teaching and learning method that is developed to improve student learning outcomes, and typically consists of techniques requiring students to be actively engaged in their learning process through specially designed activities, followed by reflection upon what they have done. Bonwell and Eison [1] stated “that in active learning, students participate in the process and students participate when they are doing something besides passively listening." The goal of active learning activities is to actively involve students in the teaching and learning process in order to increase student engagement, performance, and retention [2].

Despite the considerable published research in the literature [3-6] showing the advantage of active learning approaches in STEM and engineering education and its impact on increasing students performance, many instructors still do not implement active learning in their teaching curriculum. The time required to design, implement and revise an active learning curriculum can be restrictive, especially for junior faculty members who are typically struggling with the pre-tenure teaching, research and service expectations. One the other hand, some senior instructors who already shaped their teaching philosophy hesitate to take the risk of redesigning their course curriculum and adopt new teaching and learning strategies due to perceived effort required to implement them and the uncertainties on the outcomes. The advantage of creating a library of active learning activities is to help new instructors to become more effective and efficient in teaching and also to provide senior instructors with some tested activities that they can adopt based on their desires and needs. There have been several efforts in developing a library of active learning activities in various disciplines [7-11] . Bohnhoff and Sample-Lord created a library of group activities to promote active learning in the undergraduate soil mechanics classroom [7]. They reported that active format and activities with real world context increased undergraduate student interest in geotechnical engineering, and that similar strategies could be utilized to develop shareable activity libraries for other engineering disciplines.

This research is aimed at developing various active learning activities in engineering mechanics that could be easily adopted by other instructors and evaluate their impact on students’ learning. The developed active learning techniques were implemented in a fundamental core engineering course in sophomore and junior level. The raw exam scores from semesters before the activities were implemented were compared with student performances after implementation of the new active learning format to evaluate whether or not actively engaging the students improved their understanding of the material. After successful implementation of active learning in the mechanics of materials, the same activities were conducted in Aerodynamics 1 to begin to investigate how these activities can be tailored to other subjects and used to improve student performance in a wider variety of engineering courses.

2- Development and Application of Active learning strategies The primary purpose of active learning strategies must be to enhance student learning and a proper active learning activity should be designed in a way to be challenging, interesting, and relevant to course goals and objectives. The studied activity involved conducting practice problems during the class session using a web based polling platform. At the early stage, Kahoot was used for collecting student responses. In the third semester, a dedicated web based platform was developed to assign questions to students, monitor their progress, collect their responses and provide instantaneous feedback. An innovative feature of this study is the developed platform [12] that is developed for administering the activities and collecting data and the ability to easily share the activities with other instructors.

During the class session, the instructor spent a short amount of time to expand upon the student learning objectives addressed in the pre-class assigned videos and in demonstration of examples so that students more deeply understood how to apply the concepts. The rest of class was then structured toward active learning, a teaching approach that encompases anything students might be called on to do in class besides watching and listening to an instructor and taking notes [13]. One of the commonly cited challenges with implementing active learning activities is the loss of class time for covering new material. These pre-class assigned videos introduced new material to the students outside of class, so they were prepared to begin applying the concepts during the in-class activities. These videos were typically 5-15 minutes long and posted before every class meeting. On average, two thirds to half of the class time were spent for lecturing, working examples, reviewing the theoretical and practical aspects; In the remaining time, students collaboratively solved examples and problems and participated in the other form of activities. Students were strongly encouraged to work together in teams and to discuss the material while the instructor circulated to give guided practice.

To promote problem solving skills and higher level thinking, several practice problems were developed by the authors some of which were short and conceptual and focused on certain misconceptions observed in the past and some others were calculation intensive and required several steps to achieve the answers. Most of the calculation intensive questions have interim steps to guide students through the process and allow them to check the calculations in each step while they are approaching the final answer. The algorithmically generated practice problems provided students with the same questions in each activity but each question had a unique set of randomly generated parameters, resulting in different final answers that prevented students from sharing the final answer but encouraged them to share the ideas on how to solve the problem. The instructor also facilitated student learning by posting a scenario/hints on the board and by circulating around the room and providing feedback. A proper interaction and discussion was observed during this collaborative learning time. The students discussed, taught, and explained to one another how the problem was to be solved. It was observed that the small group activities give students the opportunity to interact with their classmates, learn from others problem-solving techniques, and work in teams in non-stressful situations.

The activities were designed to cover a wide variety of topics with various levels of difficulty. In total, more than 700 algorithmically generated practice problems were developed. The time needed for students to work on each practice problem varied from 1-min for conceptual questions to up to 30-min for complex calculation intensive problems. During the four semesters of testing this active learning technique, the questions have been used several times in various sections and the average time spent on each question and students’ average score were measured and recorded. Such data reflects the difficulty level of the questions and allows instructors to combine various problems in order to have a balanced mixture of activities that are rewarding yet challenging for students. The dedicated web-based platform not only provided students with instantaneous feedback but also allowed instructors to monitor progress of students while the activity was running. A sample dashboard with real time activity progress data is shown in Figure 1.

Figure 1.Sample of students progress data analytics collected during active learning activities

Active learning activities are a part of the learning process not assessing student knowledge; however, considering limited points specially as a bonus would encourage students participation and engagement. In our model, the active learning activities were administered as low-stakes formative assessments. Such assessments can be used by instructors during the learning process in order to reinforce student learning and improve achievement of student learning objectives. Formative assessment typically involves gathering feedback that can be used for improvement in the ongoing teaching and learning context in contrast to summative assessments, which seek to determine the measure of a student's learning. As such, formative assessments usually have lower grades compared to summative assessments that form the majority of the course grade. Based on the author's experience, using graded formative assessments consisting of relatively low point values, often even as bonus points, motivates students to get involved in the class activities.

3- Active learning Implementation, results and discussion Various active learning strategies described in the previous section have been incorporated as a part of in-class activities in Mechanics of Materials. Mechanics of Materials is a fundamental introductory course for many engineering disciplines such as civil, mechanical, aerospace, architectural, and metallurgical engineering. This course is also a part of programs such as environmental, manufacturing, nuclear, engineering management, and petroleum engineering. Mechanics of Materials at Missouri S&T is offered in small sections with 50 students and large sections with more than 100 students in each section. The same course is offered to all engineering disciplines; there are no major or non-major versions of the course. In total, 300-400 students take this course every semester. The active learning strategies were implemented in some sections; the course was offered in a traditional format in the other sections. The number of students who were in the sections with active learning strategies are shown in Table 1.

The active learning techniques were incorporated to the course gradually starting with five activities in the pilot semester mostly offered in the last quarter of the sixteen week course. Following the successful pilot of the technique, more active learning activities were conducted in subsequent semesters. Table 1 shows the number of active learning activities offered to students in each semester during the research period. Every semester, 33 to 45 in class activities were performed throughout the semester; averaging 0.9 to 1.25 activities per class session in various semesters. In the third semester, the course was redesigned to a blended format, placing a greater emphasis on the problem-solving active learning strategies. In the blended format, short video lectures are watched by students followed by a couple of questions that students should try to answer before the class session. The pre-class concepts and questions are discussed more in depth during the class and more examples are presented. Students have the chance to retake the pre-class questions after the class discussion without penalty if they have tried it at least once before the class session. In this model, students are first exposed to the course content before the class, while in-class time focuses on more difficult skills which require greater practice, student-to-student collaboration, and discussions. In this pedagogical model, the in-class practice problems are implemented as a main component of the course curriculum to facilitate active learning during the class time.

As mentioned earlier, the active learning strategies used in this research was implemented as formative assessment consisting of relatively low values, often even as non-mandatory bonus points to motivate students to get involved in the class activities. In our research, the active learning activities contributed 5% to 12% of the total course grade in various semesters. In most semesters, the active learning activities were considered just as bonus point, varying from 5% to 7% of the total course grade; The only exception is the third semester where part of the in-class activities were mandatory (5% of total grade) and the rest were optional for bonus (up to 7%). Details are provided in Table 1. The graded components of the course include homework, quizzes, four midterm exams and the final, cumulative exam, which all accumulate to 1000 points in total. Various opportunities were offered to students to collect bonus points by participating in the active learning activities during class sessions, supplementary optional homework, and other optional practice work. The bonus points were capped at 70 points in all semesters, no matter what type of extra work a student may have chosen to attempt.

Table 1- Enrollment and course data during the research period Semester Number of Active learning points Course Mandatory Number of # activities delivery class students Mandatory Bonus method attendance

1 33 0% 5% Traditional Yes 140 lecture

2 36 0% 5% Traditional Yes 135 lecture

3 45 5% 7% Semi-blended Yes 353

4 35 0% 7% blended No 205

In addition to implementing these active learning strategies in the Mechanics of Materials course, during the fourth semester of this study, several of these same active learning techniques were implemented for the first time in Aerodynamics 1, a required junior level aerospace engineering course. The majority of the students enrolled in this course were either concurrently enrolled in or had previously taken Mechanics of Materials, so the activity format was familiar and did not require much introduction. In this course, the activities were optional and low-stakes, earning only about 1% bonus on the overall course grade. The topics for the in-class problem solving activities were specifically targeted toward concepts that had given students difficulty in past semesters of the course.

The in-class activities motivate students to not only attend the class but also actively participate in their learning process. A mandatory class attendance was enforced in the first three semesters during the study period; the class attendance was optional in the last two semesters. However, the class attendance remained almost unchanged at 80-90% percent, showing that the in-class activities were able to attract students even without attendance enforcement.

During the research period, students performance was measured to determine the learning impact on students when the in-class practice problems were implemented. Students’ learning performance was assessed by four midterm exams during the semester and a comprehensive final exam at the end of semester as summative assessments. The final exam questions were pulled from a database of standard questions that were used before the research period. To prevent students from passing-on exams from one year to the next, students were not allowed to keep their final exams. The performance of students in the final exam before the research period is considered as the reference; performance of students during the research period is compared with the reference to evaluate impact of the active learning on students. Each exam covered the same topics, for example, the first exam was concerned with stress and strain in the axially loaded elements, mechanical properties and design concepts; the second exam tested for knowledge of torsion in shafts and bending in beams, and so forth.

Table 2- Academic performance of students during the research period

Semester Reference Test Difference

1 74.5 73.8 -0.7

2 67.7 70.2 +2.5

3 71.5 76.3 +4.8

4 The exam was entirely new, no reference exam

Table 2 shows the performance of students on the final exam during the research period compared to the reference baseline. A grading scale of 0 to 100 was used for the exams. The average exam score of students in the first semester was 73.8; The average performance for the same exams before implementing the active learning was 74.5. The difference is less than 1% which shows there is not a significant difference between the test group and reference group in the first semester of implementing the active learning technique. However, in the second semester of applying the active learning technique, the test group outperformed the reference group by 2.5%. The improvement in the students’ performance was higher in the third semester when the in-class practice problems were incorporated as a main component in the course curriculum. The difference between the test group and reference group was 4.8% in the third semester of applying the proposed active learning strategy. Such an improvement could be attributed to implementing the active learning technique in the course which positively impacted student learning. The exam in the fourth semester was entirely redesigned, so no comparisons were made with the previous semesters. However, the students performance was compared with the performance of two other sections at the same semester that were taught in a traditional format without any active learning activities (Figure 2). Students who used active learning strategies showed a better performance in all midterm and final exam assessments. The difference between the performance of students in the sections with and without active learning activities were +5.6%, 2.9%, 4.5%, 1.8% and 5.1% in the midterm and final exams, respectively. Even though the differences are not very large, the students who used active learning always showed a better performance than students without active learning, confirming the impact of the employed activities on students learning.

Figure 2. Comparing students performance with and without active learning at the same semester Students’ performance in the course is related to their participation and performance in the in-class activities throughout the semester. Such a correlation between in-class activities and the course grade are shown in Figure 3 for the third and fourth semesters studied in this research. The same trend was observed in the other semester. Such a correlation proves that those students who engaged more in the class activities benefited more and learned the concepts better.

Figure 3- correlation between the in-class participation and the total course score, left) 3rd semester, right) 4th semester

Additional data needs to be collected for Aerodynamics 1 in order to fully evaluate the impact of active learning activities in that course as well. However, these same activities tailored to a new subject resulted in improved exam performance on the specific topics covered during the active learning activities for students who elected to participate compared to their peers who did not. For example, students who elected to participate in the active learning activity covering thin airfoil theory averaged an impressive 91% on the exam questions covering this topic compared to an average score of 82% for their peers who did not participate. This promising result motivates additional analysis of student performance before and after implementation of the active learning strategies in this course as well.

Figure 4- Students responses to active learning survey Student’s feedback and future work: A blind survey was administered by the Center for ​ Advancing Faculty Excellence (CAFE) educational technology as an independent entity in the middle of the semester, in the first and second semesters studied in this research. To avoid any bias, no student information was shared with the instructor. Figure 4 summarizes students’ feedback for two questions related to the implementation of in-class practice problems in the course. A dedicated survey focused on active learning activities was conducted in the fourth semester. The results are summarized in Table 3. Given the collected feedback, students see the impact of active learning as a positive factor on their learning. The results presented in Figure 3 support this statement. Students’ feedback was mostly positive for both questions. The majority of students replied Excellent or Good for these two questions. It is worth noting that the student’s opinions were more positive in the 2nd semester compared to the first semester of the research period.

Finally from the attitudinal survey, as a whole, the majority of students seem to have benefitted from the active learning technique conducted during the semester. This same survey was administered to Aerodynamics 1 students after completion of the active learning activities and the responses were similar with all questions averaging 3.8 or higher and the majority of students agreeing or strongly agreeing with all of the questions asked. Comments like this were repeated on the student’s feedback:

“The in class problems are the best part of the class” “In class problems keeps us very involved and provides us with instant feedback on our understanding of the topics.” “In-class quizzes-keep you focused in class because you want to understand how to solve the problem for bonus points.”

“In-class problems have been the best in terms of in class learning. It sticks with me better than just watching examples and not practicing till I start the homework.” “The in-class quizzes are great because they allow you to solve a problem and get the correct answer so you then have a basis on how to approach the hw.” Table 3- Students responses to active learning survey ​

Student responses Survey Questions Average Strongly Agree Neutral Disagree Strongly Agree Disagree

I have a better understanding of this course 3.9 17.6% 67.2% 10.4% 1.6% 3.2% topic following the activity.

I felt prepared to complete the active learning 3.6 10.5% 52.4% 29.0% 6.5% 1.6% activity.

The instructions on how to participate in the 4.0 27.2% 55.2% 12.0% 2.4% 3.2% activity were clear to me.

I felt more engaged in the active learning 4.0 30.4% 42.4% 22.4% 1.6% 3.2% activity than in a lecture-only class.

I learned from my group members during the 3.9 23.4% 50.0% 21.8% 3.2% 1.6% active learning activity

I actively contributed to my group during the 4.0 20.2% 62.1% 15.3% 0.8% 1.6% active learning activity

I would like to do this same type of activity for 3.8 22.6% 47.6% 18.5% 7.3% 4.0% another course topic

I feel that this was an effective active learning 3.9 20.8% 56.8% 16.8% 3.2% 2.4% activity

4- Conclusion Various in-class activities in the form of practice problems were developed and implemented in an introductory Mechanics of Materials course over the course of four semesters. Analysis of student participation in active learning and scores on course exams supports the conclusion that the students’ performance improved when the class was structured to incorporate active learning. As a result of this improvement, which was visible to the course instructor in the first and second semester of implementing the active learning, the instructor has adopted the blended teaching format to allow for conducting more in-class activities. The positive impact of active learning on students' understanding of the material is in agreement with the previous studies reported in the literature [3-6], and preliminary data suggest that this active learning platform and framework can be applied successfully to additional engineering courses.

The successful application of the active learning activities into the course curriculum is encouraging to develop additional activities, specifically those that promote student-student interaction, learning-by-doing, and learning-by-teaching approaches. In this study period, a limited number of guided discussion activities were conducted and the early results were promising. Active learning through guided group discussion requires further data collection and analysis in order to identify its impact on improved learning outcomes.

An additional advantage to using this active learning platform [12] is the ability to collect and reflect on student participation and performance both during and after the activities. Monitoring students’ performance throughout a semester by observing their engagement in active learning activities could be used for developing an automated monitoring system for identifying students who are academically in danger of failing or need special help to be successful in the course. Sadati and Libre [14] used the students performances in the exam, assignments and in class activities to develop an early alert system to identify students in academic trouble before failure. Further development includes using the data collected during the several active learning activities and using data analysis techniques to enhance the model accuracy.

Preliminary data from the implementation of active learning techniques in Aerodynamics 1 showed promising early results for improving student performance, indicating that the activities that were successful in Mechanics of Materials can be tailored to work for a broader range of engineering courses. It was observed that utilizing similar active learning activities in multiple courses across the curriculum increased student comfort with the activities and lowered the barrier for other instructors to participate. When first implementing active learning in Aerodynamics 1, most students were already familiar with these activities from their previous experience in Mechanics of Materials, so there was little resistance and students were easily engaged in activities that they were already comfortable with.

As these and additional adaptable learning activities continue to be developed, validated, and shared throughout a campus and the broader engineering education community, other instructors may be encouraged to adopt similar activities in their own teaching practices with greater confidence. Equipped with templates for activities that have been used successfully in other engineering courses, instructors can more easily overcome the initial hurdle of incorporating a new instructional practice and students may be more readily engaged in the activities due to increased exposure to active learning in multiple courses. Evidence supports improvement in student performance with the incorporation of active learning techniques, and by lowering the barrier for instructors to incorporate similar activities into their teaching, student learning may be improved across a wider range of engineering courses. References [1] Bonwell, C., Eison, J. (1991). “Active Learning: Creating Excitement in the Classroom”. Information Analyses - ERIC Clearinghouse Products (071). pp. 3. ISBN 978-1-878380-08-1. ISSN 0884-0040. [2] J. Schultz, J. Wilson, and K. Hess, (2010), "Team-based classroom pedagogy reframed: the student perspective," American Journal of Business Education, vol. 3, no. 7, pp. 17-24 [3] Compeau, C.R.., Talley, A., Tran, P.Q., (2019). “Active Learning in Electrical Engineering: Measuring the Difference”. Proceedings of the ASEE Annual Conference. Tampa, FL [4] Dutson, A., Green, M., Jensen, D., and Wood, K. L., (2003) "Active-Learning with Engineering Mechanics: A BuildingBlock for Design," Proceedings of the ASEE Annual Conference. [5] Jensen, D., Wood, K.L., And Wood, J., (2003) "Hands-on Activities, Interactive Multimedia and Improved Team Dynamics for Enhancing Mechanical Engineering Curricula," International Journal of Engineering Education, Vol. 19, No. 6, pp. 874-884 [6] Prince, M., (2004) “Does Active Learning Work? A Review of the Research.” Journal of Engineering Education, Vol. 93, No. 3, pp. 223-231 [7] Bohnhoff, G., Sample-Lord, K.M., (2019) “Creating a Library of Group Activities that Promote Active Learning in the Undergraduate Soil Mechanics Classroom '', Proceedings of the ASEE Annual Conference. Tampa, FL [8] Reed, B. (2018). "Active Learning Success by Partnering Across the Institution.". Proceedings ACM SIGUCCS User Services Conference, pp. 69. doi:10.1145/3235715.3235718 [9] Adarme, M., Jabba Molinares, D. (2018). “SEED: A software tool and an active-learning strategy for data structures courses”. Computer Applications in Engineering Education, 26(2), 302-313. doi:10.1002/cae.21885 [10] Herring, B., St Jacques, R., (2019) “Using Active Learning to Increase Student Retention in Introductory Computing Courses”, Proceedings of the ASEE Annual Conference.Tampa, FL [11] Dallal, A., Clarck, R., M., (2019) “Progressive Use of Active Learning in Electrical Engineering Courses”, Proceedings of the ASEE Annual Conference. Tampa, FL [12] Library of Active Learning Activities, Mechanics of Materials database, (2020) “www.LearnItDeeper.com”, accessed April 2020 ​ ​ [13] Felder, R. M., Brent, R. and Oakley, B. A. (2016) Teaching and Learning Stem: A Practical Guide, Jossey-Bass a Wiley Brand, San Francisco. [14] Sadati, S., & Libre, N. A. (2017), “Development of an Early Alert System to Predict Students At Risk of Failing Based on Their Early Course Activities”, Proceedings of the ASEE Annual Conference., Columbus, Ohio. https://peer.asee.org/28166