MTH 086 SLOAT Assessment Study Report (Fall 2011)

MTH 086 SLOAT Assessment Study Report (Fall 2011) prepared by Ines Figueiras and Shohreh Andresky

Introduction Two full-time faculty members of the Division of Mathematics and Physics continued their participation on the College’s Student Learning Outcomes Assessment Team (SLOAT) for a second semester to conduct a study of the MTH 086 course (Introductory Algebra). The faculty members that conducted the study were Shohreh Andresky and Ines Figueiras, both Instructors.

MTH 086 is the most commonly offered course at Essex County College. Statistics provided by the Office of Planning, Research and Assessment over three fall semesters (Fall 2006, Fall 2007 and Fall 2008) shows that, on average, 82% of new students to Essex County College place into MTH 086. Approximate 4,500 students enroll in MTH 086 in an academic year. In the 2011 Mathematics Program Assessment Report, an analysis of success rates in an eight-semester time period (Spring 2006 through Fall 2009) revealed a success rate of 46%. During that time period, only 6,617 of the 14,400 students who took the course passed. The course description is as follows: “This beginning mathematics course is designed to take students from concrete arithmetic ideas to the more abstract algebraic forms of these ideas. Throughout the course, emphasis is placed on the development of arithmetic and algebraic skills and the application of these skills and concepts to the solution of practical problems. Topics covered include simplifying arithmetic and algebraic expressions, signed numbers, fractions, decimals, percents, estimations and geometric applications.” Basically, MTH 086 is a course that covers material that students should master in the elementary and middle school years (typically third grade through eighth grade). Therefore, most students taking MTH 086 need to spend a considerable amount of time studying and doing homework to master the material in this course traditionally taught in a fifteen- week time period at ECC.

Purpose This assessment study had several objectives, which are as follows:

 to analyze student performance on questions blueprinted to course-level Measurable Performance Objectives (MPOs) and compare these outcomes to those obtained during the Spring 2011 study;  to determine the impact on success rates of variables such as class size, day classes versus evening classes, full-time versus part-time instructors, and the main campus versus the West Essex campus and compare those results to those found in the Spring 2011 study;  to determine if the grading of homework has an impact on final course grades;

MTH 086 – 1  to determine student perceptions of the most difficult topics in the course and student self-reported reasons for not completing all homework assignments. The MPOs that were analyzed in this study all relate to course goal #1. Based on findings of Spring 2011 SLOAT Mathematics course studies, the MTH 086 course outline was modified to include a new topic (i.e., simplification of perfect square radical expressions). Specifically, this topic was added based on the MPO analysis results of the MTH 100 SLOAT study which indicated that simplification of radical expressions was problematic for students. Thus, a new objective (MPO 1.10) was added to the MTH 086 course outline that relates to Course Goal #1, which was analyzed in this current SLOAT study. Course Goal #1 and the corresponding MPOs examined in this assessment report are listed below.

Course Goal #1: Demonstrate knowledge of the fundamental concepts and theories from arithmetic, algebra and geometry.

1.1 perform arithmetic operations on signed numbers; 1.2 perform arithmetic operations on fractions; 1.3 perform arithmetic operations on decimals; 1.4 perform arithmetic operations on percents; 1.5 determine the perimeter and area for simple geometric figures; 1.6 determine whether a ratio is a proportion; 1.7 convert from one unit of measure to another; 1.8 simplify basic algebraic operations; 1.9 solve simple linear equations involving one operation; and 1.10 simplify perfect square radicals.

Methodology

Seven sections of MTH 086 were selected to participate in this study. These sections represented classes taught at the Main Campus and the West Essex Campus, by full-time and part-time faculty, during the day and during the evening, and with class sizes that are less than or equal to 24 and greater than 24. Instructors were asked via e-mail to complete an online survey (www.zoomerang.com) regarding MPO achievement. A separate zoomerang student survey was also sent to the instructors with instructions to administer the survey to the students in their respective classes. The student survey asked questions about which course topics the students found most difficult and asked them to identify and/or provide reasons why they do not complete all homework assignments.

Sample

The sections included in this assessment study were not randomly selected, since sections were chosen to represent students taught at the Main Campus and the West Essex

MTH 086 - 1 Campus, by full-time and part-time faculty, during the day and during the evening, and with class sizes that are less than or equal to 24 and greater than 24. The sample included 7 sections with 170 students enrolled, but data from only 111 students was included in the MPO analysis. Only 13 students took the online student survey. Information regarding the sections that participated in the study is detailed in the table below.

# of Main (M) Full-time (F) Day (D) Section Students Instructor West Essex Adjunct (A) Evening (E) Enrolled (W)

005 29 John Bottger A D M [email protected]

019 27 Ming McCall F D M [email protected]

025 31 Ines Figueiras F D M [email protected]

Kristen Schluz 046 21 Kristen_schulzg@ A D M yahoo.com

Nataliya Chentsova CW6 25 nchentsova@ A D W yahoo.com

Gamal Safa ODC 22 Gamalmath086@ A E M yahoo.com

ORC 15 Errol Campbell A E M [email protected]

Total: 170

MTH 086 – 3 Some general information regarding the sections that participated in the study is summarized in the table below.

% Taught % Taught % of West % of % of Main by Full- by Part- % of Day Essex Evening Campus Time Time Students Campus Students Students Faculty Faculty Students 29% 71% 71% 29% 91% 9%

Thus, the typical student participating in this study is taught by an adjunct instructor at the Main Campus during the day.

MPO Results MPO results were determined by examining student performance on the multiple-choice section of the final exam. Each multiple-choice question on the final exam was blueprinted to one of the MPOs related to Course Goal #1. A total of 111 students (65% of the original student cohort) took the final exam. During the Spring 2011 semester, 63% of the students initially enrolled took the final exam. A total of 59 students (or 35%) either withdrew or stopped attending class. No students received a grade of “I” (incomplete). The instructors were asked to count the number of students who answered each multiple-choice question correctly. Those results were submitted through an online survey, which also requested responses to other questions needed for this study. An MPO is considered ‘achieved’ in this study if 70% or more of the students answered the question successfully. All 10 of the MPOs analyzed were achieved as is summarized in the table below. These are the same results as found in the Spring 2011 SLOAT MTH 086 assessment study.

MPO % Achieved 1.1 perform arithmetic operations on signed numbers 77% 1.2 perform arithmetic operations on fractions 71% 1.3 perform arithmetic operations on decimals 75% 1.4 perform arithmetic operations on percents 81% 1.5 determine the perimeter and area for simple geometric figures 84% 1.6 determine whether a ratio is a proportion 79% 1.7 convert from one unit of measure to another 77% 1.8 simplify basic algebraic operations 75% 1.9 solve simple linear equations involving one operation 79%

MTH 086 - 1 1.10 simplify perfect square radicals 80%

Other Variables Impacting Student Success

The course success rate for this sample is only 40%, which is just slightly higher than the success rate calculated for the Spring 2011 semester. However, this is concernedly lower than the average MTH 086 course success rate of 46% described in the 2011 Mathematics Program Assessment Report. Course success rate is calculated by adding all of the students who earn a passing grade in the course and dividing that sum by the total number of students in the cohort (including those who earn “Withdraw” and “Incomplete” grades). Course success rates among the sections for this study ranged from 17% to 68%. Just as in the Spring 2011 study, the biggest impact on the calculation of success rate are the students who either withdraw or simply stop attending and thus earn a default grade of “F.” Specifically, 59 of the 170 students in this study cohort did not complete the course, which represents 35% of the students in the study. Of these 59 non-completers, thirty-four students (20% of the study cohort) did not show up to take the final exam, and therefore earned ‘F’ grades; and twenty-five students (15% of the study cohort) withdrew from the course. This means that only 25% (43 out of 170) of the students in the study cohort failed the final exam.

The average course success rate for the 7 sections is 40%, and the median course success rate is 48%. The chart below depicts the course success rate for each section that participated in the study by full-time and part-time faculty.

MPO results are significantly higher than the course success rates previously discussed. All 10 of the MPOs analyzed were achieved, but the course success rate for this student cohort is only 40%. Since the course success rate calculation includes all students enrolled in the cohort, students who do not take the final exam significantly lower the course success rate calculation. Omitting students who did not take the final exam, either because they withdrew or stopped attending, returns a much higher course success rate of 61% (i.e., the number of students who passed the final exam divided by number of students who took the final exam). These are the exact same results found in the Spring 2011 study. In addition, the multiple-choice section of the final exam is typically a bit easier than the rest of the exam. Therefore, it is reasonable that the MPOs analyzed on the final exam were all achieved despite the low course success rate for this sample.

Several factors were compared to conduct further analysis of course success rates including class size, day classes versus evening classes, full-time versus part-time instructors, and the main campus versus the West Essex campus,. The following findings are noteworthy:

MTH 086 – 5  The three MTH 086 sections with class sizes of 24 or less had a higher course success rate of 45% than the four sections with class sizes greater than 24 (38%). These are similar result to the Spring 2011 assessment report where the sections with smaller class sizes had an average course success rate of 43% as compared to the 34% average course success rate for sections with larger class sizes.

 Only two evening MTH 086 sections participated in the survey. The evening sections had a significantly higher success rate at 54% than did the day sections (36%).

 Full-time faculty members taught two MTH 086 sections, and part-time faculty taught five sections. The average course success rate for full-time faculty was only 34%, as compared to the 43% average course success rate of part-time faculty.

 Six MTH 086 sections taught on the Main Campus and one section from the West Essex campus participated in this study. The sections held on the Main Campus had an average course success rate of 35%, whereas the sections taught at the West Essex campus had a significantly higher rate of 68%.

Learning Strategies Students are typically taught through lectures. Instructors use various teaching techniques including board work, seatwork, and group work. Students are made aware by their instructors of the various tutoring facilities located throughout the College, notified of online resources, and encouraged to do all homework assignments.

This study revealed that 86% of the instructors involved in this study grade homework and count the homework scores in the final course grade. As the use of technology continues to increase in the MAP Division, this study also revealed that 50% of the instructors who reported grading homework use online homework systems (i.e., WebAssign or MyMathLab). This is a significant increase over the 36% of instructors who reported using online homework in the Spring 2011 SLOAT study. The instructors who grade homework but do not use online homework systems collect the homework on paper. The average homework grade for the students who took the final exam was 61%. This is lower than the average homework grade of 72% reported in the Spring 2011 study. The importance of doing all homework assignments in a timely manner is crucial to student success in MTH 086, regardless of whether the instructor grades the homework or not. The MAP Division provides each instructor a course outline with a complete list of recommended textbook homework problems.

In addition, the textbook selected by the MAP Division for the MTH 086 course at the College is custom packaged with a workbook and DVDs containing lectures when purchased at the ECC Bookstore. These DVD lectures provide a valuable resource for students who miss class or simply need to revisit a lecture. The textbook is also sold with

MTH 086 - 1 the WebAssign access code providing students with access to the online homework system.

Student Survey A link to an online survey was sent to all of the instructors involved in the study with instructions on how to e-mail the link to their students. Several reminders were sent over a two-week time period to send the link to their students and to remind them during class to take the brief survey. Unfortunately, only thirteen students in the entire cohort took the survey. These survey respondents represent only 8% of the original student cohort, and only 12% of the cohort students who took the final exam. Nevertheless, the responses of these 13 students provided valuable information. Some results are as follows:

 Sixty-two percent indicated studying between 1 and 4 hours per week for their MTH 086 course, 23% indicated studying more than 4 hours per week for their MTH 086 course, and 15% indicated studying less than one hour per week for their MTH 086 course.

 Sixty-two percent reported that their math background did not prepare them well before coming to ECC, and 38% believed their math background prepared them well.

 Of the students who used an online homework system, 90% felt the system helped them learn math better.

 Whereas forty-six percent indicated that they complete all homework assignments, 23% indicated that their work schedule prohibits them from completing all assignments and an additional 15% reported that they do not complete the homework because they do not understand it.

 Students identified the three most difficult topics in MTH 086 as follows: percents (58%), converting units of measure (58%), and decimals (42%).

 Students identified the top three ways to improve the MTH 086 course as follows: assign tutors at assigned times (58%), provide computer labs with math tutors (33%), and reduce class sizes (25%).

Conclusion and Recommendations In conclusion, the success rate for MTH 086 continues to be of concern. The results of this assessment study did not reveal anything new from the study conducted in the Spring 2011 semester, although many results of the previous study were confirmed. MTH 086 is the course that most students place into, and it is typically a problematic course for students to pass. This study revealed a course success rate of only 40%, although

MTH 086 – 7 students in this study cohort achieved all 10 of the MPOs analyzed. As discussed earlier, a high attributor to the low course success rate is the number of students who withdraw from or stop attending the course. In this student cohort, only 65% of the students enrolled actually took the final exam. The course success rate is much higher – 61% – when students who did not take the final exam are omitted from the course-success-rate calculation. Factors such as class sizes of 24 or less, being taught by an adjunct instructor, being taught in the evening and being taught at the West Essex campus appear to have a positive impact on success rates, but further studies with a larger sample size are needed to validate these findings.

It is recommended that a similar study be done with a larger sample size. It is noted that the current process of data collection relies primarily on the cooperation of the instructors. Since MTH 086 has many sections that are primarily taught by part-time instructors on multiple campuses, communication is often difficult and time-consuming. Student attrition in MTH 086 continues to be highly problematic. Poor math and reading backgrounds certainly contribute to the low success rate of this course, but other factors such as poor attendance, poor study habits, poor note taking and lack of homework completion are also relevant contributors. The CSS 101 (College Success Seminar) course offered at the College addresses many of these issues and should certainly emphasize how significantly student behavior and attitude correlate to student success. At this time, not all students are required to take CSS 101.

Finally and most importantly, longitudinal studies are strongly recommended. Success rates and MPO assessment are meaningless unless students can retain and apply their knowledge to successfully complete the next course. At a minimum, students should be tracked from MTH 086 or AFM 083 through the completion of their first college-level math course. In order for these longitudinal studies to occur, more efficient protocol and data access should be fully supported by College administration.

Instructor Figueiras will be class testing a new book, published by Pearson, that is linked with MyMathLab, an online computer-assisted learning technology. The model she will be using in MTH 086 section 031 is different than the way most MTH 086 sections are conducted. The course will be run with two lectures and one computer lab session per week, instead of three lectures per week. The students will work on MyMathLab while in the computer lab and will be required to work on MyMathLab outside of the computer lab session as well. MyMathLab will provide a pre-test based on the week’s lectures, a customized homework assignment based on the individual student’s pre-test results, and a post-test. Students will be graded on the post-test results. The post-test results will represent 15% of the overall course grade of the student. The next SLOAT study will highlight the results of Instructor Figueiras’ Spring 2012 MTH 086 section 031 class test.

MTH 086 - 1