Improving Alaska Students' College Math Readiness Michelle Hodara December 4, 2013

The first thing I want to do is give you some idea about the extent to which students are considered academically under-prepared for college-level math. The data I'm drawing from is the most recent national data we have; it's from the 2003 cohort. They were tracked to 2009 in a study by the US Department of Education and they sampled students across the country so that they could come up with a nationally representative sample. 2003 is the most recent cohort we have.

For the 2003 cohort 60% of those students who started at a public two-year college took a remedial math course and 17% took a remedial English course. The numbers are a little bit better when we look at public universities, but still one in three students who started in a public four-year college took a remedial math course and 7% took a remedial English course. Remediation is everywhere: it's in all types of colleges; so at the private nonprofit four-year colleges, 22% of students took a remedial math course and 9% took a remedial English course, but I don't have the numbers up here for the for- profits but the remediation numbers are a lot higher. Actually they are around the public two-year numbers, so clearly remedial math course taking is much higher than remedial English. And even though we're looking at the 2003 cohort, if you look at specific states' higher education institutions and you calculate these numbers, they are still up there, about the same. They can be a little bit lower or a little bit higher but these numbers are not going to be that much different.

If we want to look at what percentages of students earn credits for college math courses, this is reflective of passing a college math course. Among students who are pursuing an associate's degree (a two-year degree) 47% of the total cohort were able to earn credits and pass an introductory math course, usually college algebra, and 9% were able to pass calculus, or an advanced math course.

What happened to the other students? They either dropped out of college or they failed a math course and they don't have credits in a college math course. If you look at four-year students pursuing a four-year degree, again we see better numbers: 66% or two-thirds of students who were pursuing a four-year degree passed an introductory math course and 34% passed a calculus or advanced math course. There's a big difference between students pursuing different degree types, but the numbers are not even close to 100%; so there's some work to be done there.

The last thing we're going to look at is persistence in the STEM major. We all know there's all sorts of consequences for not completing a college math course: you are less likely to be able to graduate, it can have implications for the types of employment available to you, and students' interest and persistence and completion of STEM majors. We're looking at the same data: for the 2003 cohort, 28% of four-year and 20% of two-year students began college as STEM majors and six years later we have among AA students 37% of STEM majors dropped out, a third switched majors, and a little less than a third or 31% persisted and completed their STEM major. Among students pursuing a four-year degree, one in five dropped out, 28% switched and about half of them were able to persist. So half of 28% is a relatively small proportion of students actually earning a STEM degree. Not only does this have implications for students, but also for the American economy because we need to grow our science and technology industries to grow our economy and to be competitive globally; this is a third set of data for you to think about.

In a brief aside before getting to the research, this is about research: I'm only going to present rigorous research today and what I mean by that is research that can identify the exact 1 effect, the precise impact, of an intervention. So say you want to see what the impact of carrot juice drinking is on an individual and you observe that individual has healthy heart outcomes. So you think, "Oh that's because of the carrot juice drinking," but what if all the carrot juice drinkers just lead a healthy lifestyle and so it's really impossible to tell if their healthy heart outcomes are the effect of the carrot drinking behavior or the healthy lifestyle. This is the problem in research we're trying to address all the time: How do we actually tie an intervention to an outcome? There is tons of research out there on math interventions and reforms: that's why the original paper is 70 pages long and has all that research in it. But what I decided to do for the brief was just narrow it down to the studies that were very good at tying that intervention to the outcome and telling us about the precise impact of the intervention. So, I'm only going to talk about rigorous research today, and by that I mean randomized controlled trials and quasi-experimental designs, which had a treatment and control group who were very similar and so that we can really know what impact that intervention had; it didn't have to do with any other thing going on in that student's life. But the drawback to that is that here's very limited generalizability of the findings from these studies. They took place in a specific context with the specific population so we don't want to think, "Oh they didn't work for this population, we're not gonna consider them at all or our students." I think the benefit of this research is that we can be confident in the results and they tell us the story about what happened in that specific context and they help us think about how can we adapt that intervention for our context if we're interested in that reform or intervention and make sure to improve them so that they maximize their impact for our students. Again, there's going to be limited research I'm presenting, but it's really only randomized controlled trials and quasi-experimental designs. Everything else is not in this presentation.

When I get into strategy number one, intervening pre-matriculation, that's intervening before students start college, I'm gonna talk about the definition of each intervention and how they're typically implemented across the country. In your brief there are short definitions: those are common ways that people are using these terms and implementing them. The first is early assessment, which is basically an assessment given to high school students to get an early indication of their readiness for college. There's a national scan and right now 25 states across the country have statewide early assessment programs, which means all high school students have the opportunity to take a statewide assessment that tells them if they're going to be ready for college at a specific statewide university system. I'll talk about the California early assessment program in a little bit. Then 8 of those 25 states have transition curricula, which means that students who do not test college-ready have the opportunity to take courses in their senior year to brush up on skills that they have academic deficits in so that they can retest and see if they can test into college-level courses. Those statewide early assessment programs may become more common with the implementation of PARC and Smarter Balanced in states that have adopted those because those are essentially an early assessment of college readiness. Then we have bridge programs, which have a long history in higher education. They're typically month-long and they happen during summer, usually for students who place into remediation or students going into a STEM field who aren't ready for that STEM math course and they brush up student skills and they orient them to college life and give them some college knowledge. Then we have boot camps and brush-ups, which a national scan showed they're fairly popular at community colleges. They're fairly low-cost and easy to scale up. Boot camps usually last a week; they're similar in that they're geared to students who place into remediation and they're given to them to brush up on their skills and orient them to college culture. Brush-ups are usually just a few hours long--maybe an online course or a course given to students who don't do well on the placement exam.

This is the evidence; there are two rigorous studies. The first one was the California Early Assessment Program. The main component of the California Early Assessment Program is that when high school juniors take the state standardized tests for high school exit, they're given the option to take test items at 2 the end of the test that were developed in collaboration with high school teachers and the California State University system and those optional test items test their readiness for college-level coursework at the California State University System. The community colleges also adopted them. High school juniors receive a feedback form that says you tested ready in college-level coursework. When you go to CSU you will automatically be able to enroll in college-level coursework or you didn't test ready and you need to do something about that in your senior year. In this study the researchers found that this program has a small positive impact in lowering remediation rates; they looked at California State University, Sacramento specifically. They found that students were about 4 percentage points less likely to place into remediation and more like 4 percentage points more likely to place into college math and they hypothesized that more students were doing something in their senior year to brush up on skills in order to place into college math and so I have a question mark on positive impact because it's not clear the mechanism underlying early assessment, so early assessment is simply an assessment given to juniors usually, and so it's not clear how it's working to decrease remediation rates or to get students more likely to start college in college-level math but it may have an impact on their learning if they're actually taking more math in their senior year. Then there is a large-scale randomized controlled trial in Texas where they took recent high school graduates who placed into remediation at four community colleges in two open access 4-year colleges and they randomly assigned them to a summer bridge program or to nothing at all-- they could just spend the summer doing whatever they wanted. Then they tracked them through the summer and over a two-year period. This was a fairly intensive month long summer bridge program where they're given academic courses, they were given a stipend to incentivize them to stay, and some college knowledge curriculum. At first students seemed like they were enrolling in college math and passing college math at higher rates than the control group who didn't have a summer bridge program, but after two years there is no difference in any outcomes between the treatment and control groups. This is a fairly expensive randomized controlled trial and program and so I think that the overall lesson is that these programs that focus on improving placement test performance may have little effect on students' long-term college success. You can’t expect short-term programs to have a long-term effect. Also this took place, again in a specific context, so it's unknown what would happen if it was somewhere else, with different students or in a different area of the country, but interestingly many of the people involved in this program are now looking at another type of bridge program in the state of Washington in which it basically starts in senior year where college readiness is assessed and then they start brushing up on skills in their senior year in high school. If they're still behind at the end of their senior year they take a summer bridge program and then there are supports that go into their first year in college. It’s kind of a long-term set of supports that are given to students and they're going to look at how those effects might be different than a summer bridge program.

I'm going to give you a few minutes to talk about your experiences with early assessment, bridges, and boot camps and brush ups: if you have any student success stories or have lessons learned about challenges implementing them. If you don't have any experience with these programs or if you want to answer the second question-- what is the key to developing and sustaining strong partnerships between high schools and colleges? So many of these programs require partnerships so how do you develop and sustain them? I'm going give you a few minutes and then ask for a couple of people to share what they come up.

In systems that are going through system-wide reform and oh I'm also very passionate about this because I was a developmental education instructor so I come at this topic with a lens of thinking about my students and and that experience which was wonderful. The first model we're going to look at is learning communities, which is when students enroll in two or more courses together in the same semester and out of all the reforms, this has the longest history in higher education but interestingly in 3 all these developmental education reform efforts learning communities are not discussed as much because they're not really considered an acceleration strategy; a strategy to accelerate students' progress through developmental education, but they're still probably more common than the other three. The next is modularization, which is typically when they take a semester-long math course and break it up into discrete units or competencies and students only work on those units or competencies that they haven't mastered yet and it's usually delivered through some computer-based software. The third is compression: compression is probably the most popular. Another national scam found that compression is pretty popular for developmental education reform right now. Basically what compression does is it takes a developmental math sequence--and I know I'm preaching to the choir in a lot of this stuff- but developmental math sequences can be three or four semesters long so it takes two years to get through them if you're placed at the lowest level, or a year and a half if it's a three semester long sequence. Compression takes two or three courses and compresses them into one semester so it might take two courses that are each 4 or 5 contact hours and now students have to take one semester- long course that's 8 contact hours. The idea is that it's eliminating opportunities for students to drop out of the sequence and be pulled away for one reason or the other. The visual is basically compression; you have the compressed pathway and the traditional pathway and usually they maintain the same contact hours. Sometimes they reduce them, but a lot of times they're just basically a really long course that students take in a semester. And then finally we have mainstreaming, which is when students who place into developmental math are allowed to enroll in college-level math and take a support course concurrently that addresses their academic deficits. This is much more common in English than in math but I do have a couple of studies on it from math. The Community College of Baltimore County (CCBC) has a mainstreaming English model that's being replicated across the country, so do a few colleges in California, but in math not so much, which is an interesting thing to think about. Here's a little bit more evidence; it's going to take a little time to get through this slide. There's one randomized controlled trial of learning communities in six colleges and two of those colleges decided to pair a developmental math course with a student success course. They found that more students in the learning community passed their developmental math course than students in the non-learning community, in the traditional developmental math course. But then after tracking their outcomes over time, there were no differences between the students in the learning community and the students who weren't in the learning community. They have the same college math enrollment and pass rates, the same credit attainment and persistence, so the effect was really isolated to that semester in which they're in the learning community. There weren't any long-term impacts; interestingly though, there's also an English learning community that was also in the study and the English one had some really long-term impacts on even degree attainment. It was at Kingsborough Community College in New York City and the biggest difference between developmental English and the college English learning community and the math one was that they paired the English course with two other courses, the three courses together, so the cohort model was exponentially bigger and the other big difference was they think that there's more curricular integration between the three courses. It really matters how these interventions are implemented a lot, and one learning community is not the same as the next one. Then we have modularization and this study took place in Tennessee--it was at Jackson State Community College and Cleveland State Community College and Tennessee redesigned their math courses and those two colleges decided to take their semester-long developmental math courses and turn them into modules that students work on through computer-based software. The redesign initially had a negative impact on students' persistence: the students in the modularized courses were more likely to drop out and then over time there was no difference in persistence and credit attainment between the students in the modularized courses and those in the traditional math sequence. So, that initial negative impact is hard to figure out what happened. It could have been the change in curriculum, which meant the curriculum was modularized or it could have been the pedagogical change, which means the content was delivered through computer-based learning and perhaps there was not enough instructor guidance 4 or structure to the course or students didn't know how to use the software. There are all sorts of reasons why students have a hard time adapting to courses that are delivered through computer-based software. This is a really popular reform, so other research could find different things, but this is in a very specific couple of community colleges. Then we have compression; there's been two studies and there's a compression model at the Community College of Denver called Fast Start. They've been doing it for quite a while and they have some fairly positive results. Students in Fast Start in the compressed courses are more likely to enroll in and pass college math than students in a traditional math sequence. Fast Start also has some nonacademic supports as well. They do case management and they give students some support besides the shorter math sequence. Then the second study is Hodara and Jaggers-- that's me. We looked at the impact of shorter versus longer sequences at the City University of New York Community Colleges. There's six community colleges there and we found that students who were in shorter math sequences were more likely to enroll in and pass college math. There's also a very small, one percentage point impact on degree completion. You think one percentage point is nothing, that doesn't mean anything, but developmental math students who place into the lowest level of math at the University of New York had something like a five percent degree completion rate, so if you add one percent to that, that's six percent so that's pretty good. Compression is basically about shortening the sequence and so it seems that based on two studies--limited research-- it's a straightforward way to improve outcomes. Then we have mainstreaming. Boatman also looked at a mainstreaming reform in Tennessee at a four-year college in which students who previously placed in developmental math were now allowed to enroll in enhanced college-level math courses. They were given a lot a supplemental support that was designed specifically with their needs in mind. There's a lot of thought into how to mainstream these students and they've been seeing fairly positive results with credit attainment and persistence. That study didn't look at degree completion, but I combined them all here; in the brief you know exactly what goes with which study. The final study is an evaluation of I- Best in Washington and that model also has been found to be fairly promising. I-Best is for two-year colleges, community colleges, and an occupational instructor will team teach with a basic skills instructor. They're required to team teach at least 50 percent of the time and so all the basic skills instruction is integrated into a college-level occupational or career technical course. That's reading, math, writing, everything. Students do not take noncredit-bearing courses before they could get into their college-level courses. That program has been found to have a positive impact on their persistence and attainment of certificates, not associate or bachelor degrees, but these are occupational programs. That's a lot of evidence; there you go, it's your turn to talk. What are your experiences with reforms, developmental math programs? What do you think is key to developmental math students' college math success and how can developmental math reforms address these key factors? What's missing from some of these reforms I just talked about?

I'm going to focus on two strategies that I thought the studies were fairly good: not randomized controlled trials, but pretty good and they did a pretty good job of making sure that students who received the instructional strategy and those that didn't were similar. One of them is structured student collaboration and that's basically just cooperative learning. A lot of people use cooperative learning; that's what I'm doing right now and what you've been doing all day really, but theories and research around cooperative learning say it's not just about people coming together and talking; you really have to have some structure, some formal roles and responsibilities and that's when people really benefit from cooperative learning. We're going to look at that and the second one is using multiple representations, which is what the visual is looking at. It's when you solve math problems and think about math problems in multiple ways, symbolically, graphically, with words, with tables and graphs, with equations, and formulas; that's supposed to improve your conceptual understanding. It's not clear how common these are across the country in postsecondary math classrooms. There's really no national scans like the other interventions and reforms I talked about, there's literature on how common these 5 are, but there isn't anything about what does postsecondary math instruction look like across the country. There are some qualitative studies that took place in California and what they found is for the most part in developmental math classrooms and community colleges it was lecture based, it was skill and drill; it was problems that were devoid of application to the math students use in their degree programs or in their everyday lives. This strategy is about something different than that (project based learning). There are three studies and there's a couple of studies on structured student collaboration; they're a little bit older but they were pretty well done. Both of these were in developmental math classrooms that used a fairly simple pedagogical changes to the classroom to make sure that students were in groups and had roles and responsibilities when solving math problems. For example one in the Dee study, basically they broke up the problem and each person had a part of it and had to figure out how to put the problem together to figure out what the problem was asking and then solve it together. It was fairly simple. In both studies they found that students had higher developmental math pass rates or were performing better on exams that tested their mathematical understanding and achievement. None of these studies looked at anything except for what was going on in that classroom at the time; they didn't look at long-term outcomes so we don't know. But both of these studies showed that these small changes seemed to have a significant impact on students' math learning. The last study using multiple representations is another really interesting one to read and I really liked how they implemented the research study. This took place in a college calculus classroom and they had classes where faculty were given professional development around making sure students were learning and solving math calculus problems in multiple ways and then they had other classrooms where teachers were using the traditional textbook and going through it in a linear manner with more of a focus on procedural learning. At the end they gave all the students the same types of tests, the same standardized exams and the same open-ended questions and they found that students in the classrooms at emphasize multiple representations did much better and they're also able to answer questions that had never been introduced, which means they're transferring the skills they learned to other concepts. Not a very surprising finding, I think, but something that might validate what people think about how to teach math. That's it for the research; we're going to talk about this, but I'm going to go over one more slide about some final thoughts. But the two questions I have are what are your experiences with trying different math instructional strategies and/or curriculum? And the last one is what can developmental math faculty learn from effective math teaching at the secondary level and what can developmental college math faculty learn from each other? Is what someone does in a special education classroom in high school that really works with students who have an IEP and have math deficits, is what that teacher does okay to take and adapt to use in developmental math or is something really great that's going on in a college calculus classroom okay to take and adapt to a developmental math classroom? If yes, what's going on across sectors that we can take and we can learn from each other and take and use to improve students' college math success?

Just to end, I’m going to talk about some final considerations based on this research. It is limited so it's hard to kind of come up with some revolutionary findings, but you can glean some lessons from all this research that's been done. One is that if a lot of these programs are having a limited impact to that semester in which students are actually in that intervention, then these interventions need to be connected to a broad set of interventions that are happening throughout students' cradle to career ideally, but at least high school to college. So, when you're thinking about system-level policy, you want to design and offer all students a connected, comprehensive, and cost- effective set of supports from high school to college and that's how you're really going to reduce remediation. It has to start in high school and it has to go into at least the first year of college, it has to include all sorts of different types of interventions and supports, which is hard to do but if like you said there's a funding structure that's not burdening one particular sector or institution it makes it easier. Second, thinking about developmental education reforms, what's happening across the country is 6 shortening the sequence or having students would have been in developmental education co-enrolling in a college-level math course. In thinking about this reform it's really important to also attend to students' nonacademic needs. A lot of the research on things that have a really positive impact have other supports: they have case management, they have really intensive tutoring and supplemental supports that are targeting students exact academic deficits. Even financially: I-Best actually has financial aid help for students. When we talk about the positive impact of I-Best, students are getting some financial aid support as well. This brings up a newer form, the statistics pathways in which they eliminate the algebra, the remedial algebra sequence, and instead you have a one semester-long remedial statistics course. They don't call it remedial, it's pre-stats that leads into a transfer-level college statistics course. It's a yearlong sequence where non-STEM students can get their college-level math course and complete it. What it does is it's completely project-based; it's completely math that students use to read the newspaper and understand what they're reading when they look at a percent. It talks about the difference between a correlation and a causal inference, all sorts of math skills that you want in your everyday life. It’s for non-STEM students and that's something that's happening in California a lot and in some other states and it hasn't been studied in a rigorous way but it's definitely one that is thinking about other things besides shortening the sequence. It also explicitly talks to students about if you make a mistake it's okay, that's part of learning, that means that you're actually approaching challenging work and that's important. The last is just classroom-level change and thinking about integrating effective math pedagogy into all these interventions.

That's where I will end so we can have the panel.

7