Year Seminars on College Students' Life-Long Learning Orientations

Home , Multicollinearity

Padgett, R. D., Keup, J. R., & Pascarella, E. T. (2013). The impact of first-year seminars on college students’ life-long learning orientations. Journal of Student Affairs Research and Practice, 50(2), 133–151. doi:10.1515/jsarp-2013-0011 Available at http://dx.doi.org/10.1515/jsarp-2013-0011

Innovations in Research and Scholarship Feature

The Impact of First- Year Seminars on College Students’ Life-long Learning Orientations

Ryan D. Padgett, Northern Kentucky University Jennifer R. Keup, University of South Carolina Ernest T. Pascarella, The University of Iowa

Using longitudinal data from the Wabash National Study of Liberal Arts Education, this study measured the impact of first-year seminars on college students’ life-long learning orientations. The findings suggest that first-year seminars enhance students’ life-long learning orientations and that the effect of first-year seminars is mediated through vetted good practices. These findings provide empirical evidence that first-year seminars can positively impact complex learning in ways that were previously untested.

Findings from the Association of American Colleges and Universities’ (AAC&U) Greater Expectations Forum on Twenty-First-Century Liberal Arts Education Practice (see Leskes & Miller, 2006) identified four learning outcomes that students should be developing through college: integrative

Ryan D. Padgett, Northern Kentucky University, Senior Analyst for Co-Curricular Assessment and Research. Jennifer R. Keup, University of South Carolina, Director, National Resource Center for The First-Year Experience and Students in Transition. Ernest T. Pascarella, The University of Iowa, Mary Louise Petersen Professor of Higher Education. Correspondence concerning this article should be addressed to Padgett at [email protected].

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 133 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars learning, inquiry learning, global learning, and civic learning. In particular, the Greater Expectations report defines inquiry learning as a process to purposefully seek out solutions, conduct investiga- tions, gather additional knowledge, and ask questions that challenge the student to think like an expert in the field (Leskes & Miller). In other words, the student goes above and beyond the classroom requirements and becomes intentionally and personally invested in inquiry and engagement. Many educators identify first-year seminars and their structural and pedagogical characteristics as premier vehicles for students’ development, including intellectual and cognitive domains. Indeed, intentional first-year experience programming, including first-year seminars, is a vehicle for the educationally purposeful activities highlighted in the Defining Effective Educational Prac- tices project by the National Survey of Student Engagement (NSSE) (Kuh, Kinzie, Schuh, Whitt, & Associates, 2005) and documented as a means of fostering in- and out-of-classroom engagement for first-year student learning (Erickson, Peters, & Strommer, 2006). Further, AAC&U identifies first-year experiences—including seminars—as one of 10 “purposeful” and effective educational practices to facilitate students’ progress and achievement on the four key outcomes of undergraduate education noted in Greater Expectations (Leskes & Miller, 2006). Although far from a recent innovation in postsecondary education, first-year seminars have gained a strong foothold in American higher education over the past three decades and are now a commonly-used tool to enhance the transition and learning experience of new college students. Na- tional data indicated that 94% of accredited four-year colleges and universities in the United States offer a first-year seminar to at least some students, and more than half offer a first-year seminar to 90% or more of their first-year students (Policy Center on the First-Year of College, 2002; Padgett & Keup, 2011). Given the pervasiveness of first-year seminars and their purported connection with inquiry, cognitive, and intellectual outcomes, the purpose of the current study was to investigate the connection between participation in first-year seminars and a measurable cognitive outcome. This study seeks to fill a gap in the literature base on first-year seminars with respect to its influence on outcomes other than retention, grades, student engagement, and satisfaction. The current work also represents significant methodological advancements from previous studies. First, it uses a national longitudinal dataset, which increases the generalizability of the findings over previous single-institution studies of first-year seminars. Second, it allows for the consideration of indirect effects of first-year seminars on student outcomes via educational experiences in the classroom. Thus, the current study not only identifies if there is an influence of first-year seminars on cognitive measures but also charts the pathway of impact on this important student outcome.

Literature Review Indeed, the first-year seminar may be the most researched course in the undergraduate curriculum (summarized in Cuseo, 2009; Koch, Foote, Hinkle, Keup, & Pistilli, 2007; Pascarella & Terenzini, 2005; Tobolowsky, Cox, & Wagner, 2005). This research has established the first-year seminar as one of the most important instructional vehicles for achieving the learning and devel-

134 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars opmental objectives of undergraduate education in the United States. Most of these studies have examined the impact of the course on retention, persistence to graduation, and academic performance, yielding nearly uniform results of a positive impact of first-year seminars on these student outcomes (e.g., Barefoot, Warnock, Dickenson, Richardson, & Roberts, 1998; Fidler, 1991; Fidler & Moore, 1996; Starke, Harth, & Sirianni, 2001; Tinto, 1993). The benefits of first-year seminar participation on these key educational outcomes appear to be consistent across gender, residential and commuter students, race and ethnicity, and major (Boudreau & Kromrey, 1994; Starke et al., 2001; Sidle & McReynolds, 1999). The majority of research on first-year seminars has focused on the influence of these courses on retention and academic performance measures. Summing up this research, Pascarella and Teren- zini (2005) state that first-year seminar participation has significant and substantial “positive effects on a student’s successful transition to college and the likelihood of persistence into the second year . . . and on a considerable array of other college experiences known to be related directly and indirectly to bachelor’s degree completion” (p. 403). Despite the voluminous literature base on the characteristics and impact of first-year seminars on students’ adjustment, experience, and development, there are still gaps in the previous research on this widely used curricular intervention. Most notably, there is a dearth of research findings on the relationship between first-year seminars and the intellectual development of first-year students. Although there are some accounts of assessment and institutional effectiveness in this domain (Barefoot et al., 1998; Tobolowsky et al., 2005), empirical evidence connecting participation in first-year seminars to outcomes such as critical thinking, cognitive complexity, and multicultural maturity are generally missing in the ongoing agenda of research on this topic. This omission from the research is especially surprising given that national data on first-year seminar characteristics indicate that critical thinking is one of the most important seminar topics for approximately one-third of institutions and a much higher pri- ority for four-year campuses, smaller colleges and universities, private institutions, and especially in academic-oriented first-year seminars (Keup & Petschauer, 2011; Padgett & Keup, 2011). Among the reasons that first-year seminars offer significant learning opportunities are the structural and instructional characteristics that are not only common among these courses but that also set them apart from the rest of undergraduate classes. First-year seminars, more specifically, tend to be smaller in size than most other lower-division courses, which fosters student-faculty interaction and peer relationships; contain educationally purposeful curricular practices such as service learning, problem-based learning, interdisciplinary instruction, and experiential education; include intentional linkages with other effective interventions such as learning communities and residential life programs; and feature engaging pedagogies, which are comprised of “a variety of teaching methods, meaningful discussion of homework, challenging assignments, productive use of class time, and encouragement for students to speak in class and work together” (Swing, 2002, para. 3) (Erickson, Peters, & Strommer, 2006; Keup & Petschauer, 2011; Kuh et al, 2005; Leskes & Miller, 2006; Light, 2001; Padgett & Keup, 2011). Given that these characteristics have been widely

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 135 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars documented as beneficial to student learning and development, first-year seminars hold great -po tential to create positive gains in cognitive complexity, critical thinking, and reflective judgment as identified in numerous theories of student intellectual development (summarized in Erickson, Pe- ters, & Strommer, 2006; Evans, Forney, & Guido-DiBrito, 1998; Pascarella & Terenzini, 1991, 2005; Skipper, 2005), a point that is in need of empirical testing. The current study estimates the influence of participation in first-year seminars on students’ need for cognition. Defined as an individual’s motivation and desire to purposefully seek out, engage, and enjoy cognitive activities (Cacioppo, Petty, Feinstein, & Jarvis, 1996), need for cognition has been described more specifically as a “conceptual proxy for life-long learning orientations” (Mayhew, Wolniak, & Pascarella, 2008). Primarily measured and utilized within educational research, need for cognition is viewed as a cognitive motivation rooted in motivation theory (Cacioppo et al., 1996; Mayhew et al., 2008). Applied within higher education, need for cognition may assess students’ intrinsic motivation to engage in challenging academic or nonacademic activities. Using a longitudinal comparative research design to estimate the influence of undergraduate curricular conditions and purposeful educational practices on students’ life-long learning orientations, Mayhew et al. (2008) found growth in need for cognition can be attributed to faculty instruction focused on perspective taking, reflection, and active learning as well as students’ interactions with diverse peers. These curricular conditions often align with the mission and purpose of first- year seminars and validate the importance of examining the influence first-year seminars have on incoming students’ life-long learning orientations. Although the gap in research on more complex outcomes represents the primary limitation of the body of work on first-year seminars, other aspects of the research are also in need of attention particularly with respect to the consideration of context and means of impact of these courses. For example, previous studies on these seminars tended to rely upon single institution studies, which significantly reduce the generalizability and application of these research findings. Further, many studies that examine the effectiveness of first-year seminars included few or no controls of potentially confounding background characteristics or precollege experiences, thereby introducing bias to the examination of the effect of the seminar on the outcome of interest. Finally, the influence of first-year seminars on student learning outcomes has not been fully explored. An examination of both their direct effects on the outcomes of interest as well as their indirect effects on known cor- relates of student development, achievement, and success would greatly add to our understanding of the impact of first-year seminars.

Purpose The current study advances the research agenda on first-year seminars by addressing many of the limitations within the literature on this common curricular intervention. Most significantly, it uses an outcome measure that is related to students’ inquiry and life-long learning during the first-year of college. Further, it more intentionally explores the potential influence of first-year

136 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars seminars on this outcome measure, including the consideration of indirect as well as direct effects. Finally, it uses the results from a large, pretest/posttest longitudinal, national study of student data collected as part of the Wabash National Study of Liberal Arts Education (WNS) in order to maximize the utility and applicability of the findings for numerous institutional settings and educational contexts. College impact models (see Astin, 1993; Pascarella & Terenzini, 2005) serve as a methodological guide for this study in its efforts to address these limitations of previous research on the topic of first-year seminars. In other words, the current study acknowledges that student change and development must be contextualized to gain a full understanding of these processes. Both structural and programmatic characteristics, whether they are intentional or serendipitous, influence the learning, development, and experiences of college students. In order to fully examine the degree to which these organizational and programmatic elements of college significantly influence learning and developmental outcomes, it is necessary to account for potentially biasing background characteristics, precollege experiences, and students’ participation and engagement within the campus environment. The current study utilized college impact methodology and statistical controls to explore and identify the specific impact, and pathway of effect, of first-year seminars on students’ life-long learning orientations. The following questions guide this research:

1. What is the total effect of first-year seminars on first-year students’ need for cognition, controlling for a battery of precollege and background characteristics?

2. What are the direct and indirect effects of first-year seminars on first-year students’ need for cognition, controlling for a battery of precollege and background characteristics and college- level experiences?

Methods Institutional Sample The current analysis draws from a sample of student-level data from 48 college and universities that participated in the WNS, a longitudinal study investigating the effects of the liberal arts experiences on a series of cognitive and psychosocial outcomes associated with undergraduate education (Center of Inquiry, 2011). Institutions participating in the WNS represent a diverse selection of colleges and universities, varying in institutional type and control, size, selectivity, and location within the United States. Liberal arts colleges were purposefully overrepresented within the institutional sample given WNS’s primary purpose to study the impacts of liberal arts colleges and the liberal arts experience. According to the Carnegie Classification of Institutions, seven of the participating institutions were classified as research universities; nine were classified as regional, nongranting doctorate institutions; 29 were classified as liberal arts colleges; and three were clas-

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 137 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars sified as community colleges. We restricted the current analytic sample to four-year institutions because of the sufficiently low number of community college students who reported participating in a first-year seminar n( = 30).

Student Sample The current analysis includes first-year, full-time undergraduates who entered college in the fall of 2006, 2007, and 2008 and represent samples from three cohorts of the WNS. Utilizing all three cohorts afforded us the ability to increase the sample, thereby allowing us to account for a battery of control measures. Students from large, research universities were randomly selected from the first-year class to participate in the study, while the entire first-year student populations from the remaining institutions were sampled. Students were ensured in writing that any personal information or survey responses they provided would remain confidential and would not be included into their institutional records. Students from the 2006 cohort were offered a $50 stipend to participate in the fall administration and offered an additional $50 to participate in the spring follow-up. Students in the 2007 and 2008 cohorts were never offered a stipend for their participation, though some institutions offered incentives to boost institutional survey response rates. The awarding of a stipend to the 2006 cohort was the only survey administration difference between cohorts.

Data Collection In order to measure a comprehensive pretest-posttest examination of the first-year experience, the WNS data collection was conducted in two separate waves by ACT. The initial data collection took place in early fall of the students’ first-year of college via a proctored survey administration in multiple sites and lasted approximately 90–100 minutes. A survey was developed for the WNS project to collect precollege data on the study participants. This instrument was used as the first data collection point to capture student demographic information, family background characteristics, and high school and precollege experiences. Additionally, students also completed a number of cognitive, psychosocial, and personal development instruments theoretically associated with liberal arts education. The initial data collection yielded a student sample of 4,501 first- year students in the 2006 cohort, 3,375 first-year students in the 2007 cohort, and 9,628 first-year students in the 2008 cohort. The follow-up data collection was conducted in the spring of the first-year and lasted approximately two hours. Two complementary surveys—the NSSE student survey and the WNS Student Experiences—were simultaneously administered to measure college experiences, student engagement, and exposure to vetted good practices (see Pascarella et al., 2006; Pascarella, Wolniak, Cruce, & Blaich, 2004). Additionally, posttest measures on the identical cognitive, psychosocial, and personal development instruments from the fall survey administration were collected. Response rates

138 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars from the initial data collection to the follow-up saw an anticipated decline across each cohort, with a 48% attrition rate. The follow-up data were weighted to each institution’s first-year undergraduate population by sex (male or female), race (Caucasian, African American/Black, Hispanic/Latino, Asian/Pacific Islander, or other), and ACT score (or COMPASS/SAT equivalent), though it should be noted that the weighting algorithm does not adjust for nonresponse bias. We also performed a number of missing data analysis techniques to examine the patterns of missing data across the analytical model. Nearly 11% (n = 877) of students did not identify their parents’ level of income. However, we included the measure for parents’ income because prior research found evidence to suggest that socioeconomic status significantly impacts students’ need for cognition and cognitive development during the first-year of college (Padgett, Johnson, & Pascarella, 2012). To minimize the loss of data attributed to income, if the measure for parental income was blank or missing, we substituted students’ level of income. Furthermore, 783 cases were missing from institutional reported data. Aside for parents’ income and the missing institutional data, we can confidently conclude that the remaining data were missing at random and unbiased to the dependent measure (see Allison, 2002). After eliminating missing data using listwise deletion, acknowledging the precipitous drop in missing cases associated with parents’ income and institutional-level data, and the removal of two-year institutions from the analysis, 5,251 students from 45 colleges and universities remained in the analysis. Due to weighting and missing-data analyses, the institutional sample approximates national representation.

Dependent Variable To investigate first-year students’ life-long learning orientations, we utilized the need for cognition (NFC) scale, which drew from measures among the surveys administered to students during the follow-up data collection. The NFC is an 18-item scale that measures an individual’s desire to engage and seek out cognitive activity (Cacioppo et al., 1996; Cacioppo, Petty, & Kao, 1984). Psychometric tests indicate that the NFC scale is a highly reliable measure (a = 0.90). Individuals who measure high on NFC are more likely to seek, acquire, and reflect on information and processes in order to make sense of behaviors, relationships, and experiences in their daily lives (Cacioppo et al., 1996). Individuals who measure low on NFC are more likely to rely on others in order to make sense of their daily lives. The NFC scale has been positively associated with undergraduates’ ability to engage in evaluative conversations, high levels of verbal reasoning, a desire to maximize the intake of information (Cacioppo et al., 1996), and college grades (Elias & Loo- mis, 2002). The NFC scale has been negatively linked to undergraduates’ authoritarianism, need for structure, the anxiety in responding, and concern for self-presentation (Cacioppo et al., 1996).

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 139 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Independent Variables Though the magnitude of variables within the longitudinal WNS dataset present us with a variety of measures, we opted to create a more parsimonious model to more precisely measure the effect of first-year seminars on students’ development of NFC. As such, the selection of precollege and college covariates within this analyses were influenced by prior college impact studies that predicted the effects of the college experience and good practices on cognitive development (see Cruce, Wolniak, Seifert, & Pascarella, 2006; Padgett et al., 2012). The first set of controls account for student background and precollege experiences and includes measures for sex, three measures of race (Black versus White, Hispanic versus White, and Asian/Pacific Islander versus White; White was selected as the omitted category because of the higher percentage of students self-identifying as White within the sample), parental education, total income, ACT composite score, met with teachers outside of class in high school, studied with friends in high school, precollege academic motivation, degree aspirations, and the pretest measure for NFC. Controlling for the pretest measure allows us to more confidently attribute variation in the posttest measure to variations in the college experience measures, specifically the measure for participation in a first-year seminar (As- tin & Lee, 2003; Pascarella, 2006). Since the data for this analysis consisted of three separate cohorts of first-year students, two dichotomous measures (with one cohort serving as an omitted group) were included as a statistical control for variation across cohorts. To control for the institutional characteristics within this analysis, two measures of institutional type, as defined by the Carnegie Classification system, were incorporated into the model (regional versus liberal arts and research versus liberal arts). A number of college experiences were accounted for within the model, including end-of-first-year grades, campus residency, hours working on- and off-campus, and curriculum coursework (defined as liberal arts class versus other). The variable of interest within this study was participation in a first-year seminar. Students were asked if they “participated in a seminar designed specifically for first-year students at this college.” Nearly 70% of students (n = 3,639) in the analytical sample reported participating in a first-year seminar. To test the possibility of a more complex relationship between first-year seminar participation and students’ need for cognition via an indirect path, five vetted good practice factors similar to some of Chickering and Gamson’s (1987, 1991) “Seven Principles for Good Practice in Undergraduate Education” were introduced. More specifically, these measures included (a) frequency of interactions with faculty (4-variable factor; a = 0.70); (b) degree of positive peer interactions (8-variable factor; a = 0.87); (c) extent to which students’ first college year featured integrated ideas, information, and academic experiences (4-variable factor; a = 0.74); (d) overall academic challenge and effort (11-varaible factor;a = 0.65); and (e) diversity experiences (a = 0.65).

140 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Analyses

Ordinary Least Squares Regressions We conducted a series of ordinary least squares (OLS) regressions, introducing independent variables into the model in two separate blocks. The first block included a battery of precollege controls (e.g., gender, race/ethnicity, parents’ education, total income, ACT composite score, met with teachers outside of class in high school, studied with friends in high school, academic motivation, and degree aspirations) and the pretest measure for NFC. To measure the total effect of first-year seminars on NFC, a dichotomous measure of first-year seminars (e.g., participated in a first-year seminar will be compared to the base group, did not participate in a first-year seminar) was included in the model. To measure the direct effect of first-year seminars on NFC, the second block introduced a number of college-level controls and experiences, including: two controls for institutional type (e.g., regional colleges and research institutions will be compared to the base group, liberal arts colleges), college experiences that may contribute to students’ NFC (e.g., end-of-first-year grades, on- or off-campus residency, total hours worked on- or off-campus, and number of courses taken in the liberal arts), and the five vetted good practices: (a) frequency of interactions with faculty; (b) degree of positive peer interactions; (c) integrated ideas, information, and experiences; (d) academic challenge and effort; and (e) diversity experiences.

Design Effect To test if multicollinearity existed within the regression models, we measured the variance inflation factors and found no factor exceeded 2.00. This suggests that the collinearity between the independent measures is acceptable (Myers, 1990; Stevens, 2002). Because the analysis was conducted using multi-institutional, longitudinal data, we accounted for the nested nature of the data by performing statistical procedures throughout each model to control for the clustering effect. Without these proper adjustments, the analysis would have been exposed to the possibility of reduced bias in standard errors, commonly referred to as a type I error. The use of secondary data to analyze participation in a curricular program, such as first-year seminars, subjects the analysis to the possibility of selection bias because of the lack of true random assignment. Various statistical procedures—including propensity score matching, structural equation modeling, regression discontinuity, and analysis of covariance—attempt to accurately measure participation in a curricular program, although researchers have yet to identify a universally accepted method. A recent study that examined the effect of volunteering on first-year students’ psychological well-being using (a) propensity score matching, (b) unweighted multivariate OLS regressions, and (c) weighted multivariate OLS regressions that controlled for the clustering effect, found minimal differences in the magnitude of coefficients for volunteering across the- mod

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 141 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars els (Padgett, Salisbury, An, & Pascarella, 2010). This finding is consistent with emerging evidence (Shadish, Clark, & Steiner, 2008; Steiner, Cook, Shadish, & Clark, in press) that suggest the design of the study and its constructs, a large sample size, incorporating extensive pretreatment measures into the analytical model, along with a pretest measure can statistically reduce selection bias. The current analyses utilized weighted, multivariate OLS regression that controlled for the clustering effect and pretest measure as a means of minimizing selection bias in the study.

Results The total and direct effects from the model are presented in Table 1. We standardized all continuous dependent and independent measures so the coefficients represent effect sizes. The total effects model estimates the effects of first-year seminars on NFC, controlling for a wide- ar ray of student background and precollege characteristics, including students’ precollege academic ability and the pretest measure for NFC. Net of background and precollege characteristics, there was a small to moderate significant positive total effect for first-year seminars on NFCB ( = 0.08; p < 0.01), which was similar to the degree of impact of the other statistically significant variables on the outcome, outside of the large effect of the pretest, which suggests participation in a first-year seminar increased students’ end-of-first-year NFC score by 0.08 standard deviations, holding all other variables constant. The direct effects model introduced a number of college-level controls and experiences, including a number of good practice measures. When these measures were introduced, the first-year seminar coefficient was reduced to B = 0.03 and became nonsignificant. A substantial proportion of the total effect, nearly 60%, of first-year seminars on NFC was mediated through the college experiences that were added to the model, which suggests that the difference in coefficients between the total effects model B( = 0.08; p < 0.01) and the direct effects model B( = 0.03) can be attributed to the indirect effects of first-year seminars on NFC as transmitted through college grades and experiences with high-quality educational practices. We used the Sobel test to determine the statistical significance of the indirect effects (see Preacher & Leonardelli, 2001). Of this total indirect effect (B = 0.05), nearly half (B = 0.025) of the effect of first-year seminars on NFC was mediated through the good practice measuring the extent to which students’ first college year featured an integration of ideas, information, and academic experiences, which was statistically significant (p < 0.001) (see Table 2). The other half of the mediating effect was attributable to the combination of end-of-first-year grades, academic challenge and effort, and diversity experiences. Only the effect of the good practice measuring academic challenge and effort (B = 0.008) was statistically reliable (p < 0.05). Estimating the mediating effects within this study was useful in explaining why first-year seminars are important to students’ development of need for cognition. The mediating effects suggest that participation in first-year seminars

142 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Table 1 Standardized Total and Direct Effects of First-Year Seminars, First-Year Experiences, and Good Practices on Need for Cognition Using the Wabash National Study of Liberal Arts Education (n = 5,251)

Variables Total SE Direct SE

Background Characteristics / Precollege Experiences Male 0.03 0.03 0.07* 0.03 Race - Black -0.01 0.06 0.02 0.06 Race - Hispanic 0.10 0.06 0.10 0.05 Race - Asian -0.11** 0.03 -0.08* 0.04 Parents’ Total Education 0.02 0.01 0.01 0.01 Total Income 0.00 0.02 0.00 0.02 ACT Composite Score 0.10*** 0.02 0.08*** 0.02 Teacher Interaction in High School -0.01 0.04 -0.03 0.04 Studied with Friends in High School 0.02 0.02 0.00 0.02 Academic Motivation 0.08*** 0.01 0.04** 0.01 Degree Aspirations -0.01 0.02 -0.02 0.02 Pretest Need for Cognition 0.65*** 0.02 0.61*** 0.02

Institutional Characteristics Institutional Type - Regional College -0.02 0.05 Institutional Type - Research University -0.08* 0.03

College Experiences College Grades 0.07*** 0.01 Live On-Campus 0.06 0.06 Hours Worked per Week in College 0.01 0.01 Courses Taken in the Liberal Arts 0.02* 0.01 Participated in First-Year Seminar 0.08** 0.03 0.03 0.03

(Table 1 continued on next page)

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 143 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Table 1 continued Standardized Total and Direct Effects of First-Year Seminars, First-Year Experiences, and Good Practices on Need for Cognition Using the Wabash National Study of Liberal Arts Education (n = 5,251)

Variables Total SE Direct SE

Good Practices Frequency of Interactions with Faculty -0.03 0.02 Degree of Positive Peer Interactions -0.01 0.02 Integrated Ideas, Information, and Experiences 0.13*** 0.02 Academic Challenge 0.05** 0.02 Diversity Experiences 0.04 0.02

R2 0.56 0.59

* p < 0.05, ** p < 0.01, *** p < 0.001

Table 2 Standardized Indirect Effects on Need for Cognition of First-Year Seminars Using the Wabash National Study of Liberal Arts Education (n = 5,251)

Effect Mediated Through Estimated Indirect Effect

Integrated Ideas, Information, and Experiences 0.025*** (0.006) Academic Challenge and Effort 0.008* (0.004)

* p < 0.05, ** p < 0.01, *** p < 0.001 Standard errors are in parentheses.

significantly increases the likelihood of a first college year characterized by the integration of ideas, information, and experiences as well as academic challenge and effort, and that these two good practices, in turn, enhance first-year growth in need for cognition.

144 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Discussion This study simultaneously begins to fill a major gap in the literature and addresses a number of significant limitations in the body of work examining the impact of first-year seminars. Addi- tionally—and to the best of our knowledge—this is among the first studies to examine the effect of first-year seminars on undergraduates’ life-long learning orientations. This study highlights several insights to understanding how first-year seminars influence NFC and the potential impact they may have across college campuses. In response to the first research question, we found that participation in a first-year seminar significantly increased students’ NFC. This finding suggests that first-year seminars enhance student development across a complex measure of overall motivation to inquire. Although the primary variable of interest is specific to first-year seminars, it is important to note the outcome of students’ need for cognition is a broader measure, thus suggesting that the impact of first-year seminar participation goes beyond students’ experience in that specific course and generalizes to other academic and cocurricular learning experiences. With regard to the second question, we identified a pair of significant indirect effects that suggest first-year seminars are fostering meaningful learning objectives that further boost students’ need for cognition. Though these effects were small, the directionality of each effect provides additional evidence that first-year seminars can positively impact student development in ways that were untested. The indirect effects suggest that participation in first-year seminars significantly increase the likelihood of integrating ideas, information, and experiences as well as academic challenge and effort. Though first-year seminars are often viewed as a way to increase faculty and peer interactions and academic engagement (see Hopkins, 1988; Keup & Barefoot, 2005; Kuh, 2005; Maisto & Tammi, 1991; Starke et al., 2001), these findings legitimize first-year seminars as a vehicle for enhancing students’ integration of ideas, information, and experiences as well as academic challenge and effort. One of the primary arguments against first-year seminars is the supposed lack of rigor. This mediating effect contributes meaningful evidence to that debate and suggests that first-year seminars that are academically challenging have greater benefits for students’ life-long learning orientations. The indirect nature of the effect of first-year seminars on the outcome in this study suggests an alternate pathway of impact and the importance of identifying and measuring the intermediate effects of first-year seminars on larger student learning outcomes. In other words, first-year seminars—when intentionally designed with a purposeful curriculum program and pedagogical ap- proach that enforce students’ inquiry—may be affecting student development across additional cognitive and psychosocial measures. Understanding the pathways of impact that are illustrated in the current study can help practitioners determine reasonable and achievable outcomes for first- year seminars and begin to uncover how these courses are gateways to supporting student development in meaningful outcomes during the pivotal first year.

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 145 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Implications Previous research has yielded almost universally positive findings for the impact of first-year seminars on student persistence, performance, and engagement outcomes (e.g., Barefoot et al., 1998; Fidler, 1991; Fidler & Moore, 1996; Starke, Harth, & Sirianni, 2001; Tinto, 1993). However, there had been a dearth of empirical evidence connecting these common curricular interventions to cognitive and inquiry-based learning outcomes. This study provides empirical evidence that first-year seminars are a legitimate curricular component that impacts not only persistence and student success but also integral student development components, which suggests the first-year seminar is a per- vasive tool in the overall development of a holistic student and citizen. The most meaningful result from the current study is the provision of support for first-year seminars as valued instructional vehicles for achieving complex intellectual developmental objectives for undergraduates. Given the current economic climate of higher education, interventions with a positive impact on a wide range of cognitive and affective outcomes can be justified as not only having a high-return on investment but also being worthy of ongoing institutional funding. Findings from the current study can be used by campus leaders and higher education decision-makers as evidence to support the implementation and ongoing investment in first-year seminars as a flexible and impactful practice for first-year students. In addition to lobbying for the seminar generally, these findings can also be used to engage certain campus constituencies in the discussion and administration of first-year seminars. His- torically, these courses were the domain of student affairs (Hunter & Linder, 2005) but have now emerged as a highly horizontal structure in higher education that model effective collaboration between student and academic affairs (Keeling, Underhile, & Wall, 2007). Studies about the relationship between first-year seminars and outcomes related to life-long learning orientations have the potential to create buy-in, nurture engagement, and increase involvement from faculty and academic administrators in support of this administrative partnership. The connection of first- year seminars with integrative learning, which is defined within this study as the synthesis and connection students make between curricular and cocurricular activities, highlight the importance of involvement from student affairs personnel in the successful administration of the course. These results help attract new populations of students and enhance the fit of these courses with a broader range of institutional, college, or departmental goals. The results of this study illustrate the importance for practitioners to consider a wider range of student development outcomes when designing, delivering, and assessing this important curricular intervention for new students. These research findings also provide significant guidance for how to shape first-year seminars both structurally and instructionally. The current study indicates that these seminars are, in fact, empirically connected to cognitive outcomes and, thus, can help refine learning outcomes and program goals for first-year seminars at the institutional level to include such measures. The findings from the current study show that simply implementing a first-year seminar does not yield

146 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars guaranteed positive outcomes. Nearly half of the effect of first-year seminars on students’ need for cognition was mediated through the integration of ideas, information, and academic experiences. Academic challenge was a significant predictor of the outcome for the current study. These results should provide encouragement to faculty and staff who oversee and teach in these programs to capitalize on course pedagogies, administration, and structures that facilitate academic integration and challenge in service of life-long learning orientations among students. First-year seminar instructor recruitment and training should feature these aspects of course development and deliv- ery. Given the impact integrative learning has on NFC, first-year seminar administrators could uti- lize innovative techniques to tie the curriculum with other impactful cocurricular activities (e.g., service learning, undergraduate research, etc.). This study also has several implications for future research. First, the use of a national, longitudinal dataset allowed for a broader view of the relationships between first-year seminars and need for cognition net of institutional factors as well as student-level characteristics. Thus, these findings are generalizable and suggest a greater need for future research on first-year seminars and other high-impact practices using national data. Second, the indirect effects found within the current study indicate that future research and assessment on first-year seminars should always explore for the indirect pathway of effect on student learning outcomes in addition to direct impact. Third, to get a comprehensive understanding of the impact first-year seminars are having on life- long orientation, practitioners must evaluate the seminar outside the normal metrics of student success (i.e., persistence, grades, and satisfaction) and consider a wider range of student development and learning outcomes. Finally, the current study provides a methodological template for future research that examines the connection between other AAC&U identified high-impact practices—undergraduate research, service learning, learning communities, capstone courses—and cognitive development among students.

References Allison, P. D. (2002). Missing data. Thousand Oaks, CA: SAGE. Astin, A. W. (1993). What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass. Astin, A. W., & Lee, J. J. (2003). How risky are one-shot cross-sectional assessments of undergraduate students? Research in Higher Education, 44(6), 657–672. Barefoot, B. O., Warnock, C. L., Dickenson, M. P., Richardson, S. E., & Roberts, M. R. (Eds.) (1998). Exploring the evidence: Reporting research on first-year seminars (Vol. II) (Monograph No. 25). Co- lumbia, SC: University of South Carolina, National Resource Center for The First-Year Ex- perience and Students in Transition. Cacioppo, J., Petty, R. E., Feinstein, J., & Jarvis, W. (1996). Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological Bulletin, 119(2), 197–253.

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 147 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48(3), 306–307. Center of Inquiry. (2011). Wabash National Study 2006–2009 [Wabash College]. Retrieved from http:// www.liberalarts.wabash.edu/study-overview Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3–7. Chickering, A. W., & Gamson, Z. F. (1991). Applying the seven principles for good practice in higher education. New Directions for Teaching and Learning, no. 47. San Francisco, CA: Jossey-Bass. Cruce, T. M., Wolniak, G. C., Seifert, T. A., & Pascarella, E. T. (2006). Impacts of good practices on cognitive development, learning orientations, and graduate degree plans during the first year of college. Journal of College Student Development, 47(4), 365–383. Cuseo, J. B. (2009). The empirical case for the first-year seminar: Course impact on student retention and academic achievement. E-Source for College Student Transitions, 6(6), 5–7. Elias, S., & Loomis, R. (2002). Utilizing need for cognition and perceived self-efficacy to predict academic performance. Journal of Applied Social Psychology, 32(8), 1687–1717. Erickson, B. L., Peters, C. B., & Strommer, D. W. (2006). Teaching first-year college students. San Fran- cisco, CA: Jossey-Bass. Evans, N. J., Forney, D. S., & Guido-DiBrito, F. (1998). Student development in college: Theory research, and practice. San Francisco, CA: Jossey-Bass. Fidler, P. P. (1991). Relationship of freshman orientation seminars to sophomore return rates. Jour- nal of The Freshman Year Experience, 3(1), 7–38. Fidler, P. P., & Moore, P. S. (1996). A comparison of effects of campus residence and freshman seminar attendance on freshman dropout rates.Journal of The Freshman Year Experience and Students in Transition, 8(2), 7–16. Hopkins, W. H. (1988). College success: A transitional course for freshmen. Washington, DC: American As- sociation of State Colleges and Universities. Hunter, M. A., & Linder, C. W. (2005). First-year seminars. In M. L. Upcraft, J. N. Gardner, B. O. Bare- foot, & Associates, Challenging and supporting the first-year student: A handbook for improving the first year of college(pp. 275–291). San Francisco, CA: Jossey-Bass. Keeling, R. P., Underhile, R., & Wall, A. F. (2007). Horizontal and vertical structures: The dynamics of organization in higher education. Liberal Education, 93(4), 22–31. Keup, J. R., & Barefoot, B. O. (2005). Learning how to be a successful student: Exploring the impact of first-year seminars on student outcomes. Journal of The First-Year Experience & Students in Transition, 17(1), 11–47.

148 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Keup, J. R., & Petschauer, J. (2011). Designing and administering the course (Vol. 1). In The First- Year Seminar: Designing, Implementing, and Assessing Courses to Support Student Learning & Success (Series). Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition. Koch, A. K., Foote, S. M., Hinkle, S. E., Keup, J. R., & Pistilli, M. D. (Eds.) (2007). The first-year experience in American higher education: An annotated bibliography (4th ed.) (Monograph No. 3). Colum- bia, SC: University of South Carolina, National Resource Center for The First-Year Experi- ence and Students in Transition. Kuh, G. (2005). Student engagement in the first year of college. In M. L. Upcraft, J. N. Gardner, & B. O. Barefoot (Eds.), Challenging and supporting the first-year student: A handbook for improving the first year of college(pp. 86–107). San Francisco, CA: Jossey-Bass. Kuh, G. D., Kinzie, J. I., Schuh, J. H., Whitt, E. J., & Associates. (2005). Student success in college: Creating conditions that matter. San Francisco, CA: Jossey-Bass. Leskes, A., & Miller, R. (2006). Purposeful pathways: Helping students achieve key learning outcomes. Wash- ington, DC: Association of American Colleges & Universities. Light, R. J. (2001). Making the most of College: Students speak their minds. Cambridge, MA: Har- vard University Press. Maisto, A. A., & Tammi, M. W. (1991). The effect of a content-based freshman seminar on academic and social integration. Journal of The Freshman Year Experience, 3(2), 29–48. Mayhew, M. J., Wolniak, G. C., & Pascarella, E. T. (2008). How educational practices affect the development of life-long learning orientations in traditionally-aged undergraduate students. Research in Higher Education, 49(4), 337–356. Myers, R. (1990). Classical and modern regression with applications (2nd ed.). Boston, MA: Duxbury Press. Padgett, R. D., Johnson, M. P., & Pascarella, E. T. (2012). First-generation undergraduate students and the impacts of the first year of college: Some additional evidence. Journal of College Stu- dent Development, 53(2), 243–266. Padgett, R. D., & Keup, J. R. (2011). 2009 National Survey of First-Year Seminars: Ongoing efforts to support students in transition (Research Reports on College Transitions No. 2). Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition. Padgett, R. D., Salisbury, M. H., An, B. P., & Pascarella, E. T. (2010). Required, practical, or unnec- essary? An examination and demonstration of propensity score matching using longitudinal secondary data. New Directions for Institutional Research—Assessment Supplement (pp. 29–42). San Francisco, CA: Jossey-Bass.

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 149 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Pascarella, E. T. (2006). How college affects students: Ten directions for future research. Journal of College Student Development, 47(5), 508–520. Pascarella, E. T., Cruce, T., Umbach, P. D., Wolniak, G. C., Kuh, G. D., Carini, R. M. et al. (2006). Institu- tional selectivity and good practices in undergraduate education: How strong is the link? The Journal of Higher Education, 77(2), 251–285. Pascarella, E. T., & Terenzini, P. T. (1991). How college affects students. San Francisco, CA: Jossey-Bass. Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research (Vol. 2). San Francisco, CA: Jossey-Bass. Pascarella, E.T., Wolniak, G. C., Cruce, T. M., & Blaich, C. F. (2004). Do liberal arts colleges really fos- ter good practices in undergraduate education? Journal of College Student Development, 45(1), 57–74. Policy Center on the First Year of College. (2002). Second National Survey of First Year Academic 2002. Retrieved from http://www.sc.edu/fye/resources/assessment/ essays/Swing-8.28.02.html Preacher, K., & Leonardelli, G. (2001). Calculation of the Sobel Test: An interactive calculation tool for mediated tests. Retrieved from http://people.ku.edu/~preacher/sobel/sobel.htm Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random and nonrandom assignments. Journal of the American Statistical Association, 103(484), 1334–1356. Sidle, M. W., & McReynolds, J. (1999). The freshman year experience: Student retention and student success. NASPA Journal, 36(4), 288–300. Skipper, T. L. (2005). Student development in the first college Year: A primer for college educators. Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition. Starke, M. C., Harth, M., & Sirianni, F. (2001). Retention, bonding, and academic achievement: Suc- cess of a first-year seminar.Journal of the First-Year Experience & Students in Transition, 13(2), 7–36. Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (in press). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods. Stevens, J. P. (2002). Applied multivariate statistics for the social sciences (4th ed.). Mahwah. NJ: Erlbaum. Swing, R. L. (2002). The impact of engaging pedagogy on first-year seminars (Policy Center on the First Year of College Report). Retrieved from http://www.sc.edu/fye/resources/ assessment/essays/Swing-8.28.02.html Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). Chicago, IL: University of Chicago Press.

150 doi:10.1515/jsarp-2013-0011 http://journals.naspa.org/jsarp © NASPA 2013 JSARP 2013, 50(2) Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM Impact of First-Year Seminars

Tobolowsky, B. F., Cox, B. E., & Wagner, M. T. (2005). Exploring the evidence: Reporting research on first- year seminars (Vol. III) (Monograph No. 42 ed.). Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition.

JSARP 2013, 50(2) © NASPA 2013 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2013-0011 151 Brought to you by | Wesleyan University Authenticated | 129.133.46.167 Download Date | 8/7/13 8:16 PM