Division of Continuing Education, Harvard University

Distance Education Evaluation Division of Continuing Education, Harvard University

ACKNOWLEDGEMENTS...... 2

KEY FINDINGS AND RECOMMENDATIONS...... 3 STRUCTURE OF THE REPORT...... 4 CHAPTER 1: UNDERSTANDING DISTANCE EDUCATION...... 6

INTRODUCTION...... 6 DISTANCE LEARNING AT DCE...... 6 RESEARCH STRATEGY...... 7 STUDY DESIGN...... 8 Student Survey...... 8 Faculty Interviews...... 10 CHAPTER 2: STUDENT EXPERIENCE WITH DISTANCE COURSES...... 13

INTRODUCTION...... 13 RESULTS...... 13 The Demand for Adult Education at DCE...... 14 THE DEMAND FOR SPECIFIC COURSES...... 15 THE DEMAND FOR DISTANCE COURSES...... 17 Classroom plus Internet...... 18 Satisfaction and Experience with Distance Capabilities...... 23 STUDENT SURVEY FOLLOW-UP...... 28 CHAPTER 3: THE VIEWS OF THE FACULTY...... 38

INTRODUCTION...... 38 GOING DISTANCE...... 39 Why do it?...... 39 Advantages and Disadvantages of Distance...... 40 Readiness to teach the course...... 42 THE PRACTICE OF DISTANCE...... 42 The impact of distance on teaching performances...... 42 Attending to Students and the Role of the TF...... 46 The Creation of Community...... 47 CONCLUSIONS...... 47 APPENDIX A: STUDENT QUESTIONNAIRE...... 49

APPENDIX B: INTERVIEW GUIDE FOR FACULTY...... 58

March 2002 1 Draft -– Do not copy or circulate 1 Distance Education Evaluation Division of Continuing Education, Harvard University

Acknowledgements

This report was sponsored by the Harvard University Division of Continuing Education (DCE). We are particularly grateful to Len Evenchik, Director of Distance and Innovative Education at DCE, who took the initiative to have us carry out this evaluation. Throughout all the phases of our work, Len offered unflagging interest, support, and guidance. This research was supported generously by the Provost’s Fund for Innovation in Instructional Technology and Distance Education, and we are thankful to Elizabeth Hess for helping us to frame the original proposal. Our report has benefited from the contributions of many individuals at DCE: Dodge Fernald, who served as faculty advisor, Mary Higgins, Henry Leitner, James Peregrino, Tim Kelly, Susan McGee, Colin Kegler, Brenda Mahoney, Jan Jackson, and others. We also wish to acknowledge the guidance and support provided by Dean Michael Shinagel, the careful editing of Wayne Ishikawa, and the production help from the Promotions Office. We often asked DCE staff to accommodate our requests, and they did so in a spirit of collegiality. We thank the i-Commons group for granting us the opportunity to present a preliminary version of this work, and offering useful feedback. We are grateful also to the DCE faculty who gave us their time, and to the students who voluntarily provided us with data and insights.

March 2002 2 Draft -– Do not copy or circulate Distance Education Evaluation Division of Continuing Education, Harvard University

Executive Summary

The explosion of distance education at the post-secondary school level in the United States has captured the attention of faculty, students, university administrators, and the corporate community. As technological advances that improve course delivery over the Internet coincide with labor market changes and increased demand for learning opportunities, questions about the future of distance education are foremost in the minds of many academic planners and policy makers. For the past six years, Harvard’s Division of Continuing Education (DCE) has made a small number of its courses available to local and distance students over the Internet. In order to assist in its planning and strategy, DCE commissioned an evaluation of its distance offerings that attempted to address the following issues:  Should the program expand, and if so, how fast?  Why do students choose DCE for distance courses?  How does distance education change the teaching and learning that goes on at DCE? This report details the first phase of work in a multi-year process of strengthening the distance program by developing a greater capacity to create innovations in distance learning and to incorporate evaluation. It presents the findings of research carried out in the spring 2001 term, grounding them in an understanding of contemporary debates and management models related to distance learning.

Key Findings and Recommendations

The results of our study support a growth strategy for DCE’s distance education program. Not only did the courses receive high overall marks but both students and faculty were also enthusiastic in their support for distance courses. In addition, distance courses appear to generate increased demand for the distance modality in the future: once students take a course with a distance component, they are more likely to want that option again. We believe that the strategy for growing the program should include:  Focus growth on ‘hybrid’ courses, those that offer both live classroom meetings and Internet video availability.  Consider expanding foundation courses in the degree and certificate programs.  Courses with a technology focus or content are an appropriate area for distance growth.

March 2002 3 Draft -– Do not copy or circulate Distance Education Evaluation Division of Continuing Education, Harvard University

 Develop incentives for faculty members to put their courses online  Distance courses should have a one- to two-year rollout.  Evaluation and adoption of some of the Harvard iCommons tools.  Continued study and evaluation before growing the number of Internet-only courses.

Students chose DCE distance courses for a variety of reasons. Local students are attracted by the convenience; not always having to attend class allows them to combine hectic work and family lives with being able to pursue further education. The Harvard University image and reputation is also important for the distance program. Students from outside the Boston area are attracted to DCE in part because of the Harvard name and reputation. The farther from campus, the more important the Harvard brand. Given what we have learned, we recommend:  Different marketing strategies for local and non-local students.  Demands for education may change, but DCE must maintain its standards and academic focus.  Any rise in fees should be done with an overall price increase, not a targeted increase in the distance courses.

Teaching and learning at DCE have changed in the distance courses. Faculty and students alike are experimenting and struggling with issues of how to create communities in the classroom and online. Teaching styles are changing in some cases and the nature of teaching is also changing. Given these changes we suggest:  Instituting some differential but not preferential treatment of distance.  Better training for course personnel.  Help faculty adapt teaching styles to both classroom and distance audience.  Continued study to understand better how different student learning styles interact with technologies. .

Structure of the Report

This report begins with a discussion of the distance education program at DCE and highlights some of the important issues raised in the literature that guided our research. Chapter Two presents the results of the survey and follow-up interviews we administered to students enrolled

March 2002 4 Draft -– Do not copy or circulate Distance Education Evaluation Division of Continuing Education, Harvard University in distance courses at DCE. Chapter Three analyzes the structured interviews conducted with faculty members in the distance program. Chapter Four offers our recommendations to DCE about where we see potential for growth and how to focus on teaching and learning practices that will enhance the distance program.

March 2002 5 Draft -– Do not copy or circulate Distance Education Evaluation Division of Continuing Education, Harvard University Chapter 1: Understanding Distance Education Introduction As technological advances that allow for better mechanisms of course delivery over distance coincide with changes in the labor market, changes in the type and number of students and an increased demand for learning opportunities, questions about the future structure and provision of higher education are foremost in the minds of many academic planners and policy makers.

For several years, Harvard’s Division of Continuing Education (DCE) has made a small number of its courses available to local and distance students over the Internet. In order to assist in its planning and strategy development for distance education as it looks ahead, DCE felt it an opportune time to undertake an evaluation of its current offerings. In particular DCE wanted to focus on: whether the program should expand and at what pace, to determine why students choose DCE for distance courses, and to understand how the teaching and learning at DCE changes as the result of distance education. This report details the findings of our evaluation, grounding them in an understanding of contemporary debates about pedagogical issues and management models related to distance learning. It also offers suggestions and recommendations for the future of distance education at DCE.

Distance Learning at DCE In order to contextualize the research presented here, we start with a brief overview of distance education at DCE. In 2000–01 there were 25 distance education courses with more than 2,200 students, of whom approximately 10 to 15 percent were taking the classes exclusively at a distance either because they were living far from Harvard or because their schedule did not permit attendance in class. (In 2002, 32 courses had been made available via the Internet.) Course lectures along with other course-related material can be viewed on video over the Internet by students anywhere in the world. Registered students, including those in the Boston area, have the option of attending lectures when they are given on campus or watching them online. In some cases, courses are available online only. Students view lectures on the Internet through streaming video (video that is being fed to the user as it is being viewed.) Further information regarding the Distance Education program can be found on the Distance Education website; http://www.extension.harvard.edu/distanceed.

All students registered in a Distance Education course have access to the videos, regardless of whether they attend live lectures or not. Lecture videos are made available 48 hours after the live lecture and they remain online for the duration of the semester. DCE has a policy of not videotaping the faces of students present in class. This means that the distance students only hear the voices of their peers. Outside the classroom, much of the communication between teaching staff and students takes place via e-mail and the course website. This is true for both local and distance students. Most courses have a bulletin board or chat room to foster dialogue among students and teaching staff. Some instructors are starting to give special assignments to the distance only students, such as requiring them to post on the bulletin boards. Because students access live lectures asynchronously, they cannot participate in this more interactive arena of the course.

Chapter 1 6 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University

Distance Education and local students follow the same schedule of due dates for assignments and exams. The lag time between the actual lecture and the availability of the online video is taken into account when determining these dates. In the event that a course has an exam, arrangements can be made at a local school to have the exam proctored to the student. Because the Extension School does not distinguish between on-campus and distance students, the transcript does not specify if a course is completed via distance education or not.

Because there are no distance education courses per se, instructors have to propose a course to the Extension School and have it accepted as part of the normal course selection process that takes place each spring. If it is accepted, it might be added to the distance education offerings for the following year. At times the Extension School solicits the courses, at times instructors propose to teach in that fashion. Over the past several years, DCE articulated the following principles to guide the development of the distance education component:  There are no special noncredit or 1- or 2-unit “distance only” courses. Most Extension School courses are worth 4 units; this implies that all the courses meet a certain academic level. Any new instructional technology (such as DVDs) or new mode of delivering a course (such as the Internet) should not change this academic structure or negatively affect academic rigor. In other words, a course via the Internet should correspond to a regular face-to-face class presented in a Harvard lecture hall.

 All students are “the same.” Except for the obvious differences (that a distance student does not attend lectures on campus), students at a distance should be treated in the same manner as local students. When it comes to registration, student/teacher communication, grading, and student evaluation no distinction is made between local and distance students. In fact, at the beginning of the term instructors do not know who in the class is a distance student and who is not. Students who take distance education courses complete the same coursework and receive the same credit as students who take the courses locally.

 Technology should not take precedence over learning and teaching. An effort has to be made that technology does not dictate the pedagogical approach. Given that all the courses were large lecture type courses, the steaming video technology seemed to work well.

Research Strategy The central task faced by this study was to respond to the question of what the pace and direction of growth of distance education at DCE should be. Although the researchers examined the trends in distance education nationally as an important contextual factor that DCE faces in the development of its own program, we also assumed that its actions will be bounded by the need to fulfill and stay faithful to its core mission. That mission is to extend the teaching and research of the Faculty of Arts and Sciences to the local community. This concern with DCE’s mission also prompted us to focus on issues of how distance education may change the teaching

Chapter 1 7 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University and learning that occurs at DCE, and how ongoing research on the distance education offered by DCE can provide useful tools and feedback for enhancing the experience of students at both DCE and the Faculty of Arts and Sciences.

Study Design The research entailed a survey of DCE students enrolled in the distance courses and structured interviews with faculty members teaching in the distance program. In both cases, the questions were designed with the central themes of the study in mind and drew on key issues in the current academic and educational policy literature on distance teaching and learning.

Student Survey

The survey was administered to students enrolled in two types of distance courses at DCE. One was a ‘hybrid’ model, where the classroom lecture is taped and available over the Internet within 24 to 48 hours after delivery, along with Powerpoint slides of the instructor’s notes. Students can attend lecture or not, depending on how convenient it is for them to do so. The second type of course is the ‘all-distance’ course, which is an Internet-only version of a course that was usually filmed a semester earlier. This type of course has no live lectures and all students are distance students. The courses do, however, have live weekly section meetings.

Students enrolled in 12 courses in the spring 2001 term were asked to complete the end of term survey, which had been piloted during the fall 2000 term. Two hundred thirty students of 758 completed the questionnaire, for a response rate of 33%. We also conducted a follow-up series of interviews by email and phone in order to deepen our understanding of some of the response patterns revealed by the quantitative analysis. Appendix A provides a sample of the questionnaire.

The questions posed to students on the survey reflected central concerns of the study and also drew on the current literature on various aspects of distance learning. A number of the relevant literatures are discussed briefly here. Students were asked a series of questions about their reasons for pursuing further education, why they had chosen DCE, and their reasons for taking a distance course. Answers to these questions allowed us to look both at the relative importance students assign to the availability of distance as an option when choosing a course and to inform our understanding of the particular draw of DCE as opposed to other institutions offering distance courses. The features of the distance education market often are an important contrast to the traditional higher education market. One of the central concerns for consumers is the quality of the product; not only has less information for distance programs been available but also the fact that so many programs are relatively new means that it is often hard for consumers to make informed judgments. Studies have shown that the reputation of an institution is an important factor in students’ willingness to take distance courses.1

We also developed a series of questions that reflect the broad economic and demographic changes that scholars agree are changing the landscape of higher education. Because issues of 1 Christensen, Edward W; Uzoamaka P Anakwe; Eric H Kessler. 2001. “Receptivity to Distance Learning: The Effect of Technology, Reputation, Constraints, and Learning Preferences.” Journal of Research on Computing in Education 33 (3):263-279

Chapter 1 8 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University training and lifelong learning are now viewed as central to labor market success over the life course of individuals2, we inquired about the work-related reasons for choosing courses and distance. Other sociologists have pointed to the speed-up of work and family life, particularly for women3, in the United States4. We speculated that distance courses might be especially important for those whose work and family lives are extremely demanding and so included questions about those aspects of students’ lives.

Another issue that the literature discusses is the role of learning styles on people’s performance and interest in distance learning courses. Despite the fact that many view learning style as a crucial explanatory variable on a variety of distance learning outcomes, there is a shortage of studies on it.5 One of the particularly important points for adult learners is that they are often thought to be generally characterized by a learning style that prefers face-to-face interaction over more impersonal forms, with the implication that distance programs need to take this into account when designing courses.6 We utilized a number of questions on learning styles that other researchers, such as those associated with Project Adept7, have worked with and also designed new questions to examine how students at DCE with different learning styles viewed their experience with distance education.

Closely related to learning styles is student readiness to take a distance course. Many programs assume that the ideal distance student is one who is highly motivated, self-directed, and has good time management skills.8 Being comfortable with computers is another aspect of readiness, and the assumption is that students who are familiar with technology and use computers regularly will make better distance learners. Our survey includes questions about how well students are able to organize their time and their feelings about computers that allow us to test the relationship among these individual factors and their experience and evaluation of DCE’ distance courses.

An interesting assessment of distance learning that has been undertaken in the US occurred in 1999 at the University of Illinois. There a group of faculty met over the course of the year to explore the implications of distance education for the university. Their research, available in their online report9, focused exclusively on pedagogical aspects of distance learning.

2 Hake, Barry J. 1999. “Lifelong learning in late modernity: The challenges to society, organizations, and individuals.” Adult Education Quarterly 49 (2):79-91. 3 Home Alice M. 1998. “Predicting role conflict, overload and contagion in adult women university students with families and jobs.” Adult Education Quarterly 48 (2) 85-97. 4 See for example, Hochshild, Arlie 1987. The Second Shift. NewYork: Avon Books. and Schor, Juliet. 1992. The Overworked American. New York: Basic Books. 5 Simon, Steven John. 2000. “The relationship of learning style and training method to end-user computer satisfaction and computer use: A structural equation model.” Information Technology, Learning, and Performance Journal 18, (1) 41-59.; Wang X. Christine; D Michelle Hinn, Alaina G Kanfer. 2001 “Potential of computer- supported collaborative learning for learners with different learning styles.” Journal of Research on Technology in Education 34 (1) 75-85. 6 Christensen Edward W., Uzoamaka, P Anakwe, Eric H Kessler. 2001. “Receptivity to distance learning: The effect of technology, reputation, constrains, and learning preferences.” Journal of Research on Computing in Education 33 (3) 263-279. 7 http://www.employees.csbsju.edu/tcreed/adept/ 8 See the website http://www.learninstyles.net/ 9 http://www.vpaa.uillinois.edu/tid/report/

They assumed that characteristics of good practice in teaching and learning are generic and hence apply to all contexts. They viewed good practice as encouraging student-faculty contact, cooperation among students, and active learning. Good practice gives prompt feedback, emphasizes time on task, communicates high expectations, and respects diverse talents and ways of learning.10 Ultimately the group agreed with Hansen11, that from the pedagogical perspective, the most important principle is attentiveness to students, defined as an appropriate reaction to what students do, need, and ask.

Apart from the professor, students derive much motivation from each other. 12 “Good learning, like good work, is collaborative and social, not competitive and isolated.” Working with others often increases engagement in learning. All of these aspects are part of a high quality learning experience that needs to be incorporated into online learning. Following this literature, we included survey questions about students’ relationship to faculty, to each other, and how the interactions occurred. In addition, many of the follow-up interviews focused on these issues.

Another area that the survey considered was how the students were using the technology. This was considered important in order to best respond in the future to the needs of the distance students. We examined what the primary use for the class videos was (i.e. as a way to take the class or mainly to review it), how well the technology functioned, and their overall level of satisfaction with it. Most studies of distance learning look at how technology may change the learning process but do so from the perspective of the student who is not in the classroom. This study does so as well, considering, for example, how reading bulletin boards or chat rooms facilitate virtual discussion, how posting helps learning or the utility of linked websites. Unlike much prior work, however, this study also considered the impact of distance learning on the in- class student community by examining how the presence of the camera affected students.

Faculty Interviews

Part of our research strategy also included interviews with faculty members in the distance courses. Faculty who taught DCE distance courses were also interviewed using a structured, open-ended guide. Nine instructors participated in the study and the interview guide is reproduced in Appendix B. Five of the faculty members were in the Computer Science department and four from a variety of other departments. The interviews were coded for themes and analyzed. Together with the data from the students, they help bring into focus a clear picture of the experience of distance education at DCE.

One of our central concerns in the faculty interviews was why they had decided to teach a distance course and what they viewed as the positive and negative aspects of their experience. This set of questions is related to a broader ongoing discussion in academe about the meaning of distance education and the use of the Internet for intellectual endeavors. These are extremely contentious issues at some universities. For example, at the University of Toronto faculty members went out on strike against the university’s attempts to have faculty put course materials

10 Chickering and Gamson, 1987 11 Hansen 12 Chickering and Gamson, op cit

Chapter 1 10 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University online, fearing a loss of autonomy and control over their own intellectual property.13 The issue of remuneration has also arisen for faculty members when online courses can in theory infinitely expand the number of paying students. Faculty members have made arguments in favor of profit-sharing, a step that university administrators have generally been loathe to take. The point is, many faculty members see (and administrators both welcome and fear) a sea change in the economics of higher education. We were interested to see whether any of these dynamics were present among DCE distance faculty.

We also asked professors about the process by which they developed the online versions of their courses. In particular, we were interested in being able to make recommendations to DCE about an optimal time frame for the creation of distance courses. The issue of how well developed the electronic aspects of the course had been prior to its distance version was a specific area of interest to us because we wondered whether there might be overlapping or competing learning curves in terms of computer mediated instruction versus distance instruction. Interestingly, and in contrast to most distance programs, we felt that the faculty members’ role in the development of the electronic features of the course was likely to be high because of the particular strategy of DCE to focus on the Computer Science courses and we inquired about this.

In addition, we asked about whether they had consulted with other faculty members either when deciding about doing a distance course or in developing it because we wanted to ascertain the degree to which there is any kind of community of practice that has developed at DCE around the distance course. This would include advice on design, ongoing sharing of ideas and some forum or mechanism for exchange of ideas.

The faculty interviews also focused on their interactions with distance students and what, if anything, they did to acknowledge those students. In part, we wanted to know whether DCE’s policy of treating distance students the same as their non-distance counterparts was followed in the classroom or whether faculty members had felt that differentiated treatment was necessary. We also asked professors whether they saw any differences in the needs, behavior or performance of the students. This is the reverse side of the coin: the institution may treat students similarly but distance students may act and make demands differently than local students.

We also probed the faculty members for information about the nature of the work of teaching a distance course. In the pilot study that we carried out in the semester prior to the research for this study, we were struck by how the amount and type of work seemed to change once a course was offered at a distance. In particular, the email traffic had appeared to increase greatly and the work needed to be offloaded to Teaching Fellows. In the faculty interviews, we followed up this theme with questions about the amount of work and also how their teaching performance may have changed.

13 See Noble, David F. 2001. “Digital Diploma Mills: The Automation of Higher Education” in the peer-reviewed online journal, First Monday at http://www.firstmonday.dk/issues/issue3_1/noble/index.html

Chapter 2: Student Experience with Distance Courses Introduction This chapter examines students’ opinions and experiences with distance courses at DCE. Students in 12 courses were surveyed about their reasons for taking courses, in general and specifically at Harvard, their perceptions and use of the course’s distance component, their learning styles and overall experience in the class. In addition to the survey, some students were contacted for a follow-up interview in order to get a deeper and more nuanced understanding of how the distance component met their expectations and needs for learning.

We surveyed students from nine hybrid courses (those that met in the classroom and made a video of the class sessions available over the Internet) and from three distance-only courses, where lectures were taped in an earlier semester and then rebroadcast over the Internet with support from the instructor and teaching fellows. The 12 courses were offered during the spring term of 2001. They represent all the distance course offerings of DCE that term with the exception of one course that was excluded from the survey because its mode of delivery was fundamentally different from the rest of the distance courses. Eight of the courses were in the Computer Science department, including all three distance-only courses, and the remaining ones were offered by the departments of Classics, Government, Natural Sciences, and Philosophy. The characteristics of the sample are reported in Table 2.1.

Results The findings show that there is a high degree of satisfaction with the distance component of the courses. At the same time we note that the distance features (such as the quality of the video or the ability to follow the lectures over distance) received somewhat lower ratings than the courses in general. The evaluation of distance appears to assume a trade-off between the convenience that distance education allows many adult learners with complex professional and family commitments, and the attentiveness and interaction available in the classroom. This is a trade-off that most students seem willing to accept, particularly since there are other factors that offset the negatives. For those who live outside the area, the prestige of taking Harvard courses is an important factor weighing on their decision to enroll. For them, presumably the lack of physical proximity to the instructors is outweighed by their virtual proximity. Local students, in contrast, often enjoy the best of both worlds, attending classes when they can and catching up over the Internet when they cannot. Many of them become essentially mixed-modality students, erasing the distinction between classroom and distance learners.

Table 2.1 Descriptive Characteristics of Student Survey Sample

Sample size 230 Employment Status Respondents enrolled Full time 80.7% in hybrid courses 74.2% Part time 8.3% Respondents enrolled Not currently in Computer Science employed 11.0% courses 72.2% Education level Age High School 8.3% 18-25 14.5% Associate degree 4.8% 26-35 46.1% B.A./B.S. 50.7% 36-45 23.2% M.A./M.S. 17.5% 46-55 12.7% Grad or Professional 18.8% 55+ 3.5% Respondents enrolled in Gender degree/certificate Male 63.3% program 46.1% Female 37.7% Respondents with prior Distance from DCE courses 73.7% Campus Respondents with prior Under 10 miles 44.5% distance courses 36.0% 10 to 25 miles 24.0% Course payment Out of Boston 16.6% Self-pay 56.2% area Employer pays 41.2% Out of New 11.4% England Out of USA 3.5%

The findings also suggest that there is great demand for the hybrid courses precisely because they allow local students to combine the convenience of distance with the connectedness and attentiveness of the classroom format. Distance, in a sense, creates its own demand among local students, as many who experience it are loathe to return to a classroom-only model once having taken a hybrid course. In the following sections, we present a discussion of these findings.

The Demand for Adult Education at DCE

One of the issues examined by the survey is why respondents take courses at all. Students were asked to rate their reasons for pursuing their education. In this sample, respondents stated personal enrichment as the most important reason they were taking classes, although as Table 2.2 shows, professional concerns, especially in one’s current career also were an important factor.

Table 2.2 Mean Rating of Item’s Importance for Pursuing Education* Personal Enrichment 4.3 Professional enhancement in current career 3.9 Pursue a degree 3.7 Interest in changing careers 2.8 N=227 * Respondents used a 5 point scale where 1=not very important, 5=very important

There was little variation among the responses related to other characteristics of the students, except in what department they are taking courses. Students enrolled in Computer Science courses were much more likely to list current career concerns as a reason they were taking classes (4.3 versus 2.6 on a 5 point scale) and much less likely to cite personal enrichment as important (4.1 versus 4.7). That career concerns were more important to students in Computer Science courses is not very surprising since it is a growth field in terms of employment and one where skills need constant upgrading.

These findings suggest that differentiated marketing strategies for different types of courses may be effective and in ways that are directly relevant to DCE thinking about its plans for distance education. After controlling for whether a student was enrolled in a Computer Science course, results showed that the more important personal enrichment was in the decision to pursue coursework, the more important it was that the particular course was available on the Internet in the decision to take it. In contrast, professionally driven reasons for continuing one’s studies had no relationship with how important a course’s availability on the Internet was.

General reasons for pursuing adult education are likely a contextual factor influencing the demand for certain characteristics of specific courses. As part of the strategy for attracting distance students to courses that do not have a clear link to career trajectories, DCE might capitalize on the personal enrichment aspect of the experience. Combining that with a focus on the Harvard ‘brand’ name could be particularly effective since the prestige of Harvard is also associated with personal enrichment motivations.

The Demand for Specific Courses In addition to the issue of the pursuit of education in general, the survey was designed to examine in greater detail the reasons students gave for taking the specific course in which they were enrolled. They were asked to rate several factors related to convenience (Internet availability, location and schedule), reputation (Harvard prestige, instructor or personal recommendation), requirements, and cost. Table 2.3 reports students’ responses and shows how the importance of the different factors varies by students’ place of residence. There are a number

Table 2.3 Reasons Given for Choosing DCE Courses Variation among Students’ Residence*

Overall Living Living Living mean with/in out of out of of 25 miles Boston NE or notable area USA patterns to the Convenient Schedule 3.6 3.5 3.7 4.0 Prestige of Harvard 3.5 3.3 3.7 4.5 Available over the Internet 3.3 2.9 3.8 5.0 Convenient location 3.3 3.4 3.3 2.6 Required course 3.2 3.0 3.6 3.7 Reputation of the instructor 3.1 3.0 3.2 3.7 Cost 2.8 2.7 3.0 3.1 Recommended by someone 2.4 2.4 2.6 2.1

N 228 156 38 34 * Respondents used a 5 point scale where 1=not very important, 5=very important responses. Internet accessibility is clearly more important for those living outside of the Boston area for the obvious reason that it allows such students to enroll, and for them it is the most important factor in their decision. For local students, in contrast, it is one of the least important reasons, although as we note below in the section on the importance of distance education, prior experience with distance courses tends to increase the weight students attach to having it as an option.

Regardless of where students live, the reputation of the course or the instructor seemed to carry little weight in their decision to take the course, although the one exception was found among the students who live farthest away. For them, the instructor’s reputation was a factor in their choice, suggesting that courses with high-profile faculty will be more appealing to true distance students and might be marketed accordingly. With respect to the institution’s reputation (Harvard prestige), it gained importance as distance from Cambridge increased, again suggesting different strategies for promoting DCE’s distance courses in the Boston area and outside of it.

It is also interesting to note that cost was a relatively unimportant factor for students, suggesting that the demand for DCE course is fairly inelastic with respect to price. For some students, it was not price alone that makes DCE a good value, but the implicit economic benefit that the institution’s name adds. The link between price and prestige can be seen in the comment one student made on his survey form:

“ Considering that this is HARVARD Extension School, the tuition cost is relatively ‘cheaper’ than other schools.”

As this suggests, some students do look at what other schools offer and make their decision to take DCE courses after considering different options. Slightly over twenty percent stated that they had compared other institutions before choosing DCE. Comparison shopping for distance courses was more common among students who lived farther away and those who had taken a distance course before. It was less common among students who had taken courses at DCE in the past. To some extent, people who live in the area may look less at other institutions because they see DCE as their local school; when considering potential students outside of Boston, DCE faces a more competitive market. Once DCE attracts or captures students, whether local or those at a distance, the data suggest that students stay with DCE. Distance education, however, does mean that people are more aware of, and check out, other options, presumably because the experience of having taken a distance course changes people’s perceptions of what is available to them, suggesting that to keep distance students DCE needs to be constantly aware of how to meet their needs. The next section focuses specifically on what students want from distance courses.

The Demand for Distance Courses Student data, comments, and follow-up conversations uniformly highlight how much students appreciate the convenience that distance provides in allowing them to combine work and family responsibilities with study. Table 2.4 shows what draws students to courses

Table 2.4 Mean Rating of Item’s Importance for Taking a Distance Course* Flexibility of viewing class to fit 4.2 schedule Ability to view classes over Internet 4.0 Go back and reflect on class or 3.8 discussion Not having to be physically present 3.7 Less pressured environment 2.5 N=209 * Respondents used a 5 point scale where 1=not very important, 5=very important available over distance. Although the convenience features play a prominent role in the demand for distance courses, it is also noteworthy that the pedagogical possibilities offered by distance courses are highly valued by students as well. Being able to go back and revisit what went on in the lecture and discussion is important for many students. Students noted that this helped with their comprehension of the course materials as evidenced by this comment:

“ I do not say that classroom is always better than distance learning. With very effective Internet tools like lecture-videos-online, I have benefited being able to watch the lecture again and understand things better.”

For most students, the ideal course combines both distance and classroom components. As Table 2.5 shows, the hybrid modality of classroom-plus-Internet is by far the option preferred by students when considering taking future courses. The hybrid mode has quickly become the ideal for students, even among those who might be assumed to have other preferences. For example, local students have a strong preference for hybrid courses: only 7% of those living within 25 miles of DCE would prefer the classroom-only option, while over 75% would want the hybrid mode in choosing a future course. As one local student who was new to the distance experience explained on his survey, although having the option of viewing the class on the Internet was of no interest to him in choosing the course, he came to realize how invaluable a feature it was as the semester progressed. In the follow-up interview with him, he stated:

I've reached the conclusion that it would be unwise for me to sign up for a course without that [distance] component. I should also mention that I only live a 15 minute walk away.

Students who are currently enrolled in the distance only courses also overwhelmingly prefer the hybrid mode (84%) and only a fifth of them would take distance only again as their preferred mode, a difference which is not statistically significant compared with students in the hybrid courses. This is perhaps in part explained by the fact that most of the distance-only students who answered the survey live locally and would enjoy the in-class experience. However, it is interesting to note that the truly distant students may also have reasons to prefer hybrid courses over Internet-only ones. One student living across the country mentioned as an aside that the fact that there were ‘live’ students in the classroom made the course better for him compared with some distance programs that are only ‘talking-heads’.

Table 2.5 Preferred Course Modality (percentage) Classroom-only 6.1 Classroom plus Internet 73.0 Internet-only 14.3 Does not matter 7.5 N=230

Curiously, even students who never watched the videos of their own classes would prefer to have the Internet option in the future. Of students currently enrolled in the hybrid courses, one quarter of them never used the video option in their courses. While 19% of these students who never view the video favor the classroom-only option, it is still striking that 65% want the option of watching the course over the Internet. It may be that they see the value in having the Internet viewing capability should they need it in the future.

One way of gauging how strong the preference is for the hybrid option is to examine the willingness to pay for it. Even for students who do not use an option in a course, if it is free to them and there is no downside to having it, then they are rational to choose it, especially if they think that it might be useful under certain circumstances. However, faced with some cost for the

Chapter 1 17 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University option, students may be much less willing to choose it. That may be because they are unlikely to use it or because they do not feel that the value of the option is equal to the additional cost. For the respondents in the survey, the majority (59%) indicate that they would be unwilling to pay any additional fees to have Internet video capability and only fifteen percent would be willing to pay more than 10% extra for it.

Regression analysis (not shown) of the willingness to pay revealed that the following factors are related to being willing to pay for Internet availability:

· higher rating given to the current course · greater distance of residence from campus · using the video component of current course · less interactive learning style

This suggests that the value attached to the distance component of courses is related to a variety of factors that have different implications for pricing and advertising strategies. The importance of the perception of quality, reflected in the fact that the more highly rated a course the more likely the student is willing to pay for distance, suggests that if any additional charges for such courses were to be instituted, such an increase should only occur when the quality of the distance experience can be assured. Since one of the findings of the study is that a course’s learning curve for distance is sometimes steep in the first year, one suggestion would be to hold off a price increase until a course’s distance component runs smoothly.

Another strategy could be to fold the cost of distance into an overall tuition increase since the ill-will generated by differential pricing, particularly among local students, may not be worth the additional revenue. In addition, charging higher prices for courses available on the Internet could encourage local students to opt for a classroom-only experience of the hybrid courses, which would be a waste of the technology and counterproductive to a growth strategy. Since growth will partly need to come from distance-only local students due to the physical ceiling on classroom seats, DCE would not wish to create a negative incentive to avoid hybrid course.

The differences among students with respect to their attitudes towards the distance component of courses depending on their place of residence also bears on future Extension School strategies. For students living outside the Boston area, the distance option is clearly a necessity to be able to enroll in the courses. The importance of Internet availability for them in choosing their current course is extremely high. In addition, as we noted above, the greater the distance from Cambridge, the more heavily the prestige factor of Harvard factors into the decision to take the course. This suggests that a distance course recruitment strategy for students living outside the area could focus on being able to access the Harvard experience from anywhere in the country or world.

Locally, the situation is rather different. For local students (those living within 25 miles) the distance component is currently a highly desirable feature, yet appears to be non-essential. Not only are they much less willing to pay extra for Internet availability, it is also far less important in their decision to take a course, as Table 2.3 above shows. For them, the value of the Internet option lies in the flexibility it offers: the closer students live to campus, the more

Chapter 1 18 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University important is being able to view the course when it fits their schedules. Local students are likely to be attracted by a focus on the ability to combine taking a course with their busy lives.

The results from the local students offer another suggestion about the future growth of distance courses at the Extension School. There is some reason to think that the pressure to include a distance component in courses will rise from the local market that currently makes up the bulk of demand. While the local students may not currently view the distance option as particularly important, their behavior to some extent suggests a different picture. Certainly, the local students in the hybrid courses are oriented to the classroom. Over sixty percent of those living within ten miles and half of those living under twenty-five miles attended at least ten class sessions. At the same time, however, they also tend to use the Internet video. Two thirds of these students use the video and spend approximately 1.6 hours weekly doing so. Primarily they use it to make up for missed classes. As students have experience with this convenience, they are likely to want it in the future. As the Extension Schools expands its distance offerings, more students are likely to come in with some prior distance experience because of the repeat nature of many of the students. Of the local students in this sample, over eighty percent had taken an Extension School course prior to their current one, though fewer than half had ever taken a distance course. As more students take courses with a distance component, the demand to offer the option in many more courses is likely to rise.

We draw this conclusion from the fact that prior experience with distance influences local students’ rating of the importance of Internet availability in their decision to take a course. In our earlier discussion, we noted that Internet availability is less important to local students than to those living outside the area. However, local students who had previously taken a distance course rated its importance at 3.3 on a 5-point scale, significantly higher than the 2.6 rating local students who had no previous distance courses did. Since virtually all of the local students with prior distance experience have also taken courses at the Extension School previously, we assume a high percentage of the past distance courses were also DCE courses. This suggests that having the experience of the distance component of courses for local students creates a demand for subsequent distance courses. Combined with the fact that distance experience is also associated with a greater willingness to shop around for courses suggests that to stay competitive, DCE will need to grow its number of distance offerings.

Table 2.6 Students’ likelihood of taking different class modalities if only option offered Internet only Classroom-only Definitely 17% 16% not Probably not 32% 17% Probably yes 25% 27% Definitely yes 26% 40%

N=150 Does not N=228 include students currently in distance only course

While students may prefer a certain option and even be willing to pay extra for it, it is worth considering how they respond to the various options under conditions of limited choice, as opposed to different price levels. This is particularly informative since, as noted above, demand for Extension School courses is probably fairly inelastic with respect to price. When presented with the options of taking their current course if it were only offered as pure distance or as a pure classroom-based course, there was generally a fair amount of support for taking whatever option was offered; as Table 2.6 shows, half of the students currently enrolled in the hybrid courses would be fairly likely to take a course offered only over the Internet.

Understanding the factors associated with students’ willingness to take either of these options is useful for further assessing demand for different types of courses. Table 2.7 reports regression results for each of these. The models considered the effect of how far away a student lives from campus, whether the student’s learning style is interactive (a scale made up of several variables measuring the respondents’ assessment of how they learn), whether they had taken any distance courses before, how much they enjoy using computers, and how old they are. How well they manage their time was also added to the distance-only model on the assumption that the lack of structure provided by a fixed weekly class meeting could be a deterrent to students with below-average time management skills.

Table 2.7 Willingness to choose course modalities under conditions of choice constraint

Independent variable Estimated coefficient Standard error Model 1: Internet-only Distance from campus .334*** .071 Interactive learning -.217** .089 style .345** .156 Prior distance courses .321* .180 Employed full time .183** .089 Enjoyment of -.002 .071 computers .169** .068 Age 1.482*** .517 Time management .25 Constant R2 -.441*** .056 Model 2: Classroom-only .385*** .081 Distance from campus -.132 .135 Interactive learning -.113 .164 style -.209*** .078 Prior distance courses .118* .065 Employed full time 3.461*** .465 Enjoyment of .34 computers Age Constant R2 *** p <.01 ** p <.05 * p <.1

From the point of view of DCE, the question of what factors are related to distance-only courses is somewhat more pressing in terms of decision-making since the default is to offer classroom-based courses. The results show that to some extent the willingness to take classroom-only and distance-only options when they are the only ones available is a mirror image of itself. Where students live is a key factor for obvious reasons14, and the farther away they live the more likely they are to take a distance only course; and the closer they live, the more likely they are to take classroom-only option. However, there are also individual predispositions of students that influence the kinds of courses they would be willing to take.

14 It is interesting from the point of view of the market for different types of courses to look specifically at how students’ residence is related to their willingness to take a distance- only course if that were the only available option: roughly 60% of those living within 25 miles would be unlikely to take one, compared with 32% of those outside of Boston, 6% of those out of New England and none of those outside of the US.

Students whose learning style is more interactive15 are likely to favor the classroom-only option and less likely to choose a distance-only course. Similarly, the degree to which students enjoy using computers influences how willing they are to take certain types of courses under conditions of constrained choice.

Just as learning style influences students’ willingness to take certain types of courses in the future, it is also appears to be related to who takes current courses and how they take them. There is a significant difference between local students and those living out of the Boston area in terms of their learning styles. In the hybrid courses, local students on average have a more interactive learning style than students who live farther away16. This suggests that those willing to take a course at a distance tend to be more independent learners. Learning style also influences the behavior of local students. We examined some of the factors related to whether local students in the hybrid courses use the distance option or not. Our results (regression not shown) reveal that local students who tend to come to class are those whose learning style is more interactive and who do not have full time jobs, while the type of connection used to access lecture video, taking a Computer Science course and previous distance courses were unrelated to class attendance.

These results suggest several options for DCE to consider in its planning for distance. Since students who are more comfortable with computers are more likely to opt for distance, one priority for developing new distance courses is to target those courses that are related to computing. This does not simply mean courses in the Computer Science department but also those in other fields. For example, there are other courses in law and business that would make good candidates. There may be some courses that, because of the structure of assignments or the nature of the subject matter, are more conducive to independent learning styles. While identifying such courses would require an evaluation of the syllabus and classroom work, as well as perhaps the students, such information could be used to build the distance program.

Satisfaction and Experience with Distance Capabilities

While the evidence clearly shows that students have a preference for the hybrid model of course delivery, the study is also concerned with understanding student satisfaction and use of distance capabilities. Overall, students rate the quality of the technical aspect of the distance courses lower than the quality of the courses as a whole: the average course rating is 4.3 on a 5- point scale compared with 3.6 for the technical quality of the videos.

One significant factor that helps explain the relatively low rating of the technical quality is that not all students access the course videos using high speed cable or DSL lines. Over a third 15 The interactive learning style scale was constructed by doing a factor analysis of questions related to learning and is composed of four Likkert-type variables that asked students to indicate their agreement with the statements: In terms of student learning, I think it is important for the student and professor to be in the same room; I generally learn better by having someone explain something to me than I do by reading books or articles; In general, I learn at least as much from the discussions and comments of other students as I do from listening to a professor lecture; and I am better at getting my points across by talking to people rather than in writing. The internal reliability of the scale (Cronbach’s alpha) is .62. The scale ranges from one (low interactive style) to four (high interactive style); the highest score would represent strong agreement with all the statements. 16 Local students have an average score of 3.0 on the 4 point learning styles scale, while out of area students have an average score of 2.7.

(36%) connect to the Internet using 56K modems, which tends to give a choppy view of the course video. The technical quality rating of these students at 3.2 is significantly lower than that given by students using better technology who rate the quality at 3.9. It is also the case that students who have taken a distance course in the past rate the technical quality higher (3.9) than those who are taking their first distance course17 (3.5). It may be that students new to the distance experience have expectations about what the technology can offer that are somewhat unrealistic or that veteran distance students know what to expect and this makes them more forgiving of any problems. DCE should continue its efforts to make students aware of what they can expect with regard to the video technology and what kinds of connections and software will optimize their experience.

The lower rating given to the technical quality of the distance component may also be related to a variety of relatively minor factors that are detailed in the sections on the follow-up student survey and faculty interviews. From a policy point of view, one concern about the relatively low rating is that how students evaluate the technical quality is related to the overall rating of the course. This could have the effect of lowering the ratings in distance courses when there are technical problems, which could be seen unfairly as a reflection of the teaching staff’s performance.

We were interested in what factors affect overall ratings of courses and how important ones related to distance courses are. Table 2.8 shows a model of course ratings. In addition to the technical quality rating, distance related factors include: course modality, that is, whether it is offered only on the Internet, since the experience of DCE is that such courses typically receive lower ratings than others, and a scale (5 point) measuring how well respondents feel that the videos allow them to follow the instructional process of the class. A scale of how easy it is for students to obtain course materials is included; while not necessarily an issue of distance education, our interviews with faculty and students show that copyright issues and availability of materials beyond Cambridge sometimes do affect students who live outside the area. Two variables that reflect student experiences with the faculty and teaching staff, typically thought to influence course ratings, are whether the professor cared about the student’s progress and how quickly respondents receive feedback on their work, and those are included here as well. A number of variables, such as how far from campus a student lives, the grade he or she received in the course, and some demographic information were found to be insignificant in earlier models and so are omitted here.

17 Even when controlling for what kind of connection students use to view the video, the relationship between their rating of the distance technology and whether they have prior distance experience remains robust.

Table 2.8 Determinants of Distance Course Ratings Independent variable Estimated coefficient Standard error

Internet-only course .001 .107 Technical quality rating .107* .062 Follow instructional .135** .063 process .116*** .042 Professor cared .191*** .039 Timely feedback .216*** .061 Ease of obtaining materials 1.389*** .327 Constant .44 R2

*** p <.01 ** p <.05 * p <.1

The results show that with the exception of course modality, all of the variables entered in the model are significant predictors of the general course rating. For DCE planners and distance course faculty, the results are of interest because they suggest that not only are ratings affected by institutional factors beyond the classroom (how good the video delivery is, for example) but also because the distance component may change the dynamics of factors under the control of the faculty. That is, distance courses may change classroom dynamics in ways that need to be recognized and addressed. While a distance course may from the outside seem to be identical to its classroom-only version, the experience of students may be different.

In follow-up student interviews reported below, many students found that faculty attentiveness is an important issue. When designing distance courses, while it is important to put course mechanisms in place that deal with the obvious issues (such as how teaching staff is to communicate with students, making sure they do not feel overwhelmed by the demands of increased email traffic or devising ways of efficient assignment notification and delivery), the more difficult issue of the personal connection between students and faculty also needs to be addressed. For many students, when they do not feel that the professor is concerned with them, their experience in the course suffers. Needless to say, this is not exclusively or necessarily a problem with distance courses; however, the problem may be magnified in them. Creating that sense of connection across time and space is a task that DCE might want to focus on in working with faculty since the lack of it has negative repercussions for student satisfaction.

It is also interesting that the pedagogical aspect of the distance component is related to overall satisfaction with the course. How well students can follow what is going on in the classroom when they watch the video is an important predictor of the course rating. In the student follow-up, one of the main concerns expressed by distance students was not always being

Chapter 1 24 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University able to understand what occurred in class. This happened when professors referred to something off camera, when they forgot to repeat student questions, when classroom conversations and discussions were inaudible, and when professors projected images from their computers onto a classroom screen. Resolving these issues would enhance the distance experience for many students.

Students were surveyed on other pedagogical aspects of the courses that bear on distance education. Roughly 70% of the students were in courses that used bulletin boards. Overall, most students who used the bulletin boards found that posting to them and reading them were useful activities.18 Posting had a slightly lower average rating of 3.4 on a 5 point scale than reading others’ posting (3.6). Because much of the contemporary research on distance education highlights the role that bulletin boards can play in fostering dialogue and learning among distance learners, it is worth examining what factors influence students perceptions of how useful the bulletin boards are.

18 The breakdown of the 5 point usefulness rating (1=low, 5=high) is as follows: Posting to board Reading posts Low utility (1-2) 25.2 19.3 Medium utility (3) 21.8 21.3 High utility (4-5) 53.1 59.4

Table 2.9 Utility of Posting and Reading Posts on Class Bulletin Boards Independent variable Estimated Coefficient Model 1: Utility of Posting to Bboard Distance from campus .174 Interactive learning style ** Prior distance course .001 Internet-only course -.001 Computer enjoyment -.332 CS course .171 Constant 1.876*** R2 .770 .18 Model 1: Utility of Posting to Bboard Distance from campus . Interactive learning style 337*** Prior distance course .226* Internet-only course -.162 Computer enjoyment -.351 CS course .176 Constant 1.427*** R2 .168 *p<.05 .24 **p<.01 ***p<.001

Table 2.9 shows models of what predicts the utility of posting to class bulletin boards and reading them. In both cases, being in a Computer Science course greatly raises the perceived usefulness of boards. This may be the result of many different factors: CS students may be more familiar and comfortable with the culture of bulletin boards; CS course material may be more conducive to discussion; the staff of CS courses may be better discussion moderators; or the bulletin board software may be better in the CS courses. More research would be needed to understand the reasons for the difference. However, the results do suggest that DCE consider working with the staff of the courses outside of Computer Science about how to run their bulleting boards since the non-CS students’ opinion of their usefulness is very low, both in general and when compared with their counterparts in Computer Science.

Students with a more interactive learning style are more likely to find posting to boards, though not reading postings, more useful than other students. Since posting is an active task that assumes a responsive audience and therefore dialogue, students with interactive styles may feel that the connection to classmates is particularly beneficial to them. Well-moderated bulletin boards may be especially important for creating the sense of community and connection that

Chapter 1 26 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University some students feel is lacking in the distance aspect of courses, as we discuss in the next section. Greater distance from Boston is also associated with viewing the bulletin boards as useful. For students without the opportunity to come to class, it is likely that the bulletin boards are an important source of information, and offer them the chance to participate and interact with other students. Because bulletin boards are particularly important for students who take the courses at a distance, it is essential that part of the support DCE offer courses that decide to go distance is bulletin board management. There is an emerging body of research and literature on how to moderate and manage bulletin boards in a way that achieves pedagogical aims of courses. Rather than assume that individual classes will figure out how to work with bulletin boards, DCE needs to offer the means and training to course staff based on the most current work about the role of bulletin boards in courses.

Websites are another aspect of courses that bear on distance education. We asked students to evaluate three aspects of their course’s site: links to other sites, being able to download course materials, and course announcements. Students gave very high marks to the latter two with scores of 4.7 and 4.5 respectively on a 5-point scale. They rated the links to other sites much lower at 3.9. Interestingly, many of the variables that differentiate responses among the students on other issues were not significant for the website measures; however, there was a strong age difference in terms of how students rated the links to other sites. The younger the student, the worse the rating, and the youngest group, those 18-25, rated the links much lower than any other age group.19 This youngest group has grown up with the Internet, is used to surfing the Web and using links, and what they consider acceptable may reflect a much higher standard on their part than older students have. Classes seem to be functioning well in terms of using their websites as substitutes for paper handout/material distribution or in-class announcements (although as the next section shows, not all have completely mastered this). However, they appear to be doing slightly less well in taking full advantage of the possibilities offered by the Internet and Web-based pedagogies. Not only will this have to change as more students with extensive Internet experience form the student body, but also in order to serve distance students well who may come to expect more from on-line learning. This is another area where DCE can provide advice and support for faculty.

Student Survey Follow-up After examining the student data, we decided to follow up on certain issues related to the drawbacks of distance education that we believed would benefit from greater discussion on the part of the students. We contacted 95 students by email who had indicated on their surveys that they would be willing to talk more about their experiences and 63 of them responded almost immediately. They were very thoughtful in their responses and wanted very much to share their experiences at DCE and provide useful feedback to DCE administrators, signs that we took as program loyalty and satisfaction overall. As previously noted, students were generally enthusiastic about their experiences with both the hybrid and the Internet-only models. Over 90% of students surveyed said they would recommend the class to a friend, a figure that rises to 100% among those who lived outside of New England and the United States. However, we found some indications that when students make implicit comparisons between the classroom model and distance, distance sometimes comes up short. It is important to point out that most students are

19 For example, 18-25 year olds rated the websites’ links at 3.0 compared with 4.7 for the over 55 age group.

Chapter 1 27 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University very willing to accept the deficiencies of the distance model. As we found in our interviews with the faculty, there is a clear sense of trade-off between the models, and for students, the benefits of convenience and the ability to review the classes do outweigh the negatives. Nevertheless, it is worth understanding what the negative results truly mean in order to think about whether and how they can be corrected.

The survey results showed that fully half of the students (51%) felt that the learning opportunities via distance were less than those available in a traditional course. In addition, two fifths (41%) felt that distance students inevitably missed out on part of the learning experience of the course. Less problematic were the responses to questions about whether students had sufficient opportunity to participate (only 13% felt they did not) and whether they were bothered by having the camera in the classroom (about 10% were). We followed up on these issues by individually emailing those students who indicated deficiencies about these aspects of the course and asking them to explain their opinions in more depth. Students were extremely thoughtful in their answers and most of those contacted responded.

We identify a number of common themes and suggest the different ways that DCE might consider how to respond. The first problem is the breakdown in attention from course personnel that distance and hybrid students sometimes experience. In some circumstances, this is relatively easy to correct. A number of the students felt that assignments and emails were not returned in a timely fashion by the TAs or the professor, and as the previous section noted, this is an important determinant of overall satisfaction with courses. DCE must make sure that the technology for exchange of coursework is adequate if the course relies on it. More importantly (since this is the primary cause of student discontent), the administration needs to remind faculty that they need to have in place a mechanism for responding to students. As a number of faculty members noted, the volume of email has increased with the distance model. If this translates into a lack of timely attention to the distance students, then faculty need to develop new models of course management, such as overseeing the email speed of TAs. One student noted:

The only down side I found with the class was knowing where I stood on assignments. My TA was having a difficult time grading his assignments and getting them back to the students. I talked to the instructor several times and emailed both the head TA and instructor. They did respond back to me but I felt they waited to long to escalate the matter and step in and help the TA out. I assume that my TA never asked for help and that is why they never stepped in. To the credit of the instructor, he did tell all the students waiting for their homework grades and comments, that if our TA did not get them to us by a specific date that he would grade them. Unfortunately this date only allowed 2-3 weeks left for the class.

The same student later went on:

I have touted the class and program to my co-workers but no takers yet, they saw how frustrated I was waiting for and never getting my grades or feedback on my coding.

Clearly this points to one of the downside risks for DCE of failing to take the distance students into account, which is the effect it has on future recruitment. In addition, student comments highlight the issue of the support that DCE provides for distance students. For some students who experience delays in the availability of lectures on line, late return of assignments and the like, there is the feeling of being a “second-class student,” in the words of one respondent.

Many of the students noted that there should be extra support for the distance students. It must be stressed that this is not a plea for special treatment and should not be viewed by DCE as such. Rather, students look at the need for support as a way of leveling the playing field with the classroom-based students, whom they see as benefiting from immediate feedback, sometimes being privy to information that only later, if at all, makes it onto the website, etc.

The belief that distance students deserve this extra attention from DCE is visible in this comment:

I just think that people paying the same for a distance class as they would for an in-person one are going to expect a bit more attention.

The implication of this is that students value attentiveness, not simply in the sense of appreciating it but also in a very real economic sense. Attention outside the course and from the institution is seen as a fair compensation for not getting the in-class attention of the professor found in traditional courses. The kind of attention provided to the distance student in a situation where a competitive market for courses exists may prove to be an important factor in attracting and retaining students. Some evidence from the follow-up interviews suggests that attentiveness is a factor helping students decide among DCE courses. One distance student living abroad noted that only three of the five professors he contacted about their courses during the open enrollment period responded to his emails and that this was a decisive factor in his course selection.

There also seemed to be some sentiment among local students that professors were not always sensitive to their need to use the distance option as their primary mode and the requests for help that follow from them. For many local students, the distance option is not merely a convenience but a necessity, given their work, travel and family schedules, and without it, they would not be enrolled at DCE. One student noted:

When I told professor that I live in [a nearby town] he was surprised that I am a distance education student. He didn't even think that it might be the case that I am working full-time (often overtime) and am unable to attend lectures and sections. In fact, distance education is the only way for me to do my Masters Degree.

In our conversations with faculty, we also noticed an occasional lack of understanding about why students living locally would chose to take the course over the Internet. Making the professors more aware that the hybrid model is the first choice of students may help to sensitize them to the needs of all those who use the distance elements.

Part of the difficulty in attending to the needs of students that some courses face appears to stem from a lack of experience with the distance format in terms of the administrative details of running a course. Not all courses are adept at posting the current information about assignments, due dates, and the like on the website. For faculty, making the transition to documenting everything on the website is necessary but may not be automatic, at least at first. In the interviews with faculty, many of them, understandably, were focused on the “big picture” of distance teaching, such as getting the distance students involved in the class, creating dialogue and community with them or worried about whether the video technology was working. To the extent that they discussed the nuts and bolts of the business side of the course under a distance model, it was in the context of making allowances for the delay that distance students face in accessing the lectures. They made little mention of how their own practices with respect to course organization were changing.

In part, it may be because this is often the work of the teaching assistants, and in part, because in a traditional classroom-based course, there are numerous informal mechanisms for disseminating information. In addition to verbal announcements by the professor or TA, information may be passed around by the students to each other in class or individually obtained when students approach the teaching staff during breaks or before class. When students do not attend class, they lose these sources of information and a communication gap is created, making it critical that everything be documented on the website. Even when information is stated during the lecture, this may not be sufficient for distance students who may miss part of the lecture due to technical difficulties or their own habits of viewing the lectures in discrete sections. The following statement by a student shows not only how frustrating the lack of attention to information can be, but also how it works counter to the potential of the course:

The course is fabulous. The professor was wonderful and actually so was the TA. But I think the problem was that none of the ‘business’ of the course was ever televised so no one ever got that. They’d post an assignment on the web but that wasn’t the same as the rest of the class got and there were some discrepancies on when things were due... I think it’s maybe because the professor is used to doing it that way in the day school but like this it doesn’t necessarily work. I actually went to one of the classes but it had been cancelled and there were about 10 of us there waiting for the class...(we didn’t know) there was no class (because it had been announced off camera...People just aren’t thinking about what they need to post, they need to have every single detail there and put it somewhere for people to see.

The need to document everything that is necessary for students to know about the class means that the habits of faculty (or their TAs) must change. What used to happen somewhat informally in terms of getting information out needs to become formalized; the use of the course website must become habitual. The preceding quotation also suggests that some faculty may erroneously make a hard distinction between students who take the course in person and those who take it via distance. This is related to the lack of faculty awareness of the hybrid nature of many of the students’ course participation. By failing to realize that many students live locally but often attend virtually, faculty may inadvertently cause difficulties for the students. In the example here, if a professor assumes that only the in-class students need to know that a future

Chapter 1 30 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University class is cancelled, there is a great risk that the hybrid students who are not always present will miss out on this critical piece of class news.

A different issue of attention that the follow-up raised is faculty contact. This is more difficult from a DCE policy standpoint because by definition, faculty members cannot be increased when students are added the way that TAs can. However, students very clearly distinguish between having contact with the faculty member and with the course assistants. Distance students keenly feel that they miss something in terms of interaction with the professors. One student took a DCE course as a distance student in the spring and compared his experience with an on-campus summer course from the same department in the following way:

I felt that [the professor] was sincerely interested in bringing out the best in each student. He really cared about teaching us the classics, he even showed up at the final exam to make sure that we were ok. I've never experienced a teacher like him. He really wanted to help us, and he did. This is what's missing in the distance course, the contact. In the on line course I didn't have the feeling that I was part of a class.

Students did offer suggestions to help the situation, however. Chat rooms and on-line discussions may offer students some feeling of connection. One student suggested that the professor occasionally drop in to the electronic discussion groups rather than simply having the TA run them. Another student suggested having such a group for a course that did not feature it. DCE might consider working with faculty (or creating electronic materials) to help them maximize the benefits of such groups and making sure that they work. DCE should ensure that such innovations are functional before the course goes online. Another suggestion was that professors make reference to specific points raised in emails or other conversations with distance students in subsequent lectures so that they are captured on the video. This would give distance students the feeling of having their concerns heard and acknowledged.

At the same time, students made requests that are unfeasible where there are large numbers of distance students. For example, they asked for more phone availability from the professors, which would certainly cease to be viable when class size reached some threshold. Faculty probably need to make it clear from the outset what kind of interaction distance students can expect so that there are no feelings of having been misled.

A second issue with the distance model identified in the follow-up is the inability to sometimes have access to all the relevant information necessary for understanding the course material. Many students, both those who take the course at a distance and those who attend online and in the classroom, complained that much of what is visible to students in class is not accessible to them over the Internet. A typical comment is:

There were also a lot of questions from the classroom audience, many of which were very interesting. Unfortunately, the answers were almost uninterpretable because of the fact that you are not in the room to hear the exchange. There is something significant that is lost when two people interact and you as an observer cannot hear them both.

One policy response here could be to equip the classrooms with desk microphones so that such exchanges could be audible. Currently, students appear to feel that professors usually make the effort to repeat the questions. However, the distance students notice that it is not close to 100 % of the time and they feel things are lost in the condensing and paraphrasing. In addition, some of the students complain about the repetition of questions because they feel it interrupts the discussion and hampers the spontaneous flow of ideas.

Distance students (primarily in the Computer Science courses) also noted that when professors projected images from their computers in class, they were not visible over distance. For example, they complained that when professors show and modify computer programs, they cannot see this. One student noted the following irony:

[L]ectures given using hyperlinked web content are keyed to the web content available at the time of the lecture. For example, the professor pulled up his myYahoo screen in class and described (in words) how he had customized it, etc. However, on my home computer I was looking at the default Yahoo home page. This is more a function of the information that is to be delivered in an internet- focused class, but if you can't talk about the internet on the internet, then you definitely are not doing things quite right!

Other technical issues that students face emerged in the follow-up interviews. A common theme from the students was that they wondered whether they were the only ones to experience a problem and there have clearly been problems they were unable to resolve. In the absence of a distance community, students feel acutely isolated when it comes to experiencing technical difficulties. In Hirschman’s (1970) terms, they may not feel they have a ‘voice’ option; that is, rather than speak up about a problem, they are left with two options – loyalty (hope the problem will be resolved or live with it based on some feeling of trust that the institution will rectify the situation) and exit (try what the competitor has to offer). For an organization in a competitive environment (such as distance education), encouraging voice is critical because it is the best way to ensure that problems that would drive away clients get identified and resolved. Loyalty provides an important buffer that allows an organization to keep members while the problems are being dealt with, but unless it is aware of the problems, they cannot be fixed.

The problems identified by the students are not insurmountable in the sense that they do not ruin their ability to take the course, but they are more than minor annoyances. One student mentioned the deteriorating video quality as the lecture progressed so that the “professor turned into a blob of pixels by the end.” Another was forced to reinstall constantly RealPlayer, while Jabber often caused machines to crash. There may be unidentified problems with software that DCE is unaware of because students do not complain. It may be worthwhile trying to identify some of those simply to gauge the extent of the problem. Since the students in the survey consider themselves highly computer competent (4.5 on a 5 point scale) and enjoy computers (4.4 on a 5 point scale), they may be more willing to forgive and live with glitches than those who are less ‘trailblazers.’ Failure to identify and correct problems could lead to higher dropout rates in the future, especially among those who are less likely to feel loyalty towards what DCE offers and more likely to adopt an exit strategy.

Another area where distance students feel they miss out is in not being able to view section meetings. For many students, much of the real explanation of the material goes on in section. DCE might consider looking into the cost of taping section meetings. Perhaps as a first step, DCE might try to identify those courses where a large majority of students plan to be primarily distance learners and run an experiment in those to see how much benefit students feel they get from watching sections.

An interesting finding from the follow-up is how central the concept of ‘real’ is to students with respect to their learning. This emerges when they contrast being in class to being present over distance. What students typically are referring to is interaction with the other members of the course: professors, TAs and other students. Typical comments along these lines are:

I’d much rather be present in the class than watch the video from the web because the human side, the feeling of REALLY being part of the class is missing when just watching the video. You miss the conversations with your classmates, you cannot ask the professor questions and you [are] not really THERE. and Sometimes the classes go a little bit longer than the internet schedule, or the professor will share little anecdotes with us that the distance students miss out on, and other times it's exactly the same as the distance class but I just feel I'm a real participant.

What the students seem to be saying is that what makes a course real for them is being linked to a community or being part of the group. They want to know that other members of the course have a sense of them as individuals. When they are in class, they get something extra in terms of contact with the others. This is a strong argument for working with the teaching staff on the issue of how to create and merge the online and classroom communities, a point we discuss at much greater detail elsewhere in the report. However, there is a broader sense of community that goes beyond the classroom that DCE as an institution might consider. The responses here pointed to students’ desire to belong to the Harvard community, and clearly many of them feel that the sense of Harvard as an institution should inform their experience. This is part of what distinguishes DCE from its competitors, as the following quote indicates:

The thing about distance students is that, their situation already makes them feel ostracized. I think the successful distance program and one that is able to differentiate itself from the rest is its ability to give distance students involvement. People want to feel like they belong to a great institution, not just passing by.

Given that the Harvard name is more important in the decision to take a DCE course for non-local students than those who live closer by, DCE might consider strategies over and above those employed by individual courses to make true distance students feel less like visitors. In future surveys, questions can be included to see whether students would value things such as the

Chapter 1 33 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University creation of virtual professional networks of DCE students and alumni, access to Internet lectures by Harvard professors, and other mechanisms designed to make distance students feel like part of a community.

The student interviews also allow us to go somewhat beyond the main focus of the survey and the study in general. Although our concern was with the distance aspect of DCE’s courses and how this is experienced by faculty and students, the interviews shed some light on how the distance component of courses shifts the in-class experience. We note above that the hybrid modality enjoys tremendous popularity among the local population, but there are some trade-offs that having the distance component may represent in terms of changes in the classroom atmosphere. For example, although the vast majority (90%) of students were not bothered by having the camera in the classroom, those who were felt that it discouraged participation by making students hesitant to speak up.20 As one student put it:

The camera was intrusive in the classroom… When you ask a question, you can feel you need something clarified, but you are taking away from other students by asking for this question. I know that most students probably will want the question asked, but the asker is putting him/herself out there. The camera is extra intimidation against asking a question. The questioner and the teacher cannot clearly tell if the rest of the class is interested in an answer. And being able to participate is the only way I learn, but the camera makes people shy from participating.

Other students felt that some of the behavioral requirements for making the distance component function impeded the learning process in the classroom. Some students present in the classroom complained about the disruption to the flow of the discussion when they had to wait as the professor repeated the question or comment for the online students, or it stopped while a microphone was brought to students who wished to make a comment. Miking the desks so that student comments would be transmitted directly to the video feed would resolve this issue for in- class students as well as the problems noted above by the distance students.

Many students find that a stimulating classroom discussion is among the top benefits of attending the class and almost impossible to reproduce at a distance. While such discussions do not characterize every course or every class session, when they happen, students feel mentally challenged and stimulated. They acknowledge that because of the possibility of watching the course from home, class attendance is generally down; for some of the classes, this has the effect of reducing classroom discussion. They see the trade-off between the convenience of not having to be in class and the impact of absent students on the nature of learning and participation.

Faculty reaction to distance teaching also influences dynamics in the classroom. The next chapter discusses how faculty view their own behavior vis-à-vis the camera and the other requirements of the distance mode but here we anticipate that by examining students’ perceptions of teachers’ behaviors. Some of the faculty interviews revealed that that not all professors are comfortable in front of the camera, and are uncertain about how to engage two

20 DCE policy prohibits showing students on camera unless they have previously signed a release form.

Chapter 1 34 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University distinct audiences. A student taking the course of one such professor described the professor’s style almost exactly as he himself had:

Because of the nature of the beast, the teacher can act like a distracted lover, directing his/her attention to the camera and to the vast unseen presence that comprises the rest of the class.

For a few of the faculty members, the fact that the course has a distance component has a negative impact on the dynamics of the in-class experience. In the next chapter, we discuss in greater detail how teaching styles under the distance model vary and offer suggestions about how to enhance their effectiveness.

Not all of the students’ comments about distance education reflect factors under the direct control of DCE faculty and administrators. Numerous students making a comparison between distance and traditional course models note that their own characteristics influence how effective they view each model. Students’ awareness of their learning styles means that there is a self- selection process into a specific modality. For some, coming to class is a necessary motivational feature of courses. For them, the mostly or all distance model is not a feasible option as the following students’ comments suggest:

As a working professional, finding time to concentrate on the course is always difficult. Attending class and the sections as part of my schedule makes it easy for (compels!) me to be on track with the course, meet the professor, TAs and of course my other classmates to discuss questions, issues and doubts. For me, to rely on technology alone would require a huge amount of effort and will-power on my part to complete the course. And When I have the freedom to watch a lecture at my convenience, I may procrastinate. By going to a regularly scheduled lecture, I'm forced to sit thorough it without interruption. Although this is probably adolescent behavior, it's a reality. Because I work full-time, try to have a social life, etc., sometimes I like being forced to something that I wouldn't normally be in the mood to do.

For others, the distance option is preferable because of their own learning and time-management styles. For students who are distracted by other students, distance courses are the better option. Also some suggested that they thrive over distance because they enjoy the efficiency of using the taped lectures, reviewing what they need to and skipping the parts that they find superfluous.

Not all students may be as self-aware as some of the respondents about how they learn best. One of the questions faced by distance program administrators is whether distance courses should assume an ideal distance learner and structure programs and courses accordingly or whether they should create distributed learning spaces that allow for a variety of learning styles. The implicit assumption behind the questions some programs ask potential students to consider before they enroll is that the ideal distance learner is highly motivated, capable of working independently, pressured for time, and possesses strong organizational skills. Certainly, courses based on such an approach can work well for many students, though they may fail those students

Chapter 1 35 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University whose learning style is somewhat different. More importantly, however, this approach neither takes advantage of the unique possibilities for learning offered by distance technology, nor does it reflect the insights from distance research that suggest that the creation of networks and collaborative environments are essential for learning (Leifer and Friedlander, 2001).

Some institutions are committed to developing new pedagogical techniques and the tools to offer innovative and highly effective distance learning. Such efforts, as the experience of places like the Stanford Learning Lab (http://ssl.stanford.edu) shows, must be long-term, based on ongoing research, experimentation and evaluation, and part of a well-coordinated delivery system. While we argue that a strategy focused on creating new tools and models for distance learning is the appropriate one for DCE, implementing it requires a long-term commitment of the time and resources of the institution. In the recommendations section, we offer several options for DCE along these lines.

Finally, the follow-up provides information useful for the Internet-only courses offered by DCE. Some specific issues highlighted there concern mainly course management. A number of students indicated that they felt the teaching staff (both faculty and TAs) were sometimes remiss in keeping current with the videos, that is, reviewing them before trying to respond to student questions or holding sections. Others perceived a lack of attention to the courses in the sense of responding to students in a timely fashion or lapses in thinking about how to structure the class to foster student learning. Also, because one of the biggest advantages to students that distance provides is being able to fit the course into their schedules and plan ahead, they feel that there is absolutely no reason that the all distance courses cannot provide them with the assignments and due dates from the inception.

Chapter 1 36 Understanding Distance Education Distance Education Evaluation Division of Continuing Education, Harvard University Chapter 3: The Views of the Faculty Introduction In this chapter, we report the results of interviews with faculty who teach courses with a distance component or over the Internet. Open-ended interviews were conducted with eight faculty members and one teaching fellow. A common guide was used and is reproduced in Appendix B. Five of the faculty members were in the Computer Science department and four from a variety of other departments. Two of the CS courses were offered exclusively over the Internet. For three of the faculty members, this was the first time they had taught with distance capabilities while the remainder had offered their courses previously with the distance option. By design, the courses tended to be large. DCE’s strategy of developing distance courses concentrated on those where demand was high, where much of the work of grading and out-of-classroom interaction with students was already delegated to TFs and where professors were willing to try the distance option. Because of the size of these courses, professors were already to some extent ‘distant’ from the students so that DCE’s strategy in part was based on trying a distance experiment under conditions that did not have to be vastly different from what already existed. The lessons learned from the experiment could then be used to grow and adapt the program as necessary without incurring undue risks from making too diverse a collection of courses and course types available over distance.

The themes addressed in the interviews focused on the advantages and disadvantages faculty felt distance teaching provides to them and the students, what kind of support they received and would like to have, if and how their practice or courses had changed as a result of going distance ,and what the role of the TF is in distance courses. We also examined the interviews for common themes that emerged as conversations progressed and the analysis and implications of those themes are also presented here.

Like the students, faculty offered a very positive view of distance courses, although as we show below, they tend to see the benefits of the courses as accruing to students rather than themselves. They offered numerous suggestions for how DCE could improve its support, but they saw these as changes at the margin since they were generally pleased with the help they received from DCE.

The interviews challenged a number of assumptions about distance courses at DCE as well as highlighted other notions held by faculty that are untenable in light of the student data. One message that emerged from the interviews is that going distance is not business as usual for most courses. In some cases, faculty members were relatively unaffected, but the nature and scope of work for their TFs changed considerably. In others, the amount of added work for faculty was significant because of a steep learning curve about how to manage the distance and web component. Additionally, we found that many of the faculty either changed their teaching practices or have begun to wonder about how they might. Distance is therefore disruptive to the status quo in ways that may be both positive and negative.

We also noticed that many faculty were preoccupied by how distance students experienced the course. They tended to view the distance students as a ‘black box’ and worried about how to create a sense of community with them, and what they should be doing to reach them. Perhaps because of this sense of not knowing who they were, many faculty thought of students in separate categories -as

37 Distance Education Evaluation Division of Continuing Education, Harvard University distance students or classroom students- and failed to recognize the distinctly hybrid nature of many of them.

In addition to their particular experiences with distance teaching, we were intrigued to discover they had different fundamental perspectives about the possibilities offered by distance. These world views appeared to shape how they approached distance and what kinds of experimentation or modifications to the class they were willing to undertake. One perspective saw distance as an essentially flawed version of a traditional course, necessary perhaps to offer access to those who would not be able to take the course otherwise, but a second best option for students. The other view saw distance as opening new possibilities for teaching and learning, although often these faculty members were unsure about how to harness these possibilities.

In the sections that follow, we explore these and other themes. Together with the findings from the student survey, they provide a comprehensive look at DCE’s distance courses. They form the basis for a discussion about what we believe DCE might want to take into account as it contemplates the future of the distance program.

Going Distance Why do it? Since DCE currently has no separate distance program as such, there is no sense that any specific courses must include a distance component and accordingly no pressure on faculty to do so. Nor is there any significant financial incentive for faculty that would encourage them to go distance21. The motivation, then, for faculty members to offer their courses at least partially over the Internet was of interest to us. As the number of distance offerings expands, knowing what draws faculty to do it will be useful in planning for new courses.

We found two broad, sometimes overlapping, responses to this question. The first, mainly from the Computer Science faculty, was simply that they were approached and asked by their DCE colleagues to do so. They assumed that their workload for the course would remain unchanged, not only because DCE offered the necessary support in terms of developing the websites (if they did not have them already) or putting the slides from lectures up, but also because they assumed that as computer scientists, they would be able to tackle any problems that did arise. To them, the decision to try teaching a distance course was easy because they saw this an example of a new frontier for computers and themselves as pioneers.

A second response also entailed this sense of wanting to be on the cutting edge of a new phenomenon; a number of professors approached DCE about teaching their courses with a distance component because they were curious about the possibilities of the Internet and felt it was the place to be. In part, the curiosity was driven by an awareness of changes taking place in higher education towards a more market-driven, entrepreneurial model of delivery. Even though there are no incentives from DCE to faculty who offer distance courses, there is some sense that the market may provide some pay-off down the road for them and so in that way, there are perceived external incentives. One professor had a colleague who had been approached by a for-profit firm to develop a CD-rom version of his course; this made the professor curious about such possibilities and he wanted to see what

21 There is a small additional stipend that faculty receive for teaching distance courses, but it is not reason enough to do so.

38 Distance Education Evaluation Division of Continuing Education, Harvard University teaching a course with a computer component involved. Other instructors see the courses as in part places where they can test the development of software or web-designs that are linked to actual or potential commercial ventures. Another professor saw the Internet as a technological revolution and wanted to be in its vanguard.

The faculty were not without concerns before taking on the challenge of distance courses. Many worried about the pedagogical implications. How to ensure quality when students are not present was something they wondered about. In addition, they thought about how their particular type of course would translate under distance; especially those professors whose courses are discussion-based wondered how distance students would fare. They also thought about their own personal styles of lecture and student-interaction and were unsure of what distance would mean for those.

Course management concerns also surfaced prior to taking on distance, especially whether it would mean more work for TFs. Some of the trepidations focused on the external issues we identified in the systems approach to distance education, particularly the broad policy issues surrounding intellectual property that are largely unresolved at many universities. Some faculty members mentioned the worry that their ideas would be out there, not under their control, perhaps available for use by others who had no part in their creation, and without attribution.

Advantages and Disadvantages of Distance Faculty members saw both positives and negatives to distance courses. At first glance, they tended to feel that there were few very notable advantages for professors. Rather, they believed that it is the students who really gain from the distance option, allowing them to get access to courses that they would otherwise not be able to take. Some probing, however, revealed three types of benefits for faculty members. The first centered on their own organization and what that meant for their ability to communicate with students. Over and over, professors mentioned how the requirements of the distance courses in terms of having prepared slides meant that they had to be more organized and prepared further in advance of the class. One professor noted that:

It makes you do your homework; it keeps you honest. A week beforehand you have to have everything done. You can’t wing it. All of this is what a good professor should do anyway, but you know, you don’t always do it, some days you come in and talk off the cuff. Well, you can’t do that here.

They felt this helped them communicate better with the students. One professor who had initially thought the new, highly structured nature of his lectures would be a drawback for students because they would miss out on insights that arose out of a less structured lecture style was quite surprised to find that they actually preferred the structure.

A second benefit for professors related to their own intellectual work. Because the videotaping of the courses produces a permanent record of class lectures and discussions, a few faculty members saw the possibility of using it to help with their scholarly production. One professor presents the ideas he is working on for a book to his class and the ensuing tape of his own and his students remarks allows him much better recall of the ideas that he develops in the give and take of the classroom. In the future, this may be a particularly compelling incentive for other faculty members who want to do distance classes at DCE. Since it is often the case that professors have more leeway in choosing the subject

39 Distance Education Evaluation Division of Continuing Education, Harvard University matter of the courses they teach at the Extension School than they do in their home departments (where they may have to cover required or introductory courses), those who want to present their ongoing research and test out their ideas may appreciate having taped class sessions.

The third type of advantage to distance that we encountered in the interviews was not a direct, individual gain but rather related to potential gains to programs that professor work in and to their home departments. One professor saw the distance courses as being a potentially powerful recruiting tool for students across the globe to become aware of and perhaps join the Master’s program in his home Harvard department. Thus the possibility of expanding the pool of good graduate students is an indirect benefit of the distance courses. Another professor wanted to do a distance course at DCE because she felt the experience would be useful for her home department, where she was helping develop online learning resources. This programmatic type of advantage should not be minimized because it is directly related to one of the core missions of DCE: “to experiment and innovate with courses and teaching techinques that enhance learning in adult learners.”22 As a testing ground, the DCE distance program provides faculty the opportunity and means to try out innovations that they may not have the ability to do currently in their home departments. By drawing lessons from these, faculty may be able to enhance their teaching when offering courses in their own departments. For those in professional schools, this may be particularly true as their student bodies may be similar to DCE’s in many respects.

The drawbacks to distance courses that faculty encountered were in some ways the obverse of the advantages. At the same time as distance forces professors to be more organized, they lose spontaneity. Some felt that this tended to constrain dialogue and for them the increased structure represented a trade-off between student centered spontaneous pedagogical activity and highly structured presentations. Some also felt that the packaged nature of the slides and notes did not allow for the addition of new ideas or changes. Not all felt hamstrung by doing these in advance, however. One professor used the brief period between class and posting the slides as an opportunity to annotate his notes in a way that reflected the class discussion. He viewed this as a chance to make up-to-the-minute corrections and additions to his course, something that was inconceivable prior to doing his course over distance.

Another disadvantage that many professors seemed to feel acutely was the faceless nature of the students at home. Many complained that they did not know who the distance students were and felt that when it came time to perform, it was “a shot in the dark.” Many find the in-class interaction with students, much of the time non-verbal, essential for gauging the mood and learning of the class. By looking around at faces or body language, professors can tell whether students are confused or whether the homework presented particular challenges. Knowing that, they can adapt their performance and give the class at the appropriate level. They feel they lose this ability with respect to the distance students. From their perspective, there is a noticeable communication gap that needs correction. While it may seem that professors in large courses are traditionally unable to pay attention to their individual students, the feeling of a void created by the distance audience indicates that there is a level of attentiveness to the student body that influences teaching performance.

Other professors noted that the actual content of their course is to some extent variable and that the subject matter in any given semester reflects in part the particular interests of the students. Because

22 From The Mission of the Harvard University Extension School.

40 Distance Education Evaluation Division of Continuing Education, Harvard University they do not know the distance students, they worry that they may not be addressing their needs and interests. Because of this many professors of hybrid courses claimed they would never want to teach an Internet-only version of their course.

Minor technical issues involved in producing the course videos were annoying to a few faculty members and seen as small disadvantages. Because the classrooms need to be set up for taping at the beginning of each class, rather than having the equipment permanently there, faculty felt they lost valuable class time during set-up. One professor felt that the delay threw off his sense of timing for his lecture.

Readiness to teach the course We have seen the reasons that professors decide to teach over distance, but simply having the desire to do so does not guarantee readiness. Preparation is essential to the experience faculty will have, at least in the first offering of the distance course. We found that readiness includes a number of aspects: having materials ready on the web (i.e. a website), having the capacity to manage the website either oneself or via delegation to a skilled TF, and anticipating logistical issues like copyrights for web material and transcribing materials to be broadcast over the web.

Having a website prior to going distance seems imperative because dealing with the newness of the distance mode and creating a website was particularly stressful for courses that did these simultaneously. For those courses where the professor was not facile with computers, the work of building a course website fell to the TAs and DCE staff. One of the requirements for going distance in the future is that courses already have websites that are functioning and a teaching staff that is at least semi-familiar with how it works. For DCE, this means that part of the planning for new distance courses should include website development in the term prior to going distance as well as providing instructors with models of efficient websites and the emerging principles of good web pedagogy.

Another part of the preparation for distance courses is securing copyrights. The issue of obtaining copyrights for course materials arose in many of our interviews and was a source of unanticipated stress and additional work for many faculty. The additional work stems from the current legal environment surrounding electronic material and from the fact that in the past, faculty have been able to be fairly lax about adhering to copyright rules. The result, however, is that faculty spend much more time on permissions; they do not teach some material they would like to, which means they spend more time finding material that meets the requirements and redoing their syllabi; they put material on reserve that the distance students do not have access to. We recommend that DCE work with the Harvard library system to facilitate the process for faculty rather than leaving it up to each individual. Providing them with guidelines for using and posting electronic resources well in advance of the start date for their courses would be extremely helpful.

The Practice of Distance The impact of distance on teaching performances In the previous chapter, we examined student learning styles and their relationship to satisfaction and preference for distance education. Here we look at how teaching styles are affected by the adoption of the distance mode. We found a range of self-reported teaching behaviors. At one end of the continuum are those professors who change very little in response to “going distance.” For these professors, the value of distance is that it provides access to people in remote areas or who, for a

41 Distance Education Evaluation Division of Continuing Education, Harvard University number of reasons, cannot take such a course. Aside from making sure administrative and technical details function, their performance is unaffected. At an intermediate point on the spectrum are those people who have begun to redefine their role as akin to a talk-show host; a number of these professors drew on the same popular culture icon, Oprah, to describe their behavior. They focus on the live audience but are always aware this is not the

Continuum of distance teaching styles

Exclusive focus on Focus on in-class but aware Almost exclusive in class students of need to play to home audience focus on home audience intimate private setting of the traditional classroom; there are students out there watching. At the other end of the spectrum, some faculty switch their performance completely to focus on the distance audience. Unable to negotiate two audiences, the fallback position is to perform for the camera. We term this the “paradox of the camera” since while the course needs a live audience to function, the presence of the camera makes actual students not only superfluous but might also actually drive them away. There is something disconcertingly hollow about an instructor whose gaze is far away.

Many faculty members attempt their own innovations to keep the distance students engaged or wonder what they might try to maintain a better balance between the two audiences. Some call distance students on the phone to establish a personal connection; one professor goes out of his way to meet students if on business in the cities they live in and some ask distance students to send biographies and photos. However, the innovations tend to be short-lived, disjointed and easy to abandon. There tends to be little follow-through and no support to keep the innovation alive.

In terms of what students want from the courses, there is a strong link between overall satisfaction with the course and whether students felt that the professor met the needs of distance students. Students reward attentiveness on the part of the faculty when they evaluate the course. Despite the fairly individual, as opposed to systemic, nature of many of the faculty members’ attempts to draw in the distance students, overall the students in this sample felt the faculty members did do a good job of meeting distance students’ needs. As the following table shows, almost half of the students strongly agreed with the statement that, “The professor addressed the needs of distance students appropriately.” Importantly, responses did not vary significantly based on where a student lived.

42 Distance Education Evaluation Division of Continuing Education, Harvard University

Table 3.1 Student agreement with the statement “The professor addressed the needs of distance students appropriately.” Strongly disagree 1 4% 2 6% 3 19% 4 28% Strongly agree 5 42% N=191

Despite the relatively high marks given to the professors and how they deal with the distance students, we noted a fair amount of unease among those faculty members who felt that distance teaching demanded something different than traditional classroom teaching. For most of the faculty, even those who had initially thought otherwise, distance teaching implies an unavoidable change or transformation in teaching. However, many of them would like more guidance or suggestions about what techniques would be most beneficial for that transformation, rather than simply devising their own innovations. In addition, some are aware that their current distance teaching practices are not working (for example, teaching exclusively to the invisible, absent audience), but are unsure of how to correct them. These professors also felt that they would appreciate more guidance from DCE or some other entity, such as a center focused exclusively on distance education that would help them improve their pedagogical performance.

Much of the literature on distance education highlights what these faculty members have found through experience. As Moore and Kearsley point out, one of the biggest challenges for faculty when beginning to teach over distance is to change their practice.23 The initial tendency is to go on in a traditional fashion, simply delivering content. However, they argue that not only do distance students have different needs than their classroom-based counterparts, necessitating a different approach from the outset, but also that the presentation of materials needs to become much more interactive. This has a number of implications. So that distance students feel engaged, connected to the class and less anxious, it is essential that course staff members try to personalize the initial contact with them. In order that students not be passive participants, viewing the video as an object with no sense of their own subjectivity, teaching staff need to turn them into active participants from the beginning, designing tasks and assignments accordingly. The study suggests that professors also need to move away from the traditional delivery mode. As Moore and Kearsley state, “[I]t soon becomes obvious to most teachers that the delivery systems used in distance learning are better suited to two-way and multipoint interactions rather than one-way presentations.” (p.152) They conclude that good training for distance teaching should include practice with the following elements:

23 Moore, Michael G. and Greg Kearsley. 1996. Distance Education A Systems View. Belmont MA: Wadsworth Publishing Company. chapter 7.

· Designing and presenting content · Delivery technologies · Making the course more human and accessible · Techniques for stimulating student participation

Both the literature and the experience of most of the faculty at DCE suggest that distance teaching is not the same as classroom-based teaching. DCE needs to recognize that in the great majority of cases, even professors who start out thinking that distance has no implications for them beyond having a camera in the classroom are unlikely to continue to believe that. Therefore, it may make sense for DCE to invest some resources for faculty development in distance pedagogies. Rather than have faculty reinvent the wheel or engage in a hit or miss strategy of individual innovations, distance teaching staff might be introduced to some of the commonly used techniques for distance courses. This could imply providing things like a distance teaching workshop, individual work with faculty or an ongoing space where interested professors could discuss with other distance faculty the results of their own innovations. While these offerings should be voluntary, there does appear to be demand from within the ranks of current distance faculty.

Making professors aware of the ongoing research on effective distance teaching might also shift some faculty away from the pole of the spectrum that sees distance as a second best option for education. One observant student, who has had extensive experience with distance courses at DCE, echoed our own perceptions of faculty attitudes and suggested how this had an impact on the course:

I think that the pedagogical approach taken by some distance learning professors can make the entire course less effective, not just the distance learning part of it. I have had professors who, perhaps subconsciously, see the Internet as an inherently limited medium for education, while at the same time believing that in-class and distance students should all have the same experience. As a result, they limit the educational techniques the employ to those that they believe can work over the Internet. Thus, both the tools they use and their interactions with students are less dynamic they I think they would be if the class were taught to live students only. For example, I can think of three professors that I have had, all of which were involved in distance education, who prepared for class by writing up detailed notes about what they wanted to say and then, for the most part, read their notes verbatim as the content of their lectures. This is hardly effective teaching, whether it is in a distance environment or not…In short, in the cases where distance learning is less effective, I believe the in-class experience is also less effective, because the professors involved see distance education as a limiting factor. But I also believe that the extension school has some good example of classes and professors who offer very effective distance education.

Those professors who view the Internet as an inherently limited and second best option for education seemed to avoid attempts at innovations to get a better sense of the interests and strengths of the distance students. Put another way, assumptions associated with distance as intrinsically one-sided and limited diminishes the willingness to push against whatever impedes a dialogue with the distance students. Showing faculty how to overcome those barriers might not only shift their view of the possibilities of distance education, but also produce a more satisfying experience for their students.

Attending to Students and the Role of the TF

We saw in the previous chapter how important it is for students to feel that the teaching staff are attentive to them. As noted above, the courses examined in this report, and in the distance program generally, are large. The selection to introduce distance education at DCE in large courses was a policy decision, since economically it makes sense to invest in those. From the point of view of attentiveness to students, traditionally professors of large courses delegate much of the work of attending to students to their TFs. A number of structures typically exist to ensure that student needs are met in a uniform and conscientious manner. These include having a head TF, developing problem sets for sections, common grading guidelines, TF meetings, etc. Clearly to re-invent or transpose these aspects into cyberspace requires a carefully crafted plan, including developing the capacity to deal with queries, run cybersections, moderate discussion groups and supervise chat rooms.

Interviews with faculty show how, at first, the shift to distance can threaten to upset a course’s ability to provide attention to students by putting new demands on the TFs. The interviews suggested that the TF’s role is changing both in terms of workload and necessary experience. For the faculty not to feel overwhelmed, the TFs must take on the bulk of the work associated with distance: answering additional amounts of email, keeping up the website, answering students’ questions about problems with connectivity that seem to multiply as the complexity of the website grows. For some professors, the technical aspect of the distance courses, such as setting up or adding to the website, is not something they want or intend to learn how to do. In one case, the fact of having highly paid, professional TFs who have been with the course for over a decade and who are extremely computer literate, meant that going distance did not mean much additional work for the faculty, since the TFs took on the additional work. In another case, the TF was only somewhat more competent than the professor in terms of computers and this was part of the tremendous amount of extra work experienced by the professor and his inability to be responsive to all students.

The increase in email is not simply due to the fact that this is how students at a distance must communicate rather than talking to the professor after class or during office hours. It appears to be a shift born of the new possibilities for learning and reflection offered by the technology. As students are able to reflect on the lectures and collect their thoughts, they have different kinds of questions. They tend to be more reflective and at a deeper level, and this is in part driving the increase in email traffic. Faculty, on the one hand, are pleased by this type of response; on the other hand, it does imply much more effort.

Another TF-related issue that arises in the distance courses is recruiting. There are two separate issues faculty members identified. First, because much of the work is distributed onto the TF’s, familiarity with computing and especially website management becomes an important skill for TFs to possess. In the Computer Science courses, this is less problematic than in others because it is both more likely that potential TFs have it and professors are less dependent on the TFs because they themselves often have the skill. Outside of CS, it is a much more serious concern because TFs are expected to have both computer skills and content knowledge of the subject matter. A number of professors expressed the concern that this shrinks the potential pool of TF recruits. Since distance courses can accommodate many more students than purely classroom-based ones, this could present a bottleneck in the future should courses significantly grow in size. Some courses solve the problem by

45 Distance Education Evaluation Division of Continuing Education, Harvard University having what are in essence professional TFs, who stay with the course many years and manage its cyber aspects, but this is unlikely to be a feasible solution for the majority of courses since these professional TFs are paid extra apart from their DCE remuneration.

A second recruiting issue was mentioned by some of the distance only professors. In the traditional recruitment model, TFs are drawn from different sources: graduate students, undergrads and former students. As courses move to the hybrid model and the pure distance model, some faculty members feel that they cannot develop enough rapport and trust with cyber students to be able to offer them teaching fellowships, which would entail supporting students at a distance. Some viewed this as a real problem for the distance-only course in terms of the future supply of TFs. Not all faculty felt that way, however. Some professors have felt comfortable recruiting distance students and have had good experiences doing so even without ever having met the TF in person. In these courses, the professor was sufficiently involved and familiar with the students’ work to get a sense of how they think and their problem solving skills. Helping professors figure out how to develop a sense of their distance- only students as individuals or how TFs can bring potential recruits to the attention of professors would address this problem.

The Creation of Community

Many professors worried about how to draw students who learn at a distance into the classroom community and felt that it did not necessarily happen. In part this concern stemmed from their own desire to know who those anonymous students are. They also thought that being part of a community is essential to the learning process. One faculty member spoke of the experience for distance students as “terribly isolating… you don’t see that the people next to you are confused and so you don’t ask the question. The professor may miss it (the confusion), but the others in the class will not.” In the absence of these almost silent clues that come from being together in a group, the fear is that distance students lose out on part of the pedagogic process that is interactive. The face-to-face environment does promote a kind of buddy system where students gain courage to voice their questions, concerns and feelings because they feel supported by others. Distance students do not have those clues as to when it might be appropriate/legitimate to interrupt. If cyberspaces are not created for them that allow them to exhibit their confusion in a safe setting, they may lose out.

Building community in cyberspace with the goal of recreating the traditional classroom environment is an area where a great deal of research is currently being conducted.24 The lessons from the research include knowing who your students are, having occasions in which you can get close to their learning needs and overall humanity; in other words, providing the kind of attentiveness that is the hallmark of good teaching. Again, here it is clear that DCE distance faculty could benefit from more support of the pedagogical type.

24 It is also the subject of major funding efforts. The Sloan Foundation, for example, is supporting work on the creation of on-line communities through its Asynchronous Learning Network grants.

Conclusions In this chapter we have outlined the responses of the faculty to their experiences with going distance. Overall, it has been extremely positive for these trailblazers and they are generally happy with the support they receive from DCE. There are a number of lessons from our findings that we offer here in the form of recommendations:

 Develop incentives for faculty to teach distance courses. Based on our interviews, we have described the faculty members who were among the first to teach a distance course as trailblazers. They saw benefits to the courses either in terms of some long-term professional payoff or simply were enchanted by being at the forefront of a new innovation. In the future, other faculty members may need more encouragement than that. Indicating to them how their experience with distance at DCE may help them in their other courses or home departments or working with professors to help them develop new skills are all potentials incentives to attract more faculty to the distance program.

 Plan for a multi-stage roll out for new distance courses. One of the key findings of this chapter is that a course learning curve exists as teaching staff get used to the technology, both in terms of the technical aspects and of the pedagogical ones. Courses that are planned as distance should first develop a good website and slide presentations. Distance should be then undertaken in a subsequent semester. This suggests a one- to two-year time line for taking a course to distance.  Provide faculty with more support. There are two areas where some centralized support from DCE would make sense and make the distance program easier from the point of view of the faculty. The first type of support has to do with the copyright issues. Figuring out permissions in a constantly evolving legal environment is something that is beyond the ability of many faculty members given their time constraints. A second type of support, for those who feel they need it, is to work with faculty members on adapting their teaching styles and classroom performance to the distance mode. The third type of support would be assistance with working with electronically mediated environments. This includes bulletin boards and electronic section meetings. This would be useful for both community building among true distance students as well as enhancing the learning of all students using those features of a class.

Appendix A: Student Questionnaire Harvard Division of Continuing Education Distance Education Survey

Thank you for your willingness to participate in this survey of Extension School courses. Your responses are very valuable and we appreciate your taking the time to answer the following questions. Even if you did not use the distance features of your course, it is very important to us to hear from you. Please answer all the following questions as completely as possible (except when directed to skip a question). Put N/A (Not Applicable) if the question does not apply to your course. Note that any information which identifies you will not be made available to instructors. Thank you.

Your name

Your e-mail address

1. a) Name of the course that you are taking (If you are taking more than one course, choose the course that requires the most effort.) ______b) I give researchers permission to access my grade for the course selected above with the understanding that this information will remain confidential and used for research purposes only. ____ No (0) ____ Yes (1)

2. Please indicate your student status: a) Are you taking this course as part of a degree or certificate program? ____ No (0) ____ Yes (1) b) Are you taking this course for: ____ Noncredit (1) ____ Undergraduate credit (2) ____ Graduate credit (3)

3. Have you taken any Extension School courses before? ____ No (0) ____ Yes (1) How many? _____

4. Have you taken any Distance Education courses before, either at the Extension School or elsewhere? ____ No (0) (Skip to question 6) ____ Yes (1)

5. In general, how satisfied have you been with other distance education courses that you have taken? ____Very dissatisfied (1) ____Somewhat dissatisfied (2) ____Somewhat satisfied (3) ____Very satisfied (4)

Please explain:

6. Is the Extension School course in which you are currently enrolled available over the Internet only?

____ No (0) ____ Yes (1) (Skip to question 8) 7. Would you have taken the course in which you are currently enrolled if it had only been available in a pure distance format, i.e., no live lectures or sections, videos of the lectures only, and contact with the teaching staff via email and telephone?

____ Definitely not (1) ____ Probably not (2) ____ Probably yes (3) ____ Definitely yes (4)

8. Would you have taken the course in which you are currently enrolled if it had only been available in a classroom-only format?

____ Definitely not (1) ____ Probably not (2) ____ Probably yes (3) ____ Definitely yes (4)

9. All else being equal, what course format would you prefer if choosing a course in the future? (Please choose one response) ____ Classroom only (1) ____ Classroom plus Internet video (2) ____ Internet video only (3) ____ Does not matter (4)

10. Did you consider courses with similar content at any other institutions when choosing this course? ____ No (0) (Skip to question 12) ____ Yes (1)

11. What made you decide to take the course at the Extension School?

12. We are interested in knowing what influenced your decision to pursue further education. Please indicate how important each of the following reasons was to you when searching for courses or programs. Use a five point scale where one (1) is not at all important and five (5) is very important.

Not at all Very Important Important a) Professional enhancement in your current career 1 2 3 4 5 b) Interest in changing careers 1 2 3 4 5 c) Personal enrichment 1 2 3 4 5 d) Pursue a degree 1 2 3 4 5

13. We would also like to know what factors influenced your decision to take this particular course rather than another. Please indicate how important each of the following was in choosing this course.

Not at all Very Important Important a) General prestige of Harvard 1 2 3 4 5 b) Reputation of the instructor 1 2 3 4 5 c) Recommended to you by someone you know 1 2 3 4 5 d) Schedule was convenient 1 2 3 4 5 e) It was available via the Internet 1 2 3 4 5 f) Cost 1 2 3 4 5 g) Convenient location 1 2 3 4 5

14. We are interested in the specific factors that students find appealing in taking courses over distance or courses that have a distance component. For each of the following, please indicate how important the item was in your decision to take this class. Use a five point scale where one (1) is not at all important and five ( 5 ) is very important. If you did not use the Internet videos, please mark NA.

Not at all Very Important Important a) The convenience of not having to be physically present in class 1 2 3 4 5 NA b) The flexibility of viewing the class when it fits my schedule 1 2 3 4 5 NA c) The ability to review classes via the Internet 1 2 3 4 5 NA d) The chance to go back and reflect on the lecture or class discussion 1 2 3 4 5 NA e) A less pressured environment 1 2 3 4 5 NA

15. How important to you is the grade you receive in this class? ____ Not important (1) ____ Somewhat important (2) ____ Very important (3)

16. Please indicate who is paying the tuition for this course: ____ Self (1) ____ Employer (2) ____ Other (3)

17. How far from campus do you live: ____ Under 10 miles (1) ____ 10-25 miles (2) ____ Outside Boston area (3)

____ Outside New England (4) ____ Outside USA (5)

18. If your course offered live lectures, during this semester, how many class sessions did you attend on campus? ____ None (1) ____ 1-3 (2) ____ 4-6 (3) ____7-10(4) ____ more than 10 (5)

____ No live lectures offered (9)

19. Approximately how many hours per week did you dedicate to work on the course (not including time spent in class and/or watching videos of the lecture)? ____ Hours

20. Approximately how many hours per week did you spend watching videos of the lecture? ____ Hours ____ Did not watch videos of the lectures (Skip to question 26)

21. What was the most important reason that you watched the lecture videos? (Please indicate one) ____ as the primary way of taking the course (1) ____ to reinforce material learned in class (2) ____ to make up for missed classes (3) ____ to prepare for exams or quizzes (4)

21a. What was the second most important reason that you watched the lecture videos? (Please indicate one) ____ as the primary way of taking the course (1) ____ to reinforce material learned in class (2) ____ to make up for missed classes (3) ____ to prepare for exams or quizzes (4)

22. What type of Internet connection did you typically use to view the lecture videos? ____ 56K phone modem (1) ____ Cable or DSL line (2) ____ Other (3)

23. How would you rate the overall technical quality of the course's video capabilities including its access, sound and visual quality, pause and playback capacities, etc.? 1 2 3 4 5 NA low quality high quality

24. How well, overall, did the video allow you to follow the instructional process? (Slides or notes visible, ability to follow class discussions, etc.) 1 2 3 4 5 NA not at all well very well

25. How important to your learning and understanding the course material was the ability to watch the lectures on video? Not at all Very Important Important 1 2 3 4 5 NA

26. If you attended classes on campus, were you bothered by having a camera and a technician in the classroom? ____No (1) ____Yes, somewhat (2) ____Yes, a great deal (3)

27. Please indicate how important you found each of the following items to learning and understanding the course material. If you did not use the capability or your class did not have it, please mark NA.

Not at all Very Important Important

a) Using the website to read or download course materials 1 2 3 4 5 NA b) Watching on-line section meetings 1 2 3 4 5 NA c) Attending live section meetings 1 2 3 4 5 NA d) Participating in on-line discussion groups 1 2 3 4 5 NA e) Emailing your professor or TA with questions 1 2 3 4 5 NA f) Emailing other students with questions or to do projects 1 2 3 4 5 NA g) Meeting other students in person to address questions 1 2 3 4 5 NA or to do group projects

28. Please indicate how useful you found each of the following aspects of this course's website (1 to 5 scale for each where 1=not very useful, 5= very useful). Mark N/A if you did not use it or the website did not have it. Not very Very Useful Useful a) Links to other sites of interest 1 2 3 4 5 NA b) Being able to download readings and course materials 1 2 3 4 5 NA c) Course announcements 1 2 3 4 5 NA

29. Did this course have a bulletin board? ____ No (0) (Skip to question 32) ____Yes (1) ____ Don't Know (9) (Skip to question 32)

Not very Very Useful Useful 30. How useful did you find it to post items or respond to other people's postings? 1 2 3 4 5 NA

31. How useful did you find it to read other people's postings? 1 2 3 4 5 NA

32. For each of the following general statements, please indicate your level of agreement. Use a 5 point scale where one (1) is strongly disagree and five (5) is strongly agree. If you don't know or the statement is inapplicable to your situation, please mark NA.

Strongly Strongly Disagree Agree a) In terms of student learning, I think it is important for the student and professor to be in the same room. 1 2 3 4 5 NA b) In general, I learn at least as much from the discussions and comments of other students as I do from listening to a professor lecture. 1 2 3 4 5 NA c) Generally, I prefer to work in groups rather than on my own. 1 2 3 4 5 NA d) I generally learn better by having someone explain something to me than I do by reading books or articles. 1 2 3 4 5 NA e) I have excellent time management skills. 1 2 3 4 5 NA f) I consider myself an extrovert. 1 2 3 4 5 NA g) Being able to discuss my point of view in class is essential to my learning. 1 2 3 4 5 NA h) I am better at getting my points across by talking to people rather than in writing. 1 2 3 4 5 NA i) I am better at getting my points across in writing than I am by talking to people. 1 2 3 4 5 NA

33. How would you rate your overall degree of competence using computers? Please use a five point scale where one (1) is low and five (5) is high.

1 2 3 4 5 low high

34. Please rate the degree to which you enjoy using computers. Use a five point scale where one (1) is low enjoyment and five (5) is high enjoyment.

1 2 3 4 5 low high

35. Imagine that you are working on your computer when you encounter the problem of not knowing how to do something within the program you are using or in writing a program. Which one of the following would be your likely first response to the problem?

____ Consult the manual (1) ____ Seek out on-line or electronic help (2) ____ Consult a third party reference book (i.e.Web Publishing for Dummies) (3) ____ Ask a friend or colleague (4) ____ Use what you do know to find a way around the problem (5) ____ NA (9)

36. For each of the following statements about this course, please indicate your level of agreement. Use a 5 point scale where one (1) is strongly disagree and five (5) is strongly agree. If you don't know or the statement is inapplicable to your situation, please mark NA.

Strongly Strongly Disagree Agree a) Students taking this course at a distance invariably miss out on part of the learning experience. 1 2 3 4 5 NA b) The professor in this class cared about my progress. 1 2 3 4 5 NA c) The instructor's presentations were enthusiastic and interesting. 1 2 3 4 5 NA d) I felt that I had sufficient opportunity to participate in this class. 1 2 3 4 5 NA e) It was easy to obtain the required materials. 1 2 3 4 5 NA f) My professor answered my email questions in a timely manner. 1 2 3 4 5 NA g) The teaching assistant answered my email questions in a timely manner.1 2 3 4 5 NA h) I received feedback on my work in a timely manner. 1 2 3 4 5 NA i) The assignments and examinations were well chosen. 1 2 3 4 5 NA j) The assignments and examinations allowed me to gauge my progress. 1 2 3 4 5 NA k) The professor addressed the needs of distance students appropriately. 1 2 3 4 5 NA

37. Taking into account the time and effort involved with the distance components of this course compared with a traditional course, would you say that the distance components allowed you to learn: ____ much more effectively (1) ____ somewhat more effectively (2) ____ about the same (3) ____ somewhat less effectively (4) ____ much less effectively (5) ____ NA (9)

38. In the future, what is the maximum amount of additional tuition you would be willing to pay to take a course that featured a distance component, i.e. videos of the lectures available over the Internet? ____ 0 (0) ____10% (1) ____ 20% (2) ____ 30% (3)

39. Would you recommend this class to someone else? ____ Definitely ____ Probably ____ Probably ____ Definitely not (1) not (2) yes (3) yes (4)

40. Overall, how would you rate this course? Please use a 5-point scale where one (1) is low and five (5) is high. 1 2 3 4 5 low high

Finally, we would like to know some general information about you.

41. Please indicate your gender: ____ Male (1) ____ Female (2)

42. Please indicate your age: ____18 – 25 (1) ____ 26 – 35 (2) ____ 36 – 45 (3) ____46 – 55 (4) ____over 55 (5)

43. In terms of your current employment, are you: ____ Not currently employed (1) ____ Employed part time (2) ____ Employed full time (3) (Skip to question 45)

44. In a typical week, how many hours do you spend at your job? ____ Hours

45. In a typical week, how many hours do you spend on other work such as other course work, child care, volunteer work, job search, etc.? ____ Hours

46. Please indicate the highest level of education you have attained: ____ High school (1) ____ Associate degree (2) ____ BA or BS (3) ____ MA or MS (4) ____ Graduate or Professional degree (5)

47. Would you be willing to be interviewed individually and completely confidentially after the term is over about your experiences in this class? ____ No (0) ____ Yes (1)

Appendix B: Interview Guide for Faculty

Good Morning/Afternoon. I’m ______and I’m conducting some research on the Extension School’s distance education courses. The purpose of this is to understand better how they work and to use that information to improve distance offerings. My colleague and I are interviewing instructors to get their opinions about distance teaching and their experience with it. Course Title ______Course Number ______

Distance only course? ___ Yes ___ No

Had you taught this course or some variant of it before? ___ Yes ___ No

Had you taught this course over distance before? ___ Yes ___ No

Did the Extension School approach you about teaching over distance or was it your idea? ____ Extension School What was appealing about the idea, why did you agree?

Did you have any fears or apprehensions?

____ My idea Why did you want to do a distance course? Distance component?

Did you have any fears or apprehensions?

If distance component: do you think you would do a distance only version? (If yes, make sure you get back to this question at the end)

Before you taught the distance course, did you talk with other instructors who already had experience with distance teaching? ____ Yes ____ No

If yes: Did they offer any interesting or useful advice? If no: Is there any particular reason why not?

Before you taught the distance course, were you already using computer enhancements for your course such as a well-developed website, electronic discussion groups, etc? If distance only, did you start with distance component? ____ Yes ____ No

If yes to website, would you have considered doing the distance component if you had not already had the website and electronic materials

What, if any, do you think are the main benefits for the instructors of distance teaching? (Full or part time)

What, if any, do you think are the main drawbacks for the instructors of distance teaching? (Full or part time)

Let’s talk a little bit about the students now.

What do you think are the main benefits for students of taking a distance course? What, if any, do you think are the main drawbacks for students of taking a distance course? In this particular course? Do you make any special effort to acknowledge that there are distance students taking the course? Do you or the students in the classroom find the camera or other aspects of the distance technology obtrusive?

Compared to the non-distance version or for the distance only students, have you made any changes to the way you teach. Probes for: different style of presenting the material, giving different types of assignments, arranging discussion groups, office hours Other

If yes to any probe: How well do you think these innovations worked? Please explain As a professor, were there any surprises or unexpected pedagogical discoveries you found in teaching the course over distance versus your non-distance experience? That is, did you find that some types of activities or assignments worked less well than you expected or conversely, that some things worked much better?

For distance only: In terms of the responses or behaviors of students in the class, did you find any differences between this class and the non-distance version that you think are attributable to the fact that it was over distance?

Part Distance: Have you take note of how distance students interact differently? Any conjectures?

Developing materials. How involved were you with developing the website and other on-line materials for the course?

How satisfied were you with the website25, including its ease of use, content, appearance, utility in furthering your goals for the class, etc.? Please explain. How satisfied were you with how the technical aspects of the distance course such as students being able to access and use the distance features etc.? Please explain.

Did the Extension School provide you with the support you needed to make the distance part of the class run smoothly? Please explain.

Would you say that teaching the course over distance required you to do more work for the course than you normally would have, less work or about the same? Was the work of a different nature?

Did the distance aspect of the course change the role of the TF in any way or your interactions with him or her? Probe with:  does he or she spend more time answering email,  troubleshooting technical problems;  do you need to rely on him or her more for feedback about students etc.

Do you feel that any characteristics of the distance course either allowed you to monitor the progress of the students better than a traditional course or impeded your ability to do so?26

If and when you teach this course again in the future, what changes or innovations would you want to make to the course, either ones you do yourself or that the Extension School could do?

If part distance, would you consider 100% distance? If no, why not? Under what conditions yes?

Which of the following did this course have? ___ Links to other websites ___ Links to resources such as journal articles or papers ___ On-line discussion groups or section meetings Who monitored this? ______

Let’s step back for a moment and let me ask you to think about your general aims and goals for students in this class. Can you explain to me what it is that you want to accomplish, what you

25 Here we should find out how one evaluates websites for educational purposes. Let’s ask someone from CTE…. 26 This question is about teachers starting to use the IT affordances to inform them about student performance.

59 Distance Education Evaluation Division of Continuing Education, Harvard University want them to come away with? Does the distance component affect your ability to do that in any way or does it change your goals in any way?

Background Now we just have a few final questions about background information.

Gender ____ Male ____ Female

How many years teaching experience do you have ___

Do you come from: ____ Extension School ____ Harvard College Faculty ____ Faculty of another university ____ Private sector ____ Non-profit sector

How would you rate your overall degree of competence using computers? Please use a 5 point scale where 1 is low and five is high. 1 2 3 4 5

Please rate the degree to which you enjoy using computers. Use a 5 point scale where 1 is low enjoyment and 5 is high enjoyment. 1 2 3 4 5

Have you ever designed or help design a website for yourself, someone else, a course or an organization? ____ Yes ____ No