Podcasting Lectures Formative T Some Point in Their Educations, Help Them Better Retain the Biomedical Students Must Learn Copious Information Presented
Total Page：16
File Type：pdf, Size：1020Kb
Podcasting Lectures Formative t some point in their educations, help them better retain the biomedical students must learn copious information presented. evaluation Aamounts of information. To do The Office of Dental Informatics is this, they use a variety of well-known responsible for the development, imple- strategies helped strategies such as study groups, note- mentation, and evaluation of learning- taking services, and videotapes of lec- technology activities in the school. This identify a solution tures. In fall 2004, a group of first-year includes faculty development, course Web dental students at the University of pages, research with and about learning to a learning Michigan (U-M) School of Dentistry technology, and the formative and sum- asked to have all dental school lec- mative evaluation of learning technol- dilemma tures videotaped and recordings made ogy projects. The office is also respon- available on a Web site. The students’ sible for the acquisition, installation, and doubted their ability to accurately operation of classroom and computer lab By Sarah Brittain, summarize in their notes the quantity equipment. Pietrek Glowacki, of information presented in lectures. In response to the students’ request, the Jared Van Ittersum, The students thought that reviewing a Dental Informatics group applied forma- and Lynn Johnson video recording of each lecture would tive evaluation strategies to determine 24 EDUCAUSE QUARTERLY • Number 3 2006 the ultimate solution. The group deter- views and focus groups. This information Figure 1 mined that podcasting (see the sidebar) guides design decisions. audio recordings of lectures provided a Instructional Design The Design stage consists of creating a better technology solution for the stu- Process prototype, flowcharts, and storyboards. dents’ needs than the originally requested Feedback is gathered by assessing the user video recordings. audience’s attention, comprehension, information retention, personal involve- Formative Evaluation and ment, and user-computer comfort.4 These Instructional Design data guide revisions during production Instructional design and formative eval- of the product, potentially eliminating uation strategies are commonly used in costly and time-consuming changes at a developing instructional products, espe- later stage if the product does not meet cially for computer-based instruction and user needs. other learning technologies. Formative During the Development stage, the evaluation provides information to help product is created, tested, and revised monitor and improve product develop- until the client is satisfied.5 Formative ment to ensure that it meets its intended evaluations occasionally are used outside goals.1 tion. When trying to determine whether of the design, development, and imple- Instructional design consists of several cognitive or behavioral skills have been mentation stages for a product. Often it stages (see Figure 1). In each stage, design- affected, tests are generally used. Occa- is difficult to find funding, resources, and ers use formative evaluation techniques to sionally records and documents, such as support for the development of a product obtain feedback concerning the product server access log files, are used to gather without results showing that it will effec- from clients, subject matter experts, col- information on the frequency of down- tively accomplish its intended goal.6 The leagues, and learners.2 Feedback gathered loads—data that can give an idea of how results of a formative evaluation can serve via formative evaluation in one stage of much a product is being used. as an indicator of a product’s success with the instructional design process is used In the Define stage of instructional its intended audience, which in turn can in subsequent stages to help improve the design, developers begin defining the tip the scale toward securing resources product. Formative evaluation informa- scope of the learning activity, identifying needed for further development. tion is collected in four ways: self-report- learner characteristics, establishing con- Instructors often hesitate to integrate ing, observation, tests, and records.3 straints, and collecting resources. During new products or technology into their Self-reporting, the most commonly used this stage, the learning technology team courses without evidence that it will method, refers to users’ directly relaying members, who have limited knowledge benefit student learning. Information their experiences with or opinions of the of the subject matter and the intended retrieved from a formative evaluation can product, generally by means of a ques- audience, commonly make many of the help them determine whether a product tionnaire, survey, or interview with an design suggestions. They gather infor- should be implemented. evaluator. Observing users’ behavior and mation about the target audience’s prior Once the product has been deployed interaction with the products is another knowledge, interests, and experiences in its intended setting, formative evalua- popular method for collecting informa- with the subject matter through inter- tion can serve several different purposes. Developers can use feedback to make small improvements that were not antici- pated during the initial development, for Podcasting example. Feedback can also help steer Podcasting is a new technology with an evolving definition. The term is gener- future iterations of the product. If the product will be used in a different envi- ally considered to be derived from combining the words iPod and broadcasting. ronment or with a different audience, Podcasting involves making audio and video files available for download on a information gathered during this part of routine basis via subscription. the evaluation can guide reconfiguration Two important characteristics set podcasting apart from downloading audio of the product. and video from a Web site: (1) the routine and regular addition of new content; and (2) the ability to automatically receive new content that you select through Formative Evaluation: subscription. To subscribe to a podcast requires only a single interaction in which A Case Study Formative evaluation strategies are the user chooses to have updates downloaded automatically. Podcasts, like broad- used routinely when developing com- casts, deliver new content regularly. Wikipedia provides a thorough discussion of puter-based instructional programs podcasting (http://en.wikipedia.org/wiki/Podcast). such as patient simulations or tutorials. When students requested that lectures Number 3 2006 • EDUCAUSE QUARTERLY 25 be videotaped and made available on a tion 3. We examined questions 4, 5, and ing the survey. For records, we looked at Web site, they didn’t get an immediate 6 across all three pilots. server logs to gauge the frequency with decision to grant or deny the request. Certain constraints placed on the proj- which students used each of the three A research university such as the U-M ect contained costs and ensured student media types. Finally, records were kept of values inquiry. This culture made it involvement. First, students were respon- the time spent by the technical staff on natural to apply strategies to system- sible for obtaining instructor permission the project in order to calculate cost. atically determine whether videotaping to record lectures. Second, students had was the best solution. We conducted a to provide their own playback devices. Pilot 1 Results. This pilot focused on series of three pilot studies, using forma- The school provided technical support. A determining the best media format for tive evaluation and instructional design university grant from the provost’s office lectures (question 1). The three formative techniques to guide the process. Flagg7 funded the few additional expenses. evaluation measures—student survey, described four types of formative evalua- focus group, and server logs—showed tion measures, two of which we used in Pilot 1—Media Format that students preferred the audio-only this project—self-reporting and records. The focus of this pilot was to answer the format. Of a possible 105 participants, No tests were administered, nor were question, What is the best media format 30 downloaded media directly from the observations of students conducted. for lecture review? Web server. Server logs revealed that 20 Students initiated and supported the percent of downloads were video, 14 project, so were equal partners from the Pilot 1 Methods. The Advisory Group percent audio synced with PowerPoint, beginning in the formative evaluation selected part of a microbiology course for and 66 percent audio only. The average process. An Advisory Group consisting the pilot because of the difficulty of the time from posting date to download was of five dental students and one repre- content and the dependence on diagrams 16.2 hours. The nature of electronic files sentative from Dental Informatics was and other visuals during the presentation. makes it impossible to determine usage formed to direct the project. The staff Faculty permission was obtained to record among those who obtained the media member supported the project’s techni- the lectures and post the electronic in other ways (from a friend burning a cal activities. presentation files on the course Web site. CD, for example, direct file transfer from The Advisory Group decided to con- The microbiology course met three another user, or media used in groups). duct a pilot in single course, with the