9/21/2012
Tutored Video Instruction
• Video of expert teacher
• Presented with the aid of the facilitator
• Theory: Interaction with the video better than ‘just watching TV’ Aug 24, 2012 From Digital Aug 24, 2012
•
Initial work: Gibbons, Stanford, 1977. • Tutored Videotape Instruction StudyHall to Digital • University of Washington • Introductory programming offerings to Community Colleges
• Recorded UW lectures used by existing community college
Microsoft Research Microsoft Research India Microsoft Research India PublicHealth instructors Richard Anderson Department of Computer Science and Engineering University of Washington, Seattle, USA 1 2
Digital StudyHall Digital StudyHall Hubs
• Founded by Randy Wang, ex-Princeton • Lucknow
• Collaboration with Urvashi Sahni, Studyhall School • Pune
• Top private school in Lucknow • Calcutta Aug 24, 2012
• DSH Methodology Aug 24, 2012
• Bangalore • Create videos of classes taught by expert teachers to low-income students • Bangladesh • Train teachers to facilitate videos • Pakistan • Target rural schools • Nepal
• Randy Wang timeline
Microsoft Research Microsoft Research India Microsoft Research India • Start DSH independently • Join MSRI • Depart MSRI, continue DSH independently • Leave India for Intel Shanghai 3 4
Does facilitated video Digital StudyHall instruction work? methodology
• Lesson videos recorded
at hub school • Use of interactive
pedagogy
Aug 24, 2012 Aug 24, 2012
• Careful matching of students • Tie to regular curriculum • Training of facilitators
• Facilitation model Microsoft Research Microsoft Research India Microsoft Research Microsoft Research India • Mix of video use and activities NSF Funded project conducted by University of • Cost realism in Washington 5 technology deployment 6 2008-2011
1 9/21/2012
Project domain: Primary Does it work? education in rural India
• Study of eleven • Does the use of Digital StudyHall improve learning outcomes?
government schools in Uttar Pradesh, India • Pre-test / Post-test performance of control and treatment classes
• Rural schools outside of • Does Digital StudyHall lead to pedagogical change in the Aug 24, 2012
large city Aug 24, 2012 school?
• Introduction of • Observe differences in classroom activities between DSH and methodology to new non-DSH classes schools • Does Digital StudyHall improve indicators such as attendance
or hours of instruction? Microsoft Research Microsoft Research India Microsoft Research Microsoft Research India • Is Digital StudyHall sustainable? • Do schools continue to use DSH on their own with modest supervision and support resources 7 8
The main result Caveats
There are significant obstacles to scalability and • We are not claiming generality of the result
sustainability of Digital StudyHall in Government • Other types of schools? Other geographies?
primary schools in Uttar Pradesh, India • Favorable evaluation from teachers and students Aug 24, 2012
Aug 24, 2012 •
Two of eleven schools wanted to continue after the end of the study • We believe that the vast majority of government schools • Example of a successful DSH school from outside of the study would not continue to use DSH without frequent monitoring area
visits • Rural government school in Uttar Pradesh Microsoft Research Microsoft Research India • Our outcome testing was inconclusive, yielding no meaningful Microsoft Research India results
9 10
Study Design Failure in testing
• 11 schools (out of 20)
selected from the block we
Even if the use of Digital StudyHall produced substantial had permission to work in • Schools had not previously improvements in learning, we would not have been able
worked with DSH to measure it
Aug 24, 2012 Aug 24, 2012
• Introduction through principals and teacher training workshops • Study design rd th • Regular observation visits • Alternate use of 3 grade English and 5 grade Math as control • Interviews or treatment class
• Year one: • Why testing was inconclusive Microsoft Research Microsoft Research India • Pre-test/Post-test with controls Microsoft Research India • No culture of evaluative testing in the classroom • Year two: • Low attendance Pre • No testing • Gradual start/end, holidays, missing tests • Expanded observations • High variability in school performance and student Mid 11 12 performance Post
2 9/21/2012
Relationship with Student view of video students implementing organization
• Implementing organization: StudyHall, well regarded private • Content creation
school • Lessons by expert
• Implementation team primarily Indian teachers Aug 24, 2012
• Adequate buy-in from principals Aug 24, 2012 • Poor, urban students
• With official permission • Students identified • Teachers generally favorable to implementing organization with video students • Trainings, involvement with StudyHall and other teachers a • The students said the
positive Microsoft Research Microsoft Research India Microsoft Research Microsoft Research India video students were • Students had a very positive view of implementing “just like them.” organization • Validation of video • Novelty creation strategy • Disruption 13 14
Facilitator understanding and Facilitator buy in to use of methodology methodology
• Interactive pedagogy • Mixed level of
• Activities adoption by teachers • Poems, songs, drama • Issues
• • Urban/Rural split Aug 24, 2012
Intervention had goal Aug 24, 2012
of changing teaching • Lack of incentives to • Training of facilitators teach and monitoring visits • Alternate usage • Observations models
• Video lessons for review Microsoft Research Microsoft Research India confirmed use of Microsoft Research India interactive techniques • Facilitated by studens • Unexpected • Difference between • Teachers viewing lessons regular teachers and in advance at home 15 para-teachers 16 • Education levels
Sustainability challenges Counterpoint: Bhal Vidyalaya
• Evidence of non- • School has been using
sustainability DSH for six years • In year one, level of use • Limited monitoring
was strongly correlated • School purchased second Aug 24, 2012
with monitoring visits Aug 24, 2012
set of equipment • Long interruptions of • Regular use of DSH in many electrical supplies to classes schools, where schools did little to restore • Full buy in from school
power • Struggled with DSH Microsoft Research Microsoft Research India • Some schools did not Microsoft Research India introduction want replacement equipment after theft • Eventually, championed • General problems with by principal teacher absenteeism 17 • Rural school 18
3 9/21/2012
Evaluating technology DSH Update interventions
• Study took an intervention to “next level” of scale • Randy Wang’s involvement severely limited
• New deployment in eleven schools over two years • External support from MONA Foundation
• Challenging environment, but with very high impact if successful • Existing assets for DSH Aug 24, 2012
• We were naïve to think we could measure educational Aug 24, 2012 •
Lucknow based organizations (~6 people) outcomes • Technical capacity in video production, distribution, and use • Randomized trials are out of reach for this type of intervention • Base in StudyHall school • There were conflicts between the research project and • Content library
teaching the kids • Several sustained DSH deployments
Microsoft Research Microsoft Research India Microsoft Research India • Example – study staff would give lessons to the kids when the • Hired Lucknow based manager teachers were absent • Shift in strategy • Recognition of a complex system where a technology • DIETs – teacher training colleges intervention is a modest part 19 20 • Girls residential school
DStar • Developed community video education for Agriculture
• Original vision – Apply Digital StudyHall methodology to • Started as a project at Microsoft Research
multiple domains • Formed NGO, currently reaching 1200 villages • Scaling with NRLM to reach 10000 villages
• DSH Methodology • Digital Green model Aug 24, 2012
• Locally created video Aug 24, 2012 • Livelihood education conducted by community facing organizations
• Facilitated use • Agricultural messaging developed by experts • Members of the community filmed demonstrating or talking about the • Hub and spoke model techniques • Content archive • Videos reviewed for content and quality
• Digital Green – MSRI project by Rikin Gandhi • Videos shown by facilitators in community group meetings Microsoft Research Microsoft Research India Microsoft Research Microsoft Research India • Digital Polyclinic – DSH Lucknow efforts in health
21 22
Digital Public Health PATH
• PATH / UW / Digital Green collaboration • Seattle based NGO working in health technologies
• Application of Digital Green methodology to maternal health • Founded 1977
• One year pilot starting August 2012 • Now working in 70 countries Aug 24, 2012
Aug 24, 2012 •
• Goal: demonstrate feasibility Program for Appropriate Technology in Health • Approximately 1000 employees world wide, 400 in Seattle • Five offices in India
Microsoft Research Microsoft Research India Microsoft Research India
23 24
4 9/21/2012
SureStart: Improving maternal and child Key aspects of Digital Green health in India • Digital Green supports field based organizations using community • Community engagement to support
video in their agricultural programs maternal and newborn health
• Key innovations of Digital Green . Governance and public health interventions • Model for community creation of video content . Mentoring support for accredited social health
activists (ASHAs) Aug 24, 2012
• Model for facilitated uses of videos in the community Aug 24, 2012
• Bill & Melinda Gates Foundation supported • Refined video production pipeline and associated technology . Maharashtra and Uttar Pradesh, India • Emphasis on measuring performance . 2006-2011 • Platform for community education in multiple domains • Focus Village ASHA at a mothers’ group meeting in UP
. Raising awareness around essential maternal and
Microsoft Research Microsoft Research India Microsoft Research India newborn care . Scaling up efforts to cover the entire rural and peri- urban populations of the designated districts . Supporting household and community actions to 25 26 complement other initiatives focused on clinical services and health systems
Platform for Digital Green and PATH: Digital Public Health low-cost community video education Goal: To test the feasibility of adapting the Digital Green model to health
PATH and Digital Green partnership developed
Video based education where content Digital Green staff provided training to PATH staff in Bhopal, India in
is both created and presented by the August 2011
Aug 24, 2012 Aug 24, 2012
community PATH transferred technology to ongoing project in Udaipur, India to • Localization of content and support midwives in providing rural home-based healthcare messaging Video training and dissemination taking place in August 2012 • Community engagement and
empowerment
Microsoft Research Microsoft Research India Microsoft Research India Enabled by low-cost consumer digital video technology • Videos created locally within two days to accommodate the local 27 language and specialize the 28
Source: www.digitalgreen.org treatment of the topics to the area videos
Digital Public Health: Feedback from rural midwives Piloting Digital Public Health Pilot objectives • More complete information • Develop a strategy for implementation
• Bridges language barriers • Assess and evaluate feasibility of integrating mobile videos into routine
community-based support • Patients understanding by seeing rather than Digital Green, PATH, Sure Start Partnership telling
• Community based partner, Gram Vikas Sansthan
Aug 24, 2012 Aug 24, 2012
• Education beyond patient/provider interaction • Bachhrawan Block, Raebareli District, Uttar Pradesh, India Focus on maternal health messaging using SureStart materials • Video extends/supports authority of provider Develop methodology in 20 villages • Identification as powerful tool in video messaging
“In video when we show the Microsoft Research Microsoft Research India Microsoft Research Microsoft Research India patient, the patient “If there is a nurse…who is understands that there is one very young, the mothers don’t “The best thing is the video. I more patient like me in the take her seriously… who is don’t explain by talking only I video …so she feels that ok she telling us this… by show the video to the patient she is also a patient like me showing the video, there is an and it is very helpful to let her and how I need to take care of elder woman in the video, and understand.” 29 30 myself and my baby.” so they think ok, so there is some doctor telling us this.”
5 9/21/2012
Digital Green and Digital Public Health Technology Stack Program Component Digital Green Digital Public Health Program structure Field based organizations Field based organizations
with agricultural mission with health mission • Direct implementation of Digital Green digital video approach
Message creation Agriculture experts Health experts and • Goal: no technology innovation
governing bodies • Basic components: Aug 24, 2012 Content creation Videos produced by field Videos produced by field Aug 24, 2012
•
organization and farmers organization, health Kodak video cameras workers and community • Microsoft MovieMaker for editing Content Delivery Facilitated use of video in Facilitated use of video in • Pico projectors for screenings community groups; use of groups and one-on-one •
pico-projectors settings Youtube for content distribution Microsoft Research Microsoft Research India Content Management Central video repository Central video repository Microsoft Research India Quality control Review videos for Multilevel review including accuracy and content details of messaging before use Reporting Dissemination and Metrics to be determined 31 32 adoption statistics
Dissemination model Message creation
• Mothers’ groups • Problem: balance community input with medical accuracy
• Established in Surestart project • Current process:
• ASHA’s meet monthly with women to teach key health messages • Group of messages created by area by experts Aug 24, 2012
• Women attend while pregnant and for a short time after baby is Aug 24, 2012 • Birth preparedness: Record phone numbers for emergency transport
born • Breast feeding: Begin feeding within one hour of birth • Add video screenings to existing group meetings • Community resource person creates storyboard • May also use Self Help Groups and Village Health and • Storyboard reviewed
Sanitation days • Video shot by community resource person Microsoft Research Microsoft Research India • Experiment with one-on-one showings using mobile phones Microsoft Research India • ASHAs on household visits
33 34
Video review M&E Plan
• Need to ensure messages are accurate • Can we adapt the DG model to support maternal health
• Give correct health advice messaging
• Compatibility with official guidelines • Focus on process indicators Aug 24, 2012
• Health messaging may require stricter review than agricultural Aug 24, 2012 • Don’t be too ambitious – we need to understand how the
messaging model works before looking for outcomes • Project has set up a Community Advisory Board to review videos
• Multistage review with video check list Microsoft Research Microsoft Research India • Important component to get right: Microsoft Research India • Establish procedures that will give credibility to the project • Maintain efficiency in production of videos 35 36
6 9/21/2012
Measuring Impact Next steps
• Health outcomes would require large scale deployment over a • Phase 1
long term • Conduct pilot to develop model and demonstrate feasibility
• Attributing adoption of specific practices to the intervention is • 20 villages, one year Aug 24, 2012
difficult Aug 24, 2012 • Phase 2
• Working in a high functioning block (so indicators already • Demonstrate impact
good) • 80 villages, two years
Microsoft Research Microsoft Research India Microsoft Research India
37 38
Thanks!
Aug 24, 2012
Microsoft Research Microsoft Research India
39
7