Observing the Sun at Mcdonald Observatory

Evaluation Report

Observing the Sun at McDonald Observatory

Executive Summary

Background: This project funded April 2005-March 2007 through IDEAS developed and provided programming for using equipment donated to McDonald Observatory’s Visitor Center to provide students with first-hand experiences of observing the sun, both those who physically visit the Observatory and those who make an electronic visit via an individual classroom videoconference.

Strategy: The recent availability of videoconference equipment in every Texas school district together with the donation of equipment to the Observatory's Visitor Center allowed McDonald Observatory’s Visitor Center to begin an innovative program of communication with classrooms and schools using the power of technology to “to do things in education that we were previously unable to do.”

Project Evaluation Methodology included 1) documentation of formative feedback about qualities of the activities and materials and 2) interviews with teachers to evaluate use and perceived qualities and effects of the videoconferences.

Results and Conclusions: Members of the project team and Advisory Board provided formative feedback using the structured assessment tool to report the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. o As a set, the eight products—e.g., “Conference DVD segments on observatory tour/solar telescope operation/careers”—were evaluated to be strongly consistent with the 5E model. All of the products were indicated to be at least moderately strong across the 5Es. Strength was especially notable for Engage, Explain, and Evaluate. o Inspection of item responses for attributes directly descriptive of the project’s stated objectives for the programming showed each of the products with exceptional strength relative to at least one of the stated objectives. Across the set of products, there was exceptional strength documented for each of the stated program objectives that were paraphrased on the assessment tool. The seven teachers who participated in the interviews brought their students to the Live from McDonald Observatory Videoconference “Observing the Sun” from all levels of schools in small towns all over Texas and beyond. o The most frequent theme in the teachers’ responses to the interview questions was that they “loved” the Observing the Sun videoconference experience and/or perceived it to be effective. o The teachers indicated awareness of and access to the pre- and post-visit suggestions and materials, but they made relatively more use of the pre- than of the post-visit resources. o Teachers indicated the videoconferencing technology enabled them do things in education they were previously unable to do—e.g., “I love technology and incorporating technology…It’s about as good as it gets to have an astronomer to literally teach your class one day!”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 1 of 19 Observing the Sun at McDonald Observatory

Goals of this project funded April 2005-March 2007 through IDEAS were to: o develop and provide effective programming on the sun for K-12 students and teachers who visit McDonald Observatory o develop and provide effective programming on astronomy, including solar viewing, for students and teachers who make visits to McDonald Observatory electronically through individual classroom videoconferences o create a team of teachers, science education staff, and scientists to develop these materials o share both the content and the process of creating these programs with other observatories involved in similar outreach programs

Context: During 2005, several donations of equipment were made to expand and enhance the K-12 educational science program at the McDonald Observatory Visitors Center in Fort Davis, Texas. This project developed the programming for using the donated equipment to help teachers address major deficiencies that students experience with items in the Texas Assessment of Knowledge and Skills (TAKS) about characteristics of the Sun and other objects in the sky. Through this programming students are provided with first-hand experiences of observing the Sun, both those who physically visit the Observatory and those who make an electronic visit via an individual classroom videoconference.

Objectives for the programming were stated as follows: o Students discover connections between astronomy and their school’s science curriculum. o Students learn about the relationship between science and technology through contact with instruments and their use. o Facilitator(s) engage students’ prior knowledge and understanding based on pre-visit materials. o Facilitator acts as a co-investigator with the students and employs guided inquiry to focus students’ attention on driving questions and concepts. o Students reflect on the experience through use of post-visit materials.

Strategy: Activities and supporting materials for student field experience were developed consistent with the 5E model for promoting student inquiry as a process for learning. The project engaged an advisory board of teachers, science education staff, and scientists and used feedback from formative evaluation to help find the right mix of video segments, interactive segments, and engaging questions that promote student response so that students who participate in Observing the Sun via video conference don’t feel they are just “watching TV.” The recent availability of videoconference equipment in every Texas school district together with the donation of equipment to the Observatory's Visitor Center allowed McDonald Observatory’s Visitor Center to begin an innovative program of communication with classrooms and schools using the power of technology to “to do things in education that we were previously unable to do.”1 Products developed included Pre-conference/Pre-visit sun assessment, Demonstrations (on DVD), Live (and pre-recorded) images of the sun and commentary, Viewing the sun with solar telescope, and Post-visit suggestions. Products specific to Student Field Experience were Use of astroscan and the Large sun and word wall. Products specific to videoconference were DVD segments on observatory, conference hand-outs for students, and conference teacher guides.

1 O’Neil, H. & Perez, R. (2003). Technology Applications in Education. NJ: Lawrence Erlbaum Associates

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 2 of 19 Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 3 of 19 Evaluation Methods

Two evaluation studies were conducted to: (1) help improve the project; and (2) communicate results to a variety of stakeholders. The study protocol was approved by the Institutional Review Board (IRB) of the University of Texas at Austin.

Study 1 provided documentation of formative feedback about the activities and materials developed for “Observing the Sun at McDonald Observatory.”

o Design: In spring 2006 and again in spring 2007, the project team and the Education Advisory Board used a structured assessment tool to evaluate extent to which each product (e.g., video segment) was consistent with program objectives and requirements (e.g., has features to help students discover connections between astronomy and their school’s science curriculum).

o Instruments: The assessment tool was adapted from narrative presented in the “Observing the Sun” project description and from checklists and rubrics for assessing consistency with the 5E model for inquiry-guided learning.2

o Analyses: Multiattribute evaluation (MAE) procedures were applied to aggregate the data. Separate reports were prepared for each product. A summary report then was prepared to document strengths and recommend improvements across program components. The appendix to this final evaluation report includes a copy of the assessment tool and the separate reports compiled from the data collected spring 2007.

Study 2 used qualitative methods to evaluate acceptance, use, and perceived effectiveness of the program. o Design: Interviews were conducted with teachers who participated in the videoconference programs during the period April 2006 through January 2007. We attempted an interview with each of the 12 teachers who participated in “Live from McDonald Observatory – Observing the Sun” videoconferences with their students during the selected time period and completed interviews with 7. We did not interview teachers who participated with their students on-site at McDonald Observatory because weather and other conditions resulted in only 3 such teachers for the selected time period. o Instruments: The interviews were conducted by telephone. Questions asked where the observatory experience fit into the class curriculum, how students prepared for their experience with the observatory, whether and how the experience helped students learn about and have contact with scientific technology and instruments, and what kinds of follow-on activities or consequences there were. The appendix to this final evaluation report includes a copy of the script for the interview.

o Analyses: With the participant’s permission, the telephone interviews were recorded, transcribed, and coded for thematic analysis.

2 The underlying rubrics are available to view on line (www.unc.edu/destiny/5Es.htm and www.thetrc.org/edu/doc/5E/5ERub0304.pdf ). The 5E Model is attributed to Bybee R (1997) Achieving Scientific Literacy: From Purposes to Practices, Heinemann: Portsmouth, NH.

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 4 of 19 Results of Study 1

Strengths of Products Developed for “Observing the Sun” Engage Explore Explain Elaborate Evaluate 5E Pre-visit sun assessment √√ √ √ √ Conference DVD sgement on Observatory √√ √ √ √ √ Viewing the sun with solar telescope √√ √ √ √ √ Large sun and word wall √√ √√ √√ √ √ Handouts for student use during the experience √√ √√ √ √ √ √ DVD of demonstraton video clips √√√ √ √√ √ √ √√ Show earth's rotation using Astroscan telescope √√√ √√ √√ √ √√ Post-visit suggestions √√ √ √√ √ √√ √√ Members of the project team and Advisory Board provided formative feedback using the structured assessment tool to report the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. During February and March 2007, assessments were submitted by 9 individuals. Several of the respondents made narrative comments that their evaluations in March were based on the “revised” descriptions and materials. The 5E phases are Engage, Explore, Explain, Elaborate, and Evaluate. The assessment tool is multi-attribute. The four attributes listed for the Engage phase, for example, are “creates interest,” “generates curiosity,” “asks questions,” and “elicits responses from students that uncovers what they know and think.” Attribute response options are “strongly agree,” “agree,” “disagree,” and “disagree strongly.” Because some of the phases have five attributes and some only four, MAE procedures recoded the item responses such that “strongly agree” = 100, “agree” = 67, “disagree” = 33, and “disagree strongly” = 0. Missing responses were not included in the aggregate scores calculated across attributes. Aggregate scores in the range 64 to 76 were accepted as evidence of moderate strength; 77 to 89 indicated strength; and 90 to 100 indicated exceptional strength. As a set, the products were evaluated to be strongly consistent with the 5E model. All of the products developed in the “Observing the Sun” project were indicated to be at least moderately strong across the 5Es. “Handouts for student use during the experience,” “DVD of demonstration video clips”, and “Post-visit suggestions” were products indicated to be at least moderately strong in each of the 5E phases. Strength was especially notable for Engage, Explain, and Evaluate across all of the products. The “DVD of demonstration video clips” and the activity and materials for “Show earth’s rotation using astroscan telescope” were indicated to have exceptional strength for Engage. Elaborate was the only phase where strength was not common across the activity descriptions and supporting materials developed for the “Observing the Sun” project. Inspection of item responses for attributes directly descriptive of the project’s stated objectives for the programming showed each of the products with exceptional strength relative to at least one of the stated objectives. Across the set of products, there was exceptional strength documented for each of the stated program objectives that were paraphrased on the assessment tool. Additional detail can be viewed in the individual product reports (Appendix B).

Strength in addressing Objectives as stated for the programming Astroscan Conf DVD Handout Demo Wordwall Post-visit Pre-visit Viewing Uncovers what they know & think √√√ √√√ √√√ √√√ √√√ √√√ √ Uses contact with instruments √√√ √√√ √ √ √ √√√ Connect astronomy to science curriculum √ √√ √√ √ √√ √ √ Encourages students to reflect √ √ √√√

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 5 of 19 Results of Study 2

Interview respondents were generally representative of the population of teachers who participated with their students in “Observing the Sun” Videoconferences April 2006 through January 2007 Date of Grade Level Students Interview Conference Taught Participating Location of the School Completed Spring 07 Elementary 16 Central TX √ Spring 07 Elementary 45 North TX Spring 07 Elementary 26 West TX √ Spring 07 High School 18 Central TX Spring 07 High School 26 Central TX Spring 06 High School 30 East TX √ Fall 06 High School 16 Outside TX √ Spring 07 Middle/JH 45 Central TX Spring 07 Middle/JH 28 Central TX √ Fall 06 Middle/JH 45 Central TX √ Spring 07 Middle/JH 48 North TX √ Spring 06 Middle/JH 30 South TX

Teachers who participated in the interviews, like the population of teachers whose students participated in Live from McDonald Observatory Videoconference “Observing the Sun” during the period April 2006 through January 2007, came from all levels of schools all over Texas and beyond. Attempts were made to reach all of 12 of the teachers whose students participated in the Videoconferences Live from McDonald Observatory. All 12 were from schools located in small or very small towns. We were unable to reach 4 of the teachers by telephone. One teacher declined the interview. The time and thoughtful responses the teachers gave to help evaluate the “Observing the Sun” project and identify options for continuing to improve the reach and the effectiveness of McDonald Observatory’s student field experiences in science and technology is very much appreciated. We promised to keep the interviews brief. The teacher’s time on the phone ranged from 7 to 20 minutes. The briefest interview was with a language arts teacher who had taken her class to the scheduled videoconference in lieu of the science class originally scheduled that, instead, went to a science camp that day. The most frequent theme in the teachers’ responses to the interview questions was that they “loved” the Observing the Sun videoconference experience and/or perceived it to be effective. They said: “It was excellent. I really enjoyed it and I learned from it and the kids enjoyed it…It was an awesome experience for this kids …” “The information was very beneficial.” “The kids were tickled…they loved it.” “They were really excited…you could see it on their face…maybe this is something I could be interested in…you know, for their future.” “It was exciting for the students…they learned a lot…it was very unique…it was great!” “It was good and I would definitely do it again.” “I really enjoyed it and the kids really enjoyed it…I think that field trip really helped them to understand and they’ll remember it better…If that question comes up on the TAKS, they’ll remember, ‘I had that field trip.’” The fit with TAKS was another frequent theme. One of the teachers said, “We brought in the 4th grade so they will be ready for the 5th grade TAKS…” Another said, “There are questions about the sun on the TAKS, even if its not a lot of questions…[this experience] actually answered some questions.” Others said: “It was for my science class. We had been talking about the solar system. This was able to fit it and explain more. Also it was part of TAKS preparation;” “…the

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 6 of 19 sun and how it fit into the different stars…one of our TEKS is star life cycle;” and “We had studied astronomy and as you know astronomy is part of the TEKS in 8th grade…”

o The most frequent theme in the teachers’ responses to the interview questions was that they “loved” the Observing the Sun videoconference experience and/or perceived it to be effective.

o For some, the videoconference was a planned enrichment for their advanced placement and gifted and talented students. For others, it was a culminating activity for a science unit. And for others, it was an opportunity that developed at the “last minute.”

o Irrespective of how or why they elected to participate, the teachers found many positive aspects of the experience and were especially impressed with the interactive capabilities of the video technology.

o The teachers indicated awareness of and access to the pre- and post-visit suggestions and materials, but they made relatively more use of the pre- than of the post-visit resources.

o A final theme touched on how the videoconferencing technology enabled the teachers to do things in education they were previously unable to do.

Although teachers provided these kinds of strong endorsements of the programming and of its relevance in their science instruction, another common theme in the interview responses was the sometimes opportunistic circumstances that brought their class to the “Observing the Sun” videoconference. The last minute or happenstance nature of the event may have constrained its capabilities for promoting student learning. Three of the 7 teachers made comments such as “We were not currently talking about space or solar system, but it is something we have talked about in the past. When the opportunity became open, I wanted [the students] to be able to see it…It was a time slot that another school had set up and I just took the class that I had at that time. It was extremely last minute…the students found out only about 5 minutes before we went.” Another said, “Our 5th graders were on a field trip that day, we brought in the 4th grade …and we had a 6th grade classroom in there too… We didn’t really interact or anything…We were just kind of watching…The science teacher wasn’t there for the video...I’m a language teacher, so...” And the third said, “They were not as prepared as I would have liked for them to have been because this was a last minute deal.” For some of the teachers, the videoconference was a planned enrichment for their advanced placement and gifted and talented students (GTP/AP). One said, “I have five classes of GTP/AP students it just so happened that the time slots that were available were with classes that I thought could benefit a lot…” Another said, “I wanted my gifted students and the higher kids to do something different…We just grabbed students from all over the 4th grade that were excelling and we wanted to give them something extra.” Others took students to the videoconference because, “It was a nice end to our unit…a culminating experience.” Irrespective of how or why they elected to participate, the teachers found many positive aspects in the experience and were especially impressed with the interactive capabilities of the video technology and the way it brings the observatory experience to students who might otherwise never visit an observatory. o One said, “I was surprised how impressed the kids were. I think of these kids as being technology literate, and they were so surprised that [the facilitator] could see them and

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 7 of 19 talk to them. …In their comments, the kids said to him things like ‘I am the one sitting in front in the red shirt.’ It was like the contact with [the facilitator] was so personal….They really felt like he was there with them.” o Another said “We had never done a distance learning with that group of 6th graders, so just seeing they could actually talk to someone that was somewhere else and ask questions and get answers was real eye-opening for them….We live in a rural area, and so it was very good they got to have that experience…They were able to see the telescope and talk to him… everyone at some point got to ask questions and answer questions…It was very good.” o Another said, “[The students] were able to answer questions when [the facilitator] asked about the black spots on the sun…The nice thing was he was able to expand, show them an example—His “hands-on” to show why the sun has sun spots…That made it more clear to the kids what a sunspot actually was.” o Another said, “[The facilitator] was great in getting the kids involved…He had questions for them and they had questions for him and it was just great, interactive...Kept it going, back and forth.” o Yet another commented, “This was the very first video teleconference our school had had…The students knew that and the district sent people out weeks ahead and we had sent it out to the news media….[The students] knew they were participating in something special, and that was a real experience with technology in itself…They were also able to see the instruments [the facilitator] showed them….They were very interested as he walked around the observatory and pointed out different parts of the telescope that we had talked about in our unit on the foundations of astronomy….He had the students engaged the entire time.” o And another indicated that “although we had a lot of technical difficulties the day of the conference…just the exposure of seeing the observatory, seeing the various instruments, exposure to having a videoconference, and [the facilitator] doing demonstrations on the sun, showing sun spots” were examples of student learning from the experience. o And the last said, “[The facilitator] was able to show the actual activity [and] they were able to actually see…It was like hands-on…they weren’t actually touching it, but they were watching him do the investigative part of it.” The teachers indicated awareness of and access to the pre- and post-visit suggestions and materials, but they made relatively more use of the pre- than of the post-visit resources. One teacher said, “I didn’t have a lot of follow-up because afterwards we just went into the next project….We did watch the intro DVD as a pre-program…We had studied astronomy and we had done some of the activities to prepare for the presentation. It was more of a culminating event than it was in the curriculum because when we were done with it we immediately went on to something else…One of the things that was so positive is that the videoconferences was such a motivating statement for the activities we did with the kids [before the conference]….It motivated the kids in preparation for the conference…They studied hard for it.” Another commented, “After the experience, I was in the middle of teaching health and so, to be honest, I have not done a lot with the information….But I do have all the materials it suggested we do for the follow-up.” Another said, “We just finished our talking about the solar system a little more, and then we had an assessment, and then we had to move on to something else….But we had done the pre- activities they had on the website….They did a drawing of the sun and then they were able to see and to talk [to the facilitator] about how their drawings were similar to or different from what they actually saw and talked about.” Another commented, “The observatory had a link on their site and I could get those activities…[The students] developed their own questions before the conference…We missed the first 3 days of the following week because of an ice storm and we never went back and did any follow-up…We were meaning to do that…”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 8 of 19 One of the teachers did, however, report “There were some follow-up activities the observatory had sent and we participated in…We also went to the observatory website ‘What are Astronomers Doing?’ and they clicked to look what different astronomers were there when we were doing the conference and what kind of research they were doing…They were very interested in the work about ‘The extra solar planets’…They were pretty excited about that.” A final theme touched on how the videoconferencing technology enabled the teachers to do things in education they were previously unable to do. One teacher said, “[the facilitator] was able to explain in more depth [than I could]…he was able to answer right off the top of his head where they could understand it better…We couldn’t actually go to the observatory, but [the students] could get the information from the observatory [where] they are more expert and could answer questions that I couldn’t answer.” Another said, “From the area that we’re coming from, it is low SES kids and so a lot of them will not be able to go to an observatory or anything like that….It gave them an opportunity to go to the observatory without leaving the classroom…they were just really excited.” Another noted, “We are so far from the observatory…It is not like we’ll ever be able to do a field trip…But this way, they were able to have the experience, see the sun through the telescope.” Another said, “Just to know the observatory is there…It is within driving distance—even though they might not go there—just exposure to what an observatory does.” A nice conclusion to this report is the observation of the teacher who said, “I love technology and incorporating technology…It’s about as good as it gets to have an astronomer to literally teach your class one day!”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 9 of 19 Appendix A – Instruments

e e e y e e e l r r r g g g g n

Please give us your opinions about characteristics of the program element named: a a

A o s i r y t l D g S e n

“ ______“ e o r r t g S a s i d

Creates interest

Generates curiosity

Asks questions

Elicits responses from students that uncovers what they know and think

Uses students’ previous experiences as a basis for explaining concepts

Provides opportunities for students to describe objects and events

Encourages students to ask questions

Presents problem(s) for students to solve

Encourages students to work together

Allows students to connect astronomy to their school’s science curriculum

Uses contact with instruments to connect science and technology

Encourages students to explain concepts in their own terms

Asks students for justification or evidence to support their explanations

Encourages students to communicate their explanations to others

Encourages students to test their explanations

Reminds students that there are alternative explanations

Encourages student to reflect on their observatory experience

Refers students to data as a resource for elaborating what they know

Builds critical and logical thinking skills

Allows the teacher to observe and assess students’ knowledge and skills

Allows students to assess their own learning

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 10 of 19 Telephone script for interview with teachers

Hello. My name is xxxxx with the Observing the Sun project at McDonald Observatory. I’m calling to ask you to help us identify ways to ensure that student experiences with the observatory are as beneficial as possible. We appreciate that you have agreed to participate in this evaluation. With your permission, I am going to record this interview. I have six questions to ask and expect that the interview should not take more than about 20 minutes. Is this agreeable to you?

These questions are about the experience of the group of xxxx students in grades xxxx that (came to the observatory) (participated in the videoconference) on xxxxx.

1. Where did the observatory experience fit into your curriculum? (prompts: Did it serve to introduce a unit on the sun? OR help teach basic content? OR provide a culminating activity? OR provide extracurricular enrichment? OR serve some other purpose?)

2. How did you decide which students to bring to the observatory experience? (prompts: Was it an entire classroom? Or was it a select group of students? If select, how were they selected?)

3. How did students get prepared for their experience with the observatory? (prompts: Were there particular materials or resources that helped students get ready? If so, what were they? What are some examples of things that happened (at the observatory) (during the videoconference) that let you know the students were OR were not well prepared for the experience?)

4. In what ways did the event help your students learn about and have contact with scientific technology and instruments? (prompts: Did you observe changes in student understanding? If so, please give me an example. Did the event give your students the opportunity to explore “guided inquiry” to frame research questions? If so, please give me an example? How well did the facilitator engage students in a “co-investigator” role?)

5. Tell me a little about what happened during the weeks after the experience with McDonald Observatory. What kinds of follow-on activities or consequences were there? (prompts: Did the post-visit materials or resources help students link their observatory experience to their classroom learning and/or help them to review and reflect on the event? If so, what are some examples? What did the students talk about? If no, what would have been helpful for linking the observatory experience with classroom learning?)

6. Would you do this (conference) (observatory visit) again? Why or why not?

THANKS very much for your help. Please feel free to call Cindy Roberts-Gray at 409/762-2499 if you have other comments or questions about this evaluation of the Observing the Sun project.

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 11 of 19 “Observing the Sun at McDonald Observatory” Formative Feedback Report Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted are paraphrase of statements of the project objectives.

Product: Show the earth’s rotation using astroscan telescope to project image of the sun Total 5E score: 79 on 100 point scale Creates interest 100 Engage Generates curiosity 100 98 Asks questions 100 Elicits responses from students that uncovers what they know and think 91

Provides opportunities for students to describe objects and events 90 Explore Encourages students to ask questions 95 88 Presents problem(s) for students to solve 91 Encourages students to work together 67 Uses contact with instruments to connect science and technology 95

Uses students’ previous experiences as a basis for explaining concepts 90 Explain Allows students to connect astronomy to their school’s science 76 curriculum 82 Encourages students to explain concepts in their own terms 81 Asks students for justification or evidence to support their explanations 81

Encourages students to communicate their explanations to others 67 Elaborate Encourages students to test their explanations 57 53 Reminds students that there are alternative explanations 43 Refers students to data as a resource for elaborating what they know 43

Encourages student to reflect on their observatory experience 52 Evaluate Builds critical and logical thinking skills 81 72 Allows the teacher to observe and assess students’ knowledge and skills 81 Allows students to assess their own learning 72

Exceptional strength of the “Show earth’s rotation using astroscan telescope to project an image of the sun” is its capabilities for engaging the learner (score of 98 on the 100 point scale). Additional strengths are the product’s capabilities for promoting the learner’s own involvement in the Explore, Explain, and Evaluate phases of the 5E model for promoting student inquiry as a process for learning (scores of 88, 82, and 72).

Elaborate (score of 53) is a phase of the 5E model where the product could be strengthened and/or where it needs support of other components of the “Observing the Sun at McDonald Observatory.”

Product: Conference DVD segment on observatory tour, telescope, and careers Total 5E score: 69 on 100 point scale Creates interest 92 Engage Generates curiosity 96 83 Asks questions 79 Elicits responses from students that uncovers what they know and think 63

Encourages students to reflect on their observatory experience 71 Evaluate Builds critical and logical thinking skills 71 71 Allows the teacher to observe and assess students’ knowledge and skills 71 Allows students to assess their own learning 71

Strength of the “Conference DVD segment on observatory tour, solar telescope operation, and careers” is its capabilities for engaging the learner (score of 83 on the 100 point scale), especially its capabilities for generating curiosity (scores of 96 on the 100 point scale). The product also was evaluated to have moderate capabilities for involving the learner in the Explore, Explain, and Evaluate phases of inquiry- based learning (scores of 68, 70, and 71).

There is considerable room for improvement and/or need for support from other products or components of the “Observing the Sun at McDonald Observatory” project in Elaborate phase of the 5E model for promoting inquiry as a process for learning.

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted in blue are paraphrase of statements of the project objectives.

Product: DVD of demonstration video clips, Teacher/Facilitator resources Total 5E score: 77 on 100 point scale Creates interest 92 Engage Generates curiosity 92 94 Asks questions 96 Elicits responses from students that uncovers what they know and think 96

Exceptional strength of the “DVD of demonstration video clips, Teacher/Facilitator resources” is its capabilities to Engage the learner (score of 94 on the 100 point scale). Another strength is the product’s capabilities for promoting the learner’s own involvement to Explain concepts (score of 86). The product also was evaluated by the project team as having moderate capabilities for involving the learner in the Explore, Elaborate, and Evaluate phases of the 5E model for promoting inquiry as a process for learning.

Product: Handouts for student use during the conference or visit Total 5E score: 76 on 100 point scale Creates interest 82 Engage Generates curiosity 85 86 Asks questions 85 Elicits responses from students that uncovers what they know and think 93

Strengths of the “Handouts for students’ use during the conference or visit” are its capabilities to Engage the learner (score of 86 on the 100 point scale) and involving the learner in the Explore and Evaluate phases of inquiry-based learning (scores of 85 and 81). The product also was evaluated to have moderate strength in the Explain and Elaborate phases of the 5E model for promoting inquiry as a process for learning (scores of 65 and 64).

Product: Large sun and word wall Total 5E score: 75 on 100 point scale Creates interest 81 Engage Generates curiosity 86 86 Asks questions 81 Elicits responses from students that uncovers what they know and think 95

Strengths of the “Large sun and word wall” are its capabilities to Engage the learner (score of 86 on the 100 point scale) and promote the learner’s own involvement in the Explore and Explain phases of inquiry-based learning (scores of 78 and 80). The product also was evaluated by the project team to have moderate capabilities for promoting in the Evaluate phase (score of 73).

Elaborate (score of 56) is a phase of the 5E model where the product could be strengthened and/or where it needs support of other components of the “Observing the Sun at McDonald Observatory.”

Product: Post-visit suggestions Total 5E score: 80 on 100 point scale Creates interest 79 Engage Generates curiosity 75 84 Asks questions 88 Elicits responses from students that uncovers what they know and think 92

Strengths of the “Post-visit suggestions” is its capabilities to Engage the learner (score of 84 on 100 point scale) and promote the learner’s own involvement to Explain concepts and to Evaluate the learning experience (scores of 81 and 89). The product also has moderate capabilities in the Explore and Elaborate phases of the 5E model for promoting inquiry as a process for learning (scores of 73 and 72).

Product: Pre-visit sun assessment Total 5E score: 69 on 100 point scale Creates interest 88 Engage Generates curiosity 84 88 Asks questions 88 Elicits responses from students that uncovers what they know and think 92

Strength of the “Pre-visit sun assessment” is its capabilities to Engage the learner (score of 88 on the 100 point scale), specifically, its capability for eliciting responses from students that uncovers what they know and think (score of 92 on the 100 point scale). The project team also indicated the product has moderate capabilities for promoting the learner’s involvement in the Explain and Evaluate phases of inquiry-based learning.

Areas where the product could be strengthened or where it needs support of other products or components of the “Observing the Sun at McDonald Observatory” project are the Explore and Elaborate phases of the 5E model for promoting inquiry as a process for learning.

Product: Viewing the sun with solar telescope Total 5E score: 72 on 100 point scale Creates interest 96 Engage Generates curiosity 92 87 Asks questions 84 Elicits responses from students that uncovers what they know and think 75

Strength of the “Viewing the sun with solar telescope” is its capabilities to Engage the learner (score of 87 on the 100 point scale). The project team also indicated the product has moderate capabilities for promoting the learner’s involvement in the Explore, Explain, and Evaluate phases of inquiry-based learning (scores of 74, 73, and 74).

Elaborate (score of 51) is a phase of the 5E model where the product needs support of other products or components of the “Observing the Sun at McDonald Observatory.”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 19 of 19