What Is the Intellectual Merit of the Proposed Activity?

Reviewer 1 Rating: Good What is the intellectual merit of the proposed activity?

The project team: Insect specialist; SD Dir.; and NMSU computer tech.

Partnerships: NY Haudenosconee tribes and SD Sioux tribes; 4H NY; 4HSD; afterschool programs; migrant workers; early childhood programs; summer programs; and others.

Delieverables: Publish results on website, and journals. Develop culturally based student activites. Develop image analysis.

Project Design: Identifying and classifying ladybugs. Building on Cornell's "citizen science" bird project. Train undergrads and grad stuents to field test with 4H groups, summer programs and afterschool programs. Kids will learn to collect, count, take pictures, identify and classify.

What are the broader impacts of the proposed activity?

Strategic Impact/Audience: Impact Native Americans who are 1st and 5th graders. They will be in the following programs: 4H in NY and SD; tribal after school, summer, and early childhood. Grad and undergrad students will be trained to train others to do real science research that would be classifying ladybugs.

Public Audience Impact: Children, family and community will be comfortable doing science. Children directly experience thematic science. Children understand the importance of presentation.

Evaluation: Pre- and post assessments of basic knowledge; surveys and interviews. Dr. Lewenstein will evaluate.

Summary Statement

Working with Native American 1st and 5th graders who will bring in their families is a great idea. Working with butterflies in their environment is culturally relevant. Having them do real science research accomplishes STEM standards. NY and SD are pilot tribal groups that could later expand to other tribal nations. This builds on the "citizen science" project that Cornell did with birds. This will allow them to tap into a bigger project that is mapping this data throughout the world. The questions they will be trying to answer and other questions that will be developed are incredible.

I'm not clear how they will recruit and train undergrad and grad students? I'm not clear how they will train program coordinators who work with the kids. And, I'm not clear how kids will learn to present their results to the general public other than the website. Reviewer 2 Rating: Very Good What is the intellectual merit of the proposed activity?

Cornell University, "Have you spotted me: Learning lessons by looking for ladybugs," targets Citizen (Science) Scientists 5-6 (first grade) and 10-11 (fifth grade) years old ûNative Americans from rural areas New York and South Dakota. Deploying a train-the- trainer model, Cornell expects to train 4H youth educators in numerous counties in NY and SD at their annual statewide meetings. Who, in turn, will train others to achieve an impact over two states. Over three years, this project is estimated to impact nearly 10,000 children.

The centerpiece of this project is ladybugs. Students will observe, collect data, investigate ladybugs to solve scientific questions e.g., Who [species] is really out there? How are they [lady bugs] impacting each other? Etc. Using interactive web sites ûappropriate for the age range, ladybug ID guides and learning modules, databases and more, Cornell's undergraduate and graduate students would conduct educational/sampling programs and demonstrate at 4H groups.

What are the broader impacts of the proposed activity?

This project has the potential to impact a wide array of states and students and is aimed at Native Americans, rural, farming and disadvantage communities. The technology components ûinteractive web site and databases appropriate for first graders, is unique.

The collaborative partners include NY and SD state 4H, Onondaga Nation School, Seneca Nation Early Childhood learning Center, Migrant Workers Children's Summer Program and more.

Summary Statement

This proposal is creative, detailed and scientific. I find this proposal to be of high quality and logically grows from previous educational experiences and research. The evaluation plan is flawless --piggybacks on Cornell Laboratory of Ornithology evaluation.

There is a concern that this ladybug project lacks detailed information related to the full scope of participation for the undergraduate and graduate students.

Reviewer 3 Rating: Fair What is the intellectual merit of the proposed activity? Cornell University is seeking $1,790,704 for a "Have you spotted me: Learning lessons by looking or ladybugs" project. This proposed four year initiative will develop a "citizen science" program as a tool for increasing science knowledge and scientific literacy. While the subject matter, ladybugs, may have an strong appeal for the targeted audience, the proposed project lacks intellectual merit.

Proposal Strengths: - The qualifications and the expertise of the project director is good - The science topic of ladybugs is fun and easy for the targeted audience - The project has the potential for providing the target audience an authentic science experiment - The project has collected a wide variety of letters of support - There is a current and active advisory board already in place

Proposal Concerns: - The written proposal itself is sloppy. Research citations are missing. There is inconsistent formatting, strange wording, missing sentences and notes intended only for the authors. The poorly written and submitted proposal indicates that this project was not clearly thought out but rather hastily put together. - This proposal needs more attention to how this project will be implemented. It is not entirely clear how the target audience will be recruited and selected. - The project will include orientations and trainings for 4-H youth development educators from every county at their scheduled statewide meetings. However, the letters of support from the state 4-H program leaders does not indicate that these trainings have been collaboratively planned. Furthermore, providing training for 4-H educators does not ensure that the program will be implemented. There is no plan for providing incentives or motivators for 4-H educators to implement this project in their county. 4-H Educators are often overworked and overwhelmed with their current responsibilities. The likelihood that they would start a new program in their county is small without some plan for motivating them. Nor is there any follow up support for these educators once they are trained. - Likewise this project indicates that 4-H educators will then train individual leaders of 4- H youth groups, after school programs, and nature centers. Again there is no plan for providing incentives for current 4-H leaders to add another new project to their volunteer work. Why would a 4-H horse leader or a 4-H clothing leader be interested or willing to lead 1st and 5th graders in a search for ladybugs? - There is no plan for how 5th graders will be trained and supported to mentor the first graders - The proposal states that rural poor and native youth are so far removed from the process of science that it has no relevance for them. There is no research citation for this statement and it's very likely that Native Americans would strongly disagree with this statement. - The proposed project does not take into consideration the developmental needs of the targeted audience. Five to six year olds and even most 10 and 11 year olds are not developmentally prepared to "make concrete connections between the their community and science" as this project is proposing and certainly not capable of making analysis of the data they collect. - This project does not appear to provide anything new or innovative beyond some aspects of technology. This project of using citizens to monitor and report scientific data on a specific species has already been developed and implemented in many other programs as this proposal already indicates. To simply change the species and add some new technology features to data collection is not intellectual merit. - It is not clear how the information and collected data from this "citizen science" program will be used.

What are the broader impacts of the proposed activity?

The Cornell project, "Have You Spotted Me" is an interesting and compelling citizen science project that could have broad appeal and impact to citizens both young and old around the country. However, as this proposal is written, there is little indication that this project will have the capability of making broad impacts. There is no strategic plan to extend the impacts beyond the target audience nor does this proposal show how the project advances the ISE field. The evaluation plan for measuring the broad impacts will need to be more carefully outlined as well.

Summary Statement

The proposers should not be encouraged to submit a full proposal. This is a compelling project but it lacks intellectual merit and broad impacts. This project might be more suitably funded through the agriculture industry where this data could be very valuable.

Reviewer 4 Rating: Good What is the intellectual merit of the proposed activity?

What is the actual merit of the proposed activity? Cornell University and South Dakota University will be developing a community focused websites that will allow children and families to learn about biodiversity through ladybugs.

Project Team (Collaboration): Cornell University and South Dakota University will be working in collaboration on this project.

Partnerships (Collaboration): Collaboration is with 4-H, South Dakota 4-H, migrant worker program, after school summer programs and nature center.

Deliverables (Innovation): A multi dimensional website that includes a reporting system from website. The website will be user friendly and easy to use. Project Design (Innovation): To build a national online database accessibly by the public and children. A site that can be used by children, families which that will inform them about biodiversity in their community.

What are the broader impacts of the proposed activity?

What are the broader impacts of the proposed activity? Strategic Impact: The site will allow the public to engaged and be part of the scientific process by collecting data, hands on activities, it is also and innovative ways to use technology to collect data and engage in technology.

Audience (type): First grade and fifth graders, rural communities, Native Americans and disadvantage communities.

Public Audience Impact: The project hopes to build familiarity of informal science to underserved communities to enable children and families to become comfortable doing science and build their awareness of the importance in biodiversity and conservation

Evaluation: They will have pre and post evaluation assessments of understanding of the context and scientific methods, they will also measure the amount of enjoyment that participants will get form the activities. The website will also be able to produce reports.

Summary Statement

Summary Statement: This proposal overall seemed well put together but lacked a few things; it did not give details to how the website will user friendly for first graders and easy to use, nor did it give details to how it would recruit and market the website to users or tack how the site is being used. Also the proposal did not explore the digital divide that exists in underserved communities and how this population would explore this resource.

Reviewer 5 Rating: Poor What is the intellectual merit of the proposed activity?

Cornell University seeks to explore the use of ladybugs to develop citizen scientists for elementary aged children in first and fifth grades from Native American, rural, farming or low income communities in NY and SD, collaborating with 4H leaders, tribal groups and other institutions to communicate information on the values of biodiversity and conservation through the use of hands-on participation in a nationwide scientific experiment. Project Team (Collaboration): The project has assembled a competent set of academic, education, community, informal science practitioners and technical staff who have the experience and vision needed to successfully implement this project. The consultants and advisors are directly involved with the participant target groups providing informal afterschool educational and/or cultural services.

Partnerships (Collaboration): The collaborative relationships seem strong and appropriate, although it is not clear when the youth agency / community center staff are trained to implement the project with their students, and what type of follow up /debriefing is needed or how the staff will incorporate the activities into a cultural context at their individual work sites.

Deliverables (Innovation): Although the ladybug project is broad in scope the project team has developed a timeline with specific deliverables. However, there is nothing listed for year four and year three seems incomplete. The deliverables timeline also does not indicate at which point the culturally based science activities are implemented with the first and fifth graders.

Project Design (Innovation): The project's technical aspects are well described; however, there are no details for website design concepts that will work with both first and fifth graders. The project design for peer learning are not described, nor the training of staff of the participating community centers, cultural centers and tribal / nation sites.

What are the broader impacts of the proposed activity?

Strategic Impact: The strategic impact of the project lies in the importance of incorporating cultural expressions into the ladybug activities. The strategic impact also involves peer learning using fifth graders to serve as facilitators and guides for first grade students. There is only a one line mention of peer facilitation. The peer facilitator concept has concrete research data supporting its implementation with ISE programs. This process needs further exploration and structure.

Audience (type): The primary audiences to be recipients of the science knowledge are the first and fifth grade children. The secondary audience is the staff of the community agencies that serve as educators and facilitators of the targeted children.

Public Audience Impact: The measures of impact are described clearly and logically. However, the timeline with deliverables does not indicate any type of culminating festival, exhibition or parent program which for children would serve as a positive reinforcement to what they have learned as well as a means for promoting the project in the general Native American, low income, and underserved community. Family and social interactions are key cultural norms that should be addressed in this project. Evaluation: The evaluation plan uses pre and post tests to assess the participants' knowledge and attitudes toward biodiversity and scientific method using surveys, site visits and online questionnaires. Summary Statement

The general idea of the proposal is sound and well stated. It identifies the target communities and the important role of ladybug science. However, while the proposal presents opportunities, it failed to explain the timeline and the deliverables for implementing the activities with the first and fifth grade students. The peer learning process description is incomplete. Also, the training of local community and 4-H educational facilitators as presenters for the ladybug activities has not been fully developed. The proposal could be strengthened by addressing the issues described above.

Reviewer 6 Rating: Fair What is the intellectual merit of the proposed activity?

Project Team (Collaboration): The project has assembled a competent set of academic, education, community, informal science practitioners and technical staff who have the experience and vision needed to successfully implement this project. The consultants and advisors are directly involved with the participant target groups providing informal afterschool educational and/or cultural services.

Summary Statement