Program & Product Evaluation Course

Total Page:16

File Type:pdf, Size:1020Kb

Program & Product Evaluation Course

Program & Product Evaluation Course Guidelines & Rubric for: Final Evaluation Report – Due Week 15 (worth 50 points)

Use this rubric to make sure you have included everything required to the level of detail required. You may also wish to consult chapter 15 of your Russ-Eft & Preskill text, “Evaluating the Evaluation.” If you submit the report by the deadline, you can expect to receive feedback at the time a final grade is submitted for the course.

The feedback below emphasizes the major strengths and weaknesses of your evaluation project and final report, and may also provide recommendations for enhancing your knowledge and skills as an evaluator.

If certain elements of the rubric did not apply to your evaluation project (for example, if elements listed were not possible or desirable for your project), it will be marked “N/A” (not applicable), and will not impact your grade for the final report. There are a total of 50 possible points for this assignment. Each item below has been assessed as Missing or Problematic, Needs Modification, Fully Addressed, or Not Applicable.

Evaluation Project Report Feedback Rubric M/P – Missing/Problematic NM – Needs Modification Evaluation Project Report Component FA – Fully Addressed Suggestions to Improve Points Points NA – Not Applicable this Component Possible Awarded Rationale 4  Does the rationale present a clear and convincing argument for the evaluation – the need for it and the potential benefits of it?  Was there anything that could have been discussed regarding the background and context of the evaluation that was not – and thus, potentially weakened the evaluation? Purpose 4  Does the purpose statement flow naturally from the evaluation’s rationale?  Does the evaluation’s purpose statement provide a succinct description of what was evaluated and how the evaluation results were intended to be used?  Does the purpose statement indicate type of evaluation done (developmental, formative, summative)? Program & Product Evaluation Course Page 1 of 5 Final Evaluation Report Rubric – rev. 1/2008 Evaluation Project Report Feedback Rubric M/P – Missing/Problematic NM – Needs Modification Evaluation Project Report Component FA – Fully Addressed Suggestions to Improve Points Points NA – Not Applicable this Component Possible Awarded Guiding/Key Questions 5  To what extent were the key questions clear and comprehensive enough to address the evaluation’s stated purpose? How could the questions have met the purpose more fully and effectively?  To what extent were all of the intended key questions addressed by the data collected? If any of the intended questions were not answered, is there a clear explanation for this change in plan?  To what extent did the evaluation findings suggest that other key questions might have been asked – for a more effective outcome? Stakeholders 3  To what extent were all relevant stakeholders identified clearly?  To what extent were stakeholders engaged appropriately in the evaluation effort as it unfolded?  How might stakeholder cooperation/collaboration in the evaluation process have been sought or increased – thus, potentially improving the evaluation? Sample/Sampling Strategy 3  Is it clear how the sample for the evaluation was obtained?  Were reasonable response rates obtained? If not, how might responses rates have been improved?  Was the sampling strategy effective for obtaining answers to the key questions? If not, how could it have been improved? Data Collection Methods and Instrumentation 5  Are the data collection methods used specified clearly by key question? Program & Product Evaluation Course Page 2 of 5 Final Evaluation Report Rubric – rev. 1/2008 Evaluation Project Report Feedback Rubric M/P – Missing/Problematic NM – Needs Modification Evaluation Project Report Component FA – Fully Addressed Suggestions to Improve Points Points NA – Not Applicable this Component Possible Awarded  Are all data collection instruments used (questionnaires, interview protocols, observation/artifact checklists, etc.) described clearly?  Are actual instruments used (whether pre-existing or developed by the evaluator) included in appendices?  Were required permissions obtained for use of any protected, pre-existing instrumentation?  Were the methods and instruments used effective in obtaining answers to the key questions? If not, how could they have been improved?  Did the methods and instruments used yield valid, reliable, and usable data?  Were data analysis methods explained clearly?  Were data analysis methods appropriate?  To what extent did the data analysis take into account the key evaluation questions? Logistics and Timeline 3  To what extent were major tasks completed in a reasonable manner and within a reasonable timeframe?  If adjustments were made to major tasks and/or timeline as the evaluation unfolded, do these changes seem reasonable and well-documented?  Given how the evaluation project played out, how could the planned-for logistics and timeline have been improved – for a more successful evaluation process? Budget 2  To what extent did the evaluation meet the projected budget?  Are all expenses accounted for clearly?  Are any unanticipated expenses explained clearly? Do

Program & Product Evaluation Course Page 3 of 5 Final Evaluation Report Rubric – rev. 1/2008 Evaluation Project Report Feedback Rubric M/P – Missing/Problematic NM – Needs Modification Evaluation Project Report Component FA – Fully Addressed Suggestions to Improve Points Points NA – Not Applicable this Component Possible Awarded they seem reasonable? Limitations 4  Have the major limitations to the evaluation been identified and described clearly?  Have appropriate recommendations for addressing limitations been included? Results 7  To what extent do the results appear to be derived from the data and the data analysis process?  To what extent are the results communicated clearly?  To what extent are the results supported effectively by the data? (Are the most convincing data highlighted in the report? Are these data represented effectively and efficiently – i.e. through tables, figures, illustrations, etc?)  To what extent do the results address the key questions?  To what extent are the results organized in a manner useful to stakeholders?  To what extent are any negative results communicated in a sensitive and appropriate manner? Discussion and Recommendations 8  To what extent do the recommendations appear to flow logically from the data, data analysis, and results?  To what extent do the recommendations relate to the evaluation key questions and purpose?  To what extent do the recommendations appear reasonable? Implement-able?  Did evaluator revise the evaluation plan-portion of the report to communicate variances to the plan or other aspects that changed as a result of carrying out the

Program & Product Evaluation Course Page 4 of 5 Final Evaluation Report Rubric – rev. 1/2008 Evaluation Project Report Feedback Rubric M/P – Missing/Problematic NM – Needs Modification Evaluation Project Report Component FA – Fully Addressed Suggestions to Improve Points Points NA – Not Applicable this Component Possible Awarded evaluation? Other 2  Are citations/references used for any non-original information included in the report and are these citations/references in APA 5.0 format?  Are appendices used appropriately and correctly?  Is the report demonstrating adherence to the guidelines and assessment criteria specified for Assignment #2? Comments: 50 /50

Program & Product Evaluation Course Page 5 of 5 Final Evaluation Report Rubric – rev. 1/2008

Recommended publications