Appalachian State University s8

Appalachian State University HPC 6330-101-Assessment and Program Evaluation in Student Affairs Fall 2010 Meeting Time: Wednesday 9-11:50am Location: ED 224

INSTRUCTOR Dr. Diane M. Waryold Office location: 223E Edwin Duncan Hall Office telephone: 262-6067 Home telephone: 297-2292 (please do not call after 9:00pm) Email: [email protected] Office Hours: Posted on office door-sign up, drop-in and by appointment GA: Jessica Abt

Course Description This course is designed to introduce graduate students to the critical issues and practice of assessment and program evaluation in student affairs. The focus of this course will be on assessing outcomes of enrollment in post-secondary institutions, assessment methodologies used for exploring student outcomes, systematic program evaluation, and the application of student development theory to practice. This course is the 2nd course in a two course series (HPC 5340 & HPC 6330).

Course Objectives 1. Develop an awareness and understanding of standard tools of assessment and evaluation in student affairs practice.

2. Become a critical consumer of student affairs and higher education literature.

3. Understand the importance of assessment and program evaluation in student affairs and its’ impact on practice.

4. Identify the strengths and weaknesses of various approaches to assessment and evaluation.

5. Understand qualitative, quantitative and the mixed methods approach to research utilized in assessment and evaluation.

6. Build confidence in your ability to carry out a systematic assessment of a genuine problem or issue in a functional unit within student affairs.

7. Become knowledgeable of a comprehensive model of student affairs assessment and evaluation.

Several approaches will be employed to attain these course objectives. These include- readings, case studies, discussions, video, projects, and several writing and presentation assignments.

Required Texts LaFountain, R.M. & Bartos, R.B. (2002). Research and statistics made meaningful in Counseling and Student Affairs. Pacific Grove: Brooks/Cole.*

1 Keeling, Richard P. (Ed). (2004). Learning Reconsidered: A Campus-wide Focus on Student Experience. Washington, D.C.: American College Personnel Association and National Association of Student Personnel Administrators.** Schuh, J. H., & Upcraft, M. L., & Associates. (2001). Assessment practice in student affairs: An applications manual. San Francisco: Jossey-Bass.** American Psychological Association. (2001). Publication Manual of the American Psychological Association (5th ed). Washington, DC: Author. *text to be used for first course- HPC 5340 ** text to be used for HPC 6330 Assessment

Other texts that you might find helpful American Psychological Association. (2009-SECOND PRINTING). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.

Banta, T. W. (Ed.). (2004). Hallmarks of effective outcomes assessment. San Francisco: Jossey- Bass.

Banta, T. W., Lund, J. P., Black, K. E., & Oblander, F. W. (1993). Assessment in practice: Putting principles to work on college campuses. San Francisco: Jossey-Bass.

Bresciani, M. J., Zelna, C. L., & Anderson, J. A. (2004). Assessing student learning and development: A handbook for practitioners. Washington, D.C.: National Association of Student Personnel Administrators

Creswell, J.W. (2003). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (2nd Edition). Thousand Oaks, CA: Sage Publications.

Keeling, R.P. (Ed.) (2004). Learning reconsidered: A campus-wide focus on the student experience. Washington, DC: ACPA and NASPA. http://www.myacpa.org/pub/documents/LearningReconsidered.doc

Keeling, Richard P. (Ed). (2006). Learning reconsidered 2: Implementing a campus-wide focus on the student experience. Washington, DC: American College Personnel Association & National Association of Student Personnel Administrators.

Keeling, R., Wall, A., Underhile, R., Dungy, G. (2008). Assessment Reconsidered: Institutional effectiveness for student success. ICSSIA, NASPA, Keeling & Associates LLC.

Lee, W.Y. (ed.) (2003). Assessment and program evaluation (2nd edition). Boston: Pearson.

Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus. Malaney, G. D. (Ed.). (1999). Student affairs research, evaluation, and assessment: Structure and practice in an era of change. New directions for student services series, No. 85. San Francisco: Jossey-Bass.

Pascarella, E. T. & Terenzini, P. T. (2004). How college affects students (Vol. 2): A third decade of research. San Francisco: Jossey-Bass.

2 Pickering, J. W., & Hanson, G. R. (2000). Collaboration between student affairs and institutional researchers to improve institutional effectiveness. New directions for institutional research, No. 108. San Francisco: Jossey-Bass. Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass. Sowell, E. J. (2001). Educational Research: An integrative introduction. Boston: McGraw Hill. Sudman, S., & Bradburn, N.M. (1982). Asking Questions: A Practical Guide to Questionnaire Design. San Francisco: Jossey-Bass Inc. Publishers. Upcraft, M.L. & Schuh, J.H. (1996). Assessment in student affairs: A guide for practitioners. San Francisco: Jossey- Bass.

On-line Resources The Association for Institutional Research http://www.airweb.org/ Association for the Study of HE http://www.ashe.ws/ NASPA’s Assessment Community http://www.naspa.org/kc/saaer/default.cfm CAS http://www.cas.edu/ National Center for Postsecondary Improvement-inventory of HE assessment tools http://www.stanford.edu/group/ncpi/unspecified/assessment_states/instruments.html ACPA’s clearinghouse for assessment in student development http://www.myacpa.org/comm/assessment/ ASU Institutional Research Office http://irap.appstate.edu/ ASU Student Development http://www.sdac.appstate.edu/ Assessment Committee ASU Peer Institutions http://irap.appstate.edu/index.php?module=pagesmith&id=63 Principles of Good Practice http://www.sacscoc.org/2010sihandouts/Monday/cleary- Principles%202.pdf UCLA HE Research Institute (HERI) http://www.heri.ucla.edu Sample size calculator http://www.surveysystem.com/sscalc.htm Word frequency counter http://www.writewords.org.uk/word_count.asp NCSU Assessment website http://www2.acs.ncsu.edu/UPA/assmt/resource.htm Learning Reconsidered http://www.learningreconsidered.org/

Overall Expectations Attendance and participation in class This course has intentionally been designed to be experiential in nature. Therefore, regular class attendance and full participation are necessary to achieve the course objectives. Effective learning is enhanced when students complete all reading assignments, written assignments and experiential exercises. The input of each student is valuable. There is much that we can learn from each other’s contributions. Unexcused absences will jeopardize the student’s final grade. Notify instructor in advance if you will miss class. Punctuality is one of the hallmarks of a professional. With this said, please provide classmates and the instructor with the professional courtesy of being on time for class and avoid leaving early.

Student Religious Observance Policy Requirement (S.L. 2010-211)- NC law authorizes a minimum of two excused absences each academic year for religious observances required by the faith of a student. If you will be missing class because of a religious holiday/observance, please e- mail me prior to class to alert me of your anticipated absence.

3 Inclement Weather In the event of inclement weather, class may be postponed. Check e-mail for notification of class cancellation. If e-mail is down due to electrical outage or if you are uncertain as to whether or not class will meet, please phone Dr. Waryold at home.

Noise Making Devices If you must bring a cell phone/pager or any other device that makes noise and has the potential to disrupt class, please ensure that it is either OFF or in the SILENT mode. If you must take an urgent call, leave the classroom.

Academic Integrity Academic Integrity is central to effective learning in all academic communities. It is expected that students will neither engage in nor facilitate cheating. Students should know and adhere to ASU’s Academic Integrity Code. It can be found at http://studentconduct.appstate.edu

Timely submission of assignments It is expected that all assignments will be submitted by the deadlines noted in this syllabus. Late assignments will not be accepted except for extraordinary circumstances that are pre-approved by instructor. Assignments are due at the beginning of class on the dates indicated on this syllabus.

Writing Style All work is expected to be of graduate level caliber. Papers should be written using APA style. You are encouraged to have your work proofread. Excessive typographical and grammatical errors detract from the content of your work and will be reflected in your grade. All assignments should be typed, with 1 inch margins, double-spaced, and stapled. Double side use of paper is encouraged. Fancy packaging is expensive and unnecessary. Please turn in original copies only- Xerox copies will not be accepted.

Accommodations ASU and the Department of Human Development and Psychological Counseling are committed to providing a classroom and institutional climate in which all students can thrive. Those seeking accommodations based on a substantially limiting disability must contact and register with The Office of Disability Services (ODS) at http://www.ods.appstate.edu/ or 828-262-3056. Once registration is complete, individuals will meet with ODS staff to discuss eligibility and appropriate accommodations. Reasonable accommodations will be made for all students without regard to race, color, national origin, religion, gender, sexual preference, disability or veteran status.

Assignments Assessment project and presentation Students should report to the first class with a solid research proposal from the last course in hand and ready for implementation. Students will implement their respective study (as outlined in their proposals) and will report the results.

Steps: Research course- review 1. Choose topic and form dyads based upon interest area. 2. Choose presentation slots. 3. Meet with functional area to pitch study. Or choose a study that has been pitched to you! 4. Write proposal. 5. Obtain IRB approval

4 6. Present proposal to class for feedback. Share proposal with your unit under study to solicit feedback. If necessary, further refine proposal.

Steps: Assessment course 7. Implement study. 8. Present results of study to class (rehearsal). 9. Present results of study to the unit/department. 10. Submit final paper. 11. Revise final paper to meet publication guidelines. 12. Submit paper for publication.

Final paper and presentation- Drafts of your paper should be submitted for review on the dates indicated on the syllabus. Good writing is critical to effective communication as a professional. Don’t be surprised (or feel dejected) if you are asked to rewrite parts of your paper. Remember, practice makes perfect! As mentioned before, have others read your work before you submit your draft. Also, if you want to be a good writer, you should be a good reader! I encourage you to read and then read some more any books that you enjoy. A final comprehensive paper will be submitted at the end of the semester. This should include all of the elements noted on the rubric (See attached). The final paper should begin with an Executive Summary which thoroughly and concisely describes the plan and the results. The Executive Summary should be 1 page in length.

Final class presentation- Students will share with the class their findings and experiences (and a copy of the Executive Summary) during a 20 minute presentation. This is a formal presentation and students are encouraged to use this as a rehearsal to the formal presentation that you will deliver to the unit under study. This presentation should include a review of the plan, the results of the assessment, and what was learned through this research experience. Class members will provide written feedback on the presentation. Creativity is encouraged. The oral portion of the presentation will be evaluated on the following criteria; (1) complexity of thought, ideas, and concept, (2) creativity (imaginative methods of communicating ideas and concepts to the audience), (3) use of audio-visual aids, (4) intensity of effort, (5) implications or contributions to practice, and (6) what was learned from your research.

Reminder- Students are expected to share their findings w/ the functional unit under study by the last class meeting. E-mail verification from the campus unit must be received in order for teams to receive full credit.

Grading/Evaluation Working drafts (each draft= 5pts) 25points Final paper and pub (15 pts. each) 30 points Executive Summary 10 points Final Presentation in class 10 points Presentation to functional unit (e-mail from unit to verify) 10 points Attendance and Participation (includes lab & consults) 15 points Total 100 points

ASU Graduate School Grading Scale A = 95-100 B+= 86-89 C+= 76-79 F = Less than 70 A-= 90-94 B = 83-85 C = 73-75 B-= 80-82 C-= 70-72

5 CALENDAR OF CLASS ACTIVITIES Note: the professor may modify the calendar of activities during the semester to accommodate the needs/progress of the class. This syllabus is subject to change at the discretion of the instructor.

DATE TOPIC ASSIGNMENT Wednesday 8/25 Course overview & expectations SU Chap. 1& 2 Review of syllabus LR pgs. 3-7 Review format of paper (rubric)

Recap-Why assessment in SA’s? Steps in Assessment Process

Impressions exercise What do you need?

Lab Time Daisy consultation Josh & Matt; Andrew & Rebecca

Wednesday 9/1 Individual v. Organizational ER Haefner & Ford Assessment pgs. 50-55 Operational definitions reviewed

Updates SU Chapter 30 (10 minutes) LR 8-19

Lab Time Draft #1 due Daisy consultation All sections Katie & Jeremiah; James & Shane

Wednesday 9/8 More on the “why’s” SU Chap. 2, 30, & “how to’s” 33 Comprehensive Model LR pgs. 17-31

Assessing Student Needs SU Chap. 9, 10, Assessing Student Cultures 11, 12 Assessing Campus Environments Assessing Learning Outcomes Assessing Student Satisfaction Examples

Lab Time Daisy consultation Grant & Kate; Erin & Kelley

Wednesday 9/15 Review of Methods Section What to do w/ my data? Draft #2 due Intro., & ROL Using Excel Workshop

6 Mark Suggs @ library

7 CALENDAR OF CLASS ACTIVITIES - continued DATE TOPIC ASSIGNMENT Wednesday 9/22 The Results Section ER Stage & Manning pgs. 211-223 Benchmarking SU Chaps. 14 Professional Standards https://www.cas.edu/ACPA-Developments.pdf

Lab Time Daisy consultation Maddy & Craig; Alex & Megan

Wednesday 9/29 Executive Summary Dissemination

Program Evaluation SU Chaps. 13 Cost Effectiveness

Lab Time Draft #3 due Methods

Wednesday 10/6 Lab Time

Wednesday 10/13 Make up trip to UK- no formal class Use time to work on project

Wednesday 10/20 Make up trip to UK-no formal class Use time to work on project

Wednesday 10/27 Lab Time Draft #4 due Results & Discussion

Wednesday 11/3 Writing for Publication

Lab Time

Wednesday 11/10 Lab Time Draft #5 due All sections

Wednesday 11/17 Final Project Presentations Final papers due 1. ______References ckd. 2. ______3. ______4. ______5. ______

Wednesday 11/24 No class- University Break

8 CALENDAR OF CLASS ACTIVITIES - continued DATE TOPIC ASSIGNMENT Wednesday 12/1 Final Project Presentations Final papers due 6. ______References ckd. 7. ______8. ______

Presentations to departments Copy of Pub due

completed during this week. (w/ cover letter Wrap-up indicating where What will the future bring? it is being submitted Issues to consider

Feedback/evaluation & class gathering, return papers

Note- If you are experiencing challenges with your project, or if you just have questions, I encourage you to e-mail me to set up an appointment for a consultation outside of class. Please remember your procrastination does not qualify as a legitimate challenge and is not my emergency. I cannot and will not help you with that!

9 Assessment Project Rubric Name: Yosef 2nd Year

__ ___ Working title- Does title describe as precisely and specifically as possible the topic under study. Comment:

__ ___ Introduction – Did the author(s) present a clear rationale for the study? Is the purpose of the study clearly stated? Does the introduction include an outline of what is being studied and why? Is the significance of the study (what the authors expected it to add to knowledge in the field) clearly articulated? Is there a description of the functional area to be assessed? Is there a description of student demographics of target population? Are intended goals noted in the form of research questions or hypotheses? Comment:

__ __Flow- Is the content and study presented in a logical manner? Comment:

__ ___Review of Literature- Is the stage set for the exploration of the topic and is the topic adequately framed? Is the reader given an adequate explanation of the topic and is previous research on the topic presented? Are interlocking findings, inconsistent findings and unanswered questions identified? Comment:

(-3) Methodology- Is the methodology clearly described? Is the methodology appropriate given the purpose of the study? Are the population(s) and/or sample clearly described? Are the instrument(s) clearly described? Is the soundness (validity and reliability) of the instrument(s) explored? Is the plan for data collection, data analysis, and interpretation of results explained? Is there a sample of the instrument? Comment: Could/should have conducted more interviews. More on how their instrument had been validated?

(-2) Results- Are the results clearly presented? Are tables and/or figures clearly deigned/described? Are tables and/or figures necessary? Are personal opinions well supported? Is the data analysis appropriate and clear? Comment: Needed to tie this section back to ROL and research questions in a more deliberate way.

(-3) Discussion- Do the author(s) demonstrate how their research impacts practice? Did the author(s) present limitations of the study? Did the author(s) discuss implications for practice and future research? Do the conclusions fit with the results (are the conclusions appropriately made given the results)? Comment: the implications for practice needs more thought. This is the section in which you can share opinions.

(-5) References- Are all citations noted in reference section? Does the text match the reference list? Comment: references a bit sloppy- should have been checked for accuracy against text.

Other sections to consider The use of professional standards (if appropriate) as a benchmark. The estimated timeline of the plan (in proposal only).

Overall comments: Overall, a very solid paper. You lost valuable points because you did not proofread.

Presentation- well done = 10/10 points Paper =17/30pts. Executive Summary = 10/10pts.