Master of Public Health Program Self-Study Report

MASTER OF PUBLIC HEALTH

Prepared for the Council on Education for Public Health August 2020

Intentionally blank

Key Abbreviations

A M APE, Applied Practice Experiences MLS, Medical Laboratory Science MOU, Memorandum of Understanding B MPH, Master of Public Health BS, Bachelor of Science MS, Master of Science MUA/P, Medically Underserved Areas and C Populations CASA, Court Appointed Special Advocate CDC, Centers for Disease Control and N Prevention NCA, North Central Association of Colleges and CEPH, Council on Education for Public Health Schools CETL, Center for the Enhancement of Teaching and Learning O CHES, Certified Health Education Specialist OD2A, Overdose Data to Action CTL, Center for Teaching and Learning OE, Operating Expenses CV, Curriculum Vitae OSRP, Office of Research and Sponsored Programs D D2L, Desire2Learn online learning and teaching P software PharmD, Doctor of Pharmacy PhD, Doctor of Philosophy E PI, Principal Investigator ERF, Electronic Resource File PIF, Primary Instructional Faculty PUBH, Public Health (course prefix) F FAQ, Frequently Asked Questions R FTE, Full-time Equivalent RCM, Responsibility Centered Management FY, Fiscal Year financial model RHES, Registered Environmental Health G Specialist GPA, Grade Point Average S H SARA, State Authorization Reciprocity HHMI, Howard Hughes Medical Institute Agreements HLC, Higher Learning Commission SAS, Statistical Analysis System HRSA, Health Resources and Services SDBOR, South Dakota Board of Regents Administration SDPHA, South Dakota Public Health Association I SDPHSA, South Dakota Public Health Student IDEA, Student Ratings of Instruction System Association IDS, Instructional Design Services SDSU, South Dakota State University ILE, Integrative Learning Experience SHS, School of Health Sciences IRB, Institutional Review Board SSOM, Sanford School of Medicine

T TBH, To Be Hired

U URM, Under-Represented Minority USD, University of South Dakota

Intentionally blank

Table of Contents

Introduction ...... 1 Template Intro-1 ...... 7 Template Intro-2 ...... 7

Section A A1. Organization and Administrative Processes ...... 11 A2. Multi-Partner Programs ...... 19 A3. Student Engagement ...... 23

Section B B1. Guiding Statements ...... 31 B2. Graduation Rates ...... 37 Template B2-1 ...... 37 B3. Post-Graduation Outcomes ...... 43 Template B3-1 ...... 43 B4. Alumni Perceptions of Curricular Effectiveness ...... 47 B5. Defining Evaluation Practices ...... 53 Template B5-1 ...... 53 B6. Use of Evaluation Data ...... 63

Section C C1. Fiscal Resources ...... 67 Template C1-1 ...... 69 C2. Faculty Resources...... 75 Template C2-1 ...... 75 Template C2-2 ...... 77 C3. Staff and Other Personnel Resources...... 83 Template C3-1 ...... 83 C4. Physical Resources ...... 87 C5. Information and Technology Resources ...... 91

Section D D1. MPH & DrPH Foundational Public Health Knowledge ...... 97 Template D1-1 ...... 97 D2. MPH Foundational Competencies ...... 101 Template D2-1 ...... 101 Template D2-2 ...... 102 D4. MPH & DrPH Concentration Competencies ...... 113 Template D4-1 ...... 113 D5. MPH Applied Practice Experiences ...... 117 Template D5-1 ...... 118 D7. MPH Integrative Learning Experience...... 127 Template D7-1 ...... 127 D14. MPH Program Length ...... 137 D20. Distance Education ...... 145 Template Intro-1 ...... 145

Section E E1. Faculty Alignment with Degrees Offered ...... 153 Template E1-1 ...... 153 Template E1-2 ...... 154 E2. Integration of Faculty with Practice Experience ...... 159 E3. Faculty Instructional Effectiveness ...... 165

E4. Faculty Scholarship ...... 171 Template E4-1 ...... 173 E5. Faculty Extramural Service...... 177

Section F F1. Community Involvement in School or Program Evaluation and Assessment ...... 183 F2. Student Involvement in Community and Professional Service ...... 189 F3. Assessment of the Community’s Professional Development Needs ...... 193 F4. Delivery of Professional Development Opportunities for the Workforce ...... 197

Section G G1. Diversity and Cultural Competence ...... 201

Section H H1. Academic Advising ...... 209 H2. Career Advising ...... 217 H3. Student Complaint Procedures ...... 223 H4. Student Recruitment and Admissions ...... 229 Template H4-1 ...... 229 H5. Publication of Educational Offerings ...... 233

Introduction

1) Describe the institutional environment, which includes the following:

a. year institution was established and its type (e.g., private, public, land-grant, etc.)

The joint University of South Dakota (USD) – South Dakota State University (SDSU) Master of Public Health (MPH) program is overseen by the South Dakota Board of Regents (SDBOR). Founded in 1862, the University of South Dakota (USD) is the state’s designated liberal arts university. South Dakota State University (SDSU) was founded in 1881 and is the state’s land grant institution as designated by state law and the Morrill Act of US Congress. The SDBOR has responsibility for operation of six state-supported universities, including USD and SDSU. Board members are appointed by the Governor, and the SDBOR has a full-time Executive Director and staff responsible for implementation of policy, data development and analysis, and preparation of academic, financial, and legal reports.

b. number of schools and colleges at the institution and the number of degrees offered by the institution at each level (bachelor’s, master’s, doctoral and professional preparation degrees)

The Master of Public Health is housed in SDSU’s College of Pharmacy & Allied Health Professions and USD’s School of Health Sciences. SDSU confers degrees through 9 colleges including the College of Pharmacy and Allied Health Professions. The College of Pharmacy and Allied Health Professions serves approximately 400 students enrolled in the Doctor of Pharmacy (PharmD) program, PhD in Pharmaceutical Sciences program, BS in Medical Laboratory Science (MLS) program, AS and BS in Respiratory Care program, and joint MPH program. SDSU offers 82 undergraduate majors, 36 master’s degree programs, 15 doctoral programs, and 2 professional doctorates.

USD consists of 8 colleges/schools including the School of Health Sciences (SHS). The School of Health Sciences offers a variety of health professions programs including nursing, occupational therapy, physical therapy, physician assistant, dental hygiene, social work, addiction counseling and prevention, and public health. USD offers 77 bachelor’s programs, 35 master’s degree programs, 13 doctoral programs, and 5 professional doctorates.

c. number of university faculty, staff and students

With a fall 2019 student enrollment of 11,518, SDSU is home to over 2,000 full-time equivalent faculty and staff (649 faculty and 1,365 staff). Fall 2019 enrollment at USD was 9,920, and the institution employs 460 faculty and 955 staff.

d. brief statement of distinguishing university facts and characteristics

The MPH program is one of the many distinguishing characteristics of the institutions. Both institutions offer undergraduate, graduate, and professional programs with the South Dakota System of Higher Education.

SDSU provides a rich academic experience in an environment of inclusion and access through inspired, student-centered education, creative activities and research, innovation and engagement that improves the quality of life in South Dakota, the region, the nation and the world. SDSU is the state’s largest, most comprehensive higher-education institution. Several of its academic programs have received national distinctions including pharmacy, nursing, sociology, education, and journalism. SDSU is a Carnegie classification high research activity institution. SDSU Extension is the outreach arm of the land-grant university. The university is also home to the South Dakota Art Museum, the South Dakota Agricultural Heritage Museum, McCrory Gardens, and the Senator Thomas A. Daschle Congressional Research Study.

As the oldest university in the state, USD serves as the flagship and the only public liberal arts university in the state. USD is the largest provider of graduate education in South Dakota, with 78 programs and nearly half of the graduate and professional students from public universities. Graduate and professional students play an integral role in education, research, and creative scholarship. USD is the home to South Dakota’s only medical and law schools. Several USD academic programs have been recognized with national distinctions including the Sanford School of Medicine, occupational therapy, physician assistant, addiction counseling and prevention, education, and business. USD is a Carnegie classification high research activity institution. USD is home to the National Music Museum, Oscar Howe Gallery, and University Art Galleries. Health- related centers include the Center for Brain and Behavior Research, Disaster Mental Health Institute. e. names of all accrediting bodies (other than CEPH) to which the institution responds. The list must include the regional accreditor for the university as well as all specialized accreditors to which any school, college or other organizational unit at the university responds

Both SDSU and USD are accredited by the Higher Learning Commission (HLC), one of six regional institutional accreditors in the US. HLC accredits degree-granting educational institutions in the North Central region. HLC is recognized by the US Department of Education and the Council on Higher Education Accreditation. SDSU’s last comprehensive review by HLC was in 2019-2020 at which time full 10-year accreditation was achieved. The last reaffirmation of accreditation for USD took place in 2011 resulting in accreditation through 2020. The next reaffirmation of accreditation will take place in September 2020.

SDSU has several academic programs with specialty accreditation or certification. In addition, some service units have accreditation or certification. Accrediting bodies to which SDSU programs respond to include: • Accreditation Board for Engineering and Technology (ABET) • Accreditation Commission for Programs in Hospitality Administration (ACPHA) • Accreditation Council for Pharmacy Education (ACPE) • Accreditation Council for the Education of Nutrition and Dietetics (ACEND) • Accrediting Council on Education in Journalism and Mass Communication (ACEJMC) • American Association of Museums Accreditation Commission (AAM) • American Association of Veterinary Lab Diagnosticians (AAVLD) • American Chemical Society • American Society of Agricultural and Biological Engineers (ASABE) • American Society of Biochemistry and Molecular Biology (ASBMB) • American Society of Health-System Pharmacists (ASHP) • Aviation Accreditation Board International (AABI) • College Reading and Learning Association • Commission on Accreditation of Athletic Training Education (CAATE) • Commission on Accreditation of Programs in Applied and Clinical Sociology (CAPACS) • Commission on Accreditation for Respiratory Care (CoARC) • Commission on Accreditation for the Exercise Sciences (CoAES) • Commission on Collegiate Nursing Education (CCNE) • Council for Interior Design Accreditation (CIDA) • Council for the Accreditation of Counseling and Related Educational Programs (CACREP) • Council for the Accreditation of Educator Preparation (CAEP) • Council on Rehabilitation Education (CORE) • Department of the Army • National Architectural Accrediting Board (NAAB) • National Accrediting Agency for Clinical Laboratory Sciences (NAACLS)

• National Association for Education of Young Children (NAEYC) • National Association of Schools of Music (NASM) • National Collegiate Honors Council • Nonprofit Leadership Alliance • Society for Range Management (SRM) • Society for Simulation in Healthcare (SSH)

Likewise, USD has several academic programs with specialty accreditation or certification as well as service units with accreditation. The specific accrediting bodies that USD responds to include: • Accreditation Board for Engineering and Technology (ABET) • Accreditation Council for Occupational Therapy Education (ACOTE) • Accreditation Review Commission on Education for Physician Assistants (ARC-PA) • Accrediting Council on Education in Journalism and Mass Communication (ACEJMC) • Administration for Developmental Disabilities (ADD) • American Chemical Society • American Dental Association Commission on Dental Accreditation • American Psychological Association (APA) • Association for Assessment and Accreditation of Laboratory Animal Care (AAALAC) • Association for the Accreditation of Human Research Protection Programs (AAHRPP) • Association of University Programs in Health Administration (AUPHA) • Association to Advance Collegiate Schools of Business International (AACSB) • Commission on Accreditation in Physical Therapy Education (CAPTE) • Commission on Collegiate Nursing Education (CCNE) • Committee on Accreditation of Educational Programs for the Emergency Medical Services Professions (CoA EMSP) • Council for the Accreditation of Counseling and Related Educational Programs (CACREP) • National Council for Accreditation of Teacher Education (NCATE) • Council on Academic Accreditation in Audiology and Speech-Language Pathology of the American Speech-Language-Hearing Association (ASHA) • Council on Social Work Education (CSWE) • Council on the Section of Legal Education and Admissions to the Bar of the American Bar • Association Foreign Programs Subcommittee of the Accreditation Committee • Liaison Committee on Medical Education (LCME) • National Addiction Studies Accreditation Commission • National Association of School Psychologists (NASP) • National Association of Schools for Theatre (NAST) • National Association of Schools of Art and Design (NASAD) • National Association of Schools of Music (NASM) • National Association of Schools of Public Affairs and Administration (NASPAA) f. brief history and evolution of the public health program (PHP) and related organizational elements, if applicable (e.g., date founded, educational focus, other degrees offered, rationale for offering public health education in unit, etc.)

The MPH program was created in partnership as both SDSU and USD are committed to serving the health care needs of the state. SDSU offers first professional degree programs that prepare pharmacists (PharmD) and nurse practitioners (DNP), as well as master’s and undergraduate programs that prepare registered nurses, medical laboratory scientists, respiratory therapists, dietitians, and athletic trainers. USD offers professional degree programs that prepare physicians (MD), physical therapists (DPT), occupational therapists (OTD), health science professionals (PhD), psychologists (PhD), and audiologists (AuD), as well as master’s programs that prepare addiction counselors (MA), social workers (MSW), speech language pathologists (MA), and physician assistants (MS).

A 2009 survey from the South Dakota Department of Health and the South Dakota Public Health Association indicated a need to develop an in-state MPH program to address public health needs within the state. All states contiguous to South Dakota (North Dakota, Minnesota, Iowa, Nebraska, Montana), except Wyoming, have MPH programs. In 2014, the SDBOR approved the interdisciplinary, collaborative MPH program. The MPH program added to the universities’ capacities to support the health and well-being of South Dakota and utilizes the unique academic strengths of SDSU and USD. The program places a particular focus on rural and underserved populations, and each institution brings the aforementioned strengths to the administration of the program.

The decision on where the MPH program would be located under the university structure was influenced by the deans leading the efforts. The MPH program at USD has been under the School of Health Sciences since the beginning of the program, for which Dr. Michael Lawler served as dean. Recently a new department of Public Health and Health Sciences was established to create better synergy between the Public Health and Health Science programs. Dean Lawler acted as the program director until he left his position in fall 2018. In fall 2018, a new position description was posted for a 50% MPH program director and 50% associate dean for diversity and inclusion for the Sanford School of Medicine. The current Program Director, Dr. DenYelle Kenyon, was chosen for that position from a national candidate pool and assumed the responsibilities in March 2019. USD faculty Chelsea Wesner has been a steady presence for the MPH program as an Instructor since 2015 and took on many coordination duties during the absence of a Program Director. Professor Wesner leads the evaluation partnership between USD and the South Dakota Department of Health for their Centers for Disease Control and Prevention (CDC) collaboration to lead the evaluation of the Overdose Data to Action (OD2A) cooperative agreement. This collaboration between state and federal agencies aims to build surveillance and prevention efforts to respond to the opioid epidemic in South Dakota. Through inclusion of MPH primary faculty Drs. Susan Strobel and Susan Puumala, there is great opportunity for MPH students to become involved in these important public health efforts.

At SDSU, the MPH program is under the College of Pharmacy and Allied Health Professions. In 2019, a new department, the Department of Allied and Population Health, was formed in the College. The MPH program is part of this new department; this reorganization was designed to provide additional support for the program and create additional opportunities for teaching and research collaboration. Last fall, the MPH program coordinator (Dr. Aaron Hunt) became one of the co-investigators of a 5-year multi-million dollar grant through the SD Department of Health as part of the CDC’s 1815 project, “Improving the Health of Americans through Prevention and Management of Diabetes and Heart Disease and Stroke.” Dr. Hunt is also project director for a one-million-dollar Health Resources and Services Administration (HRSA) grant to address the opioid epidemic in rural South Dakota. The SDSU team involved in both grants include faculty, staff and students from the Department of Allied and Population Health as well as faculty members from the Department of Pharmacy Practice, which is housed within the College of Pharmacy and Allied Health Professions. As demonstrated, this structural change provides support for the MPH program, builds a strong foundation for collaboration among disciplines, and creates research synergy.

The current curriculum outlined in section D2-1 is the newest iteration of the MPH curriculum, as several changes have been made over the years. These curriculum changes are set to go into effect for the incoming fall 2020 student cohort. Major curricular changes include moving PUBH 760 (Public Health and Native American Communities) to a required core course, changing PUBH 730 (Integrated Learning Experience) from 6 to 3 credits, and creation of a new course PUBH 729 (Leadership and Project Management in Public Health).

As of 8/19/2020, of all active and graduated students (n=172), the original curriculum is followed by n=82 (48% of all total students), with n=64 (78% of the first curriculum cohort) already graduated and n=18 active students projected to graduate within the next year or so. The second iteration of the curriculum was formally launched in fall 2018 and is followed by n=51 (30% of total students.

Eleven students on the second curriculum have graduated. The vast majority of those students are projected to graduate within the next 3 years. The incoming cohort 2020-2021 n=39 (23% of the total) is on the third iteration of the curriculum.

2) Organizational charts that clearly depict the following related to the program:

a. the program’s internal organization, including the reporting lines to the dean/director

The program is led by a program director and organized by a program coordinator. The MPH program director (housed under USD) and MPH program coordinator (housed under SDSU) collaborate on all aspects of the program including processes, curriculum, admissions, and student services. The MPH program director reports to the dean for the School of Health Sciences, who is also the chair of the Department of Public Health and Health Sciences. MPH faculty at USD report to the MPH program director. The MPH program coordinator reports to the head of the Department of Allied and Population Health, who reports to the dean of the College of Pharmacy & Allied Health Professions. MPH faculty at SDSU report to their respective department heads but also report to MPH leadership for issues and matters related to the MPH program, including courses and students.

b. the relationship between program and other academic units within the institution. Ensure that the chart depicts all other academic offerings housed in the same organizational unit as the program. Organizational charts may include committee structure organization and reporting lines

SDSU’s College of Pharmacy and Allied Health Professions is organized into three academic departments. The Department of Pharmaceutical Sciences provides instruction for the biomedical science and pharmaceutical sciences courses in the PharmD program as well as instruction and mentoring in the Pharmaceutical Sciences PhD program. The Department of Pharmacy Practice is based in Sioux Falls, SD and has faculty in Brookings, Sioux Falls, Rapid City, Fort Meade, and Watertown. These faculty teach PharmD courses, precept students on Advanced Pharmacy Practice Experiences, and provide pharmacy services at health systems throughout the state. The Department of Allied and Population Health provides instruction for the MPH program, the Medical Laboratory Science (MLS) program, the Respiratory Care program, and pharmacy social/behavioral/administrative science courses. This structure brings together faculty expertise within the College to support the academic programs in the following ways: (1) the Pharmacy Administration faculty support the MPH program, and (2) the MPH faculty support public health aspects within the PharmD program. The College’s organization chart can be found in the Organizational Charts folder in the Electronic Resource File (ERF) (SDSU College of Pharmacy and Allied Health Professions Organizational Chart).

USD’s School of Health Sciences is comprised of 9 departments (addiction counseling and prevention, dental hygiene, public health and health sciences (BS and PhD), medical laboratory science, nursing, occupational therapy, physical therapy, physician assistant, and social work). The departments and programs are based in Vermillion, with the Nursing Department having additional distance sites in Rapid City and Sioux Falls. In addition, the School houses the Center for Prevention of Child Maltreatment and Interprofessional Health Education Center. The School of Health Sciences’ Organizational Chart can be found in the ERF (USD School of Health Sciences Organizational Chart).

The program director and program coordinator serve on all the standing committees, outlined in A- 1, for the MPH program. Additional representatives from each institution are members of these committees as well. The program director and program coordinator contribute to decision-making activities outside of the unit of accreditation by service on committees at the college/school and institutional level. This creates a strong relationship between both institutions to ensure consistency and enhanced communication in the joint MPH program.

c. the lines of authority from the program’s leader to the institution’s chief executive officer (president, chancellor, etc.), including intermediate levels (e.g., reporting to the president through the provost)

The program director reports to the dean of the School of Health Sciences at USD. The dean reports to the vice president of Health Affairs as well as the provost and vice president for Academic Affairs. At SDSU, the program coordinator reports to the head of the Department of Allied and Population Health, who reports to the dean of the College of Pharmacy and Allied Health Professions. The dean reports to the provost and vice president for Academic Affairs. Organizational charts for both universities can be found in the ERF (SDSU Organizational Chart and USD Organizational Chart). d. for multi-partner programs (as defined in Criterion A2), organizational charts must depict all participating institutions

As described above, the program director and program coordinator collaborate on all aspects of the MPH program. The Department Head of Allied and Population Health, dean of the College of Pharmacy and Allied Health Professions, and the dean of the School for Health Sciences work together to identify solutions when challenges arise. These five individuals comprise the MPH program Dean’s Council. A Memorandum of Understanding (MOU) describes the collaboration between the two universities (see A2-1 MOU for MPH Program, 2019 in ERF). In 2019, the MOU was revised and approved by both institutions and the SDBOR. The deans at each institution as well as the program director and the head of the Department of Allied and Population Health worked in tandem on the revisions. This relationship builds a strong foundation for the public health program and provides a supportive environment for the faculty, staff, and students. The organizational chart depicted below shows the collaborative relationship (i.e., the dotted line) between the two entities.

The MOU is an important foundational document that outlines which institution is responsible for teaching each core course. Currently, each university leads the Applied Practice Experience and Integrated Learning Experience courses for students enrolled in their home institution. The program functioning occurs through the committees (all faculty, curriculum and assessment, dean’s council, admissions, and community advisory board), all of which have representatives from both USD and SDSU. Therefore, all decisions impacting the MPH program are made in collaboration. For example, responsibility for elective courses is decided in the Curriculum and Assessment Committee. For major curricular and program changes, discussion is held at the All Faculty Committee meeting to secure broad support and then moved to the Curriculum and Assessment Committee.

Figure 1. Joint South Dakota State University and University of South Dakota MPH Program Organizational Chart

3) An instructional matrix presenting all of the program’s degree programs and concentrations including bachelor’s, master’s and doctoral degrees, as appropriate. Present data in the format of Template Intro-1.

Template Intro-1. Instructional Matrix - Degrees and Concentrations

Master's Campus Distance Degrees Academic Professional Based Executive Based Concentration Degree Degree Generalist MPH X

4) Enrollment data for all of the program’s degree programs, including bachelor’s, master’s and doctoral degrees, in the format of Template Intro-2.

Template Intro-2. Enrollment Data Current Degree Enrollment Master's MPH* 97

Intentionally blank

Section A

A1. Organization and Administrative Processes

Intentionally blank

Divider back

A1. Organization and Administrative Processes

The program demonstrates effective administrative processes that are sufficient to affirm its ability to fulfill its mission and goals and to conform to the conditions for accreditation.

The program establishes appropriate decision-making structures for all significant functions and designates appropriate committees or individuals for decision making and implementation.

The program ensures that faculty (including full-time and part-time faculty) regularly interact with their colleagues and are engaged in ways that benefit the instructional program (e.g., participating in instructional workshops, engaging in program specific curriculum development and oversight).

1) List the program’s standing and significant ad hoc committees. For each, indicate the formula for membership (e.g., two appointed faculty members from each concentration) and list the current members.

The MPH Program has several standing committees. A table with the composition guideline and current members can be found below:

Table A1.1. Composition of MPH Committees Committee Composition Guideline Current Members All Faculty All primary instructional and non-primary Yen-Ming Huang, Aaron instructional faculty who teach or have Hunt, DenYelle Kenyon, administrative effort for the program. Erin Miller, Sharrel Pinto, Adjunct professors are not commonly Susan Puumala, Teresa included. Seefeldt, Tom Stenvig, Susan Strobel, Chelsea Wesner Admissions Three members from each institution and Aaron Hunt, DenYelle one non-voting administrative liaison. Kenyon, Sharrel Pinto, Members from USD include the MPH Bruce Vogt, Chelsea program director and 2 USD SHS faculty Wesner, Erin Miller, members. Members from SDSU include the Administrative Liaison (non- head of the Department of Allied and voting) Population Health, MPH program coordinator and one faculty member. The administrative liaison can be appointed from either institution; Spring 2020 is Johanna Gregoire from USD. Curriculum & Three members from each institution, at Aaron Hunt, Sharrel Pinto, Assessment least one student member. An academic DenYelle Kenyon, Tom affairs administrator from one of the Stenvig, Susan Strobel, institutions serves as a resource and non- Chelsea Wesner, Heidi voting member of the committee. Schultz (student), Teresa Seefeldt (non-voting) Community 10-12 members, including public health Jyoti Angal, Deana Around Advisory Board stakeholders from the region, program Him, Joshua Clayton, coordinator and program director. Invitation, Jennifer Giroux, Kris with 2-3 year terms for external members. Graham, Connie Halverson, External members are current/past Tyler Hemmingson, Aaron preceptors, alumni/current students, and Hunt, DenYelle Kenyon, other public health experts recruited from Sylvia Lasley, Mary

tribal health, mental health, service sector, Michaels, Michaela Seiber, and government. Chelsea Wesner Dean’s Council Deans from college/school that houses Aaron Hunt, DenYelle MPH program, program coordinator, Kenyon, Sharrel Pinto, Dan program director. Hansen, Haifa Samra

We also created an ad-hoc committee to prepare for accreditation. Members in June 2019 included DenYelle Kenyon, Teresa Seefeldt, Kristy Ullom, and Chelsea Wesner. Additional members (Aaron Hunt, Sharrel Pinto, Susan Strobel) attended meetings in September 2019, and Dr. Hunt was added to regular meetings.

2) Briefly describe which committee(s) or other responsible parties make decisions on each of the following areas and how the decisions are made:

a. degree requirements

The specific degree requirements including the core course list, number of elective credits, courses that count for elective credits, and total number of required credits are outlined in Section D-2 of the self-study. Ideas for changes to the program or curriculum are brought to all faculty meeting for discussion. The Curriculum and Assessment Committee reviews proposed changes to degree requirements and makes decisions based on consensus/vote. Proposed changes then go to the full faculty for review. If approved by the faculty, program modifications are reviewed at each institution through individual institution curricular procedures. At SDSU, substantive program modifications for master’s degree programs are reviewed by the university graduate council, the academic affairs committee, and the faculty senate. At USD, major curricular modifications are vetted through the Dean's office, then through the SHS curriculum and instruction committee, then to the university graduate council, and finally to the academic affairs committee. Once program modifications are approved at the institutional level, they are then reviewed at the system level. The system academic affairs council reviews substantive changes to evaluate impact on the other regental institutions. Finally, SDBOR review of the proposed modifications is conducted. General master’s degree requirements such as minimum GPA and time limitations on degree completion are set by the graduate school at each institution.

b. curriculum design

Curriculum design is the responsibility of the program faculty. The Curriculum and Assessment Committee evaluates the curriculum for potential changes and reviews ideas for curriculum modifications that come from the faculty or administration. Substantive course modifications go through the same process as described above in degree modifications. Substantive course modifications include changes to course descriptions, course instructional methods, and grading. Minor course modifications are reviewed at the institutional level only. Examples of minor course modifications are changes to course pre-requisites, registration restrictions, and course titles.

c. student assessment policies and processes

Student assessment policies and processes are determined by the Curriculum and Assessment Committee and then approved by the All Faculty Committee. Each university has an institutional assessment office which sets policies and processes for preparation and submission of assessment reports to the universities.

d. admissions policies and/or decisions

Standard admissions criteria are agreed upon in faculty meetings and executed by the joint Admissions Committee. The admissions committee reviews the entire list of applicants and recommends an admission status for each student. The acceptance lists are sent out to the graduate schools for admissions letters to be sent. If a prospective student has applied to both institutions, they will receive a follow-up joint letter of offer from the program director and program coordinator, with instructions to a) accept admission to one of the institutions and decline admission to the other institution or b) decline admission to both institutions by a certain date. e. faculty recruitment and promotion

Faculty recruitment and retention are the responsibility of the individual institutions to meet the needs of teaching the courses responsible as outlined in the MOU (see A2-1 MOU for MPH Program, 2019 in ERF). The USD program director and SDSU department head are ultimately responsible for assigning course loads and accounting for the workload of their primary faculty. If primary faculty are at capacity, the program director and department head identify faculty from other departments with public health expertise in the area needed for the course and collaborate with that faculty member’s primary department for workload to teach the PUBH course. If there are no available university faculty, the program director and department head identify potential adjuncts with public health expertise in the area needed for the course. Historically, this process has worked well with Dr. Stenvig, an associate professor of nursing at SDSU as a non-primary faculty, pulling in previous faculty to continue to teach (Dr. Killian, USD adjunct), as well as new adjuncts the program has strong relationships with who are known public health leaders that have served as previous practicum advisors (Ms. Melstad, USD adjunct).

Search committees are used in all faculty searches. The program director sits on search committees for SDSU faculty, and the program coordinator is a member of search committees for USD faculty. The search committees give recommendations to the dean of the School of Health Sciences for USD searches and the department head for Allied and Population Health for SDSU searches.

Standards documents for faculty promotion are developed at the department or program level. Policies for promotion are set by each institution. School of Health Sciences’ and College of Pharmacy and Allied Health Professions’ Promotion and Tenure Committees make recommendations to the Deans’ offices, which then go to the university level Promotion and Tenure Committees. Final approval is done by the SDBOR. Although it is rare someone directly from the public health program is on one of the review committees, there is a strong knowledge of public health within the interdisciplinary schools and colleges. The letters from the faculty themselves as well as the program director and department head to help translate the standards for the public health arena if necessary. Historically, there has been great admiration and reputation of the public health program at the institutions. f. research and service activities

The joint program has a good reputation for research (particularly evaluation research) and a very strong reputation for service activities. There is a great need to meet the public health research and service demands in South Dakota, as there is a dearth of public health professionals in this area. Faculty, under the direction of the program director and dean of the School of Health Sciences at USD or the department head for Allied and Population Health and dean of the College of Pharmacy and Allied Health Professions at SDSU, make decisions about research and service activities, taking faculty workload into account. Research support is provided at the College level. For example, the College of Pharmacy and Allied Health Professions has an associate dean for research, a grant proposal specialist, and a Graduate Studies and Research Committee. These individuals and the committee support faculty research grant proposals and make recommendations on research infrastructure. For USD,

the structure differs, but the support is comparable. The SHS has a 10% Research Director, Dr. Will Schweinle, and the Office of Research and Sponsored Programs (ORSP) provides support and workshops to support research efforts.

3) A copy of the bylaws or other policy documents that determine the rights and obligations of administrators, faculty and students in governance of the program.

See ERF for the MPH Student Handbook 2020-21 (A1-1) and MPH Governance Document (A1- 2).

4) Briefly describe how faculty contribute to decision-making activities in the broader institutional setting, including a sample of faculty memberships and/or leadership positions on committees external to the unit of accreditation.

MPH faculty serve on a variety of committees at the college/school and institutional level. The table below lists the committee memberships for the MPH program director and primary instructional faculty.

Table A1.2. Faculty Contribution to Broader Institution Faculty University Leadership Positions/Memberships External to PHP Aaron Hunt Diversity and Inclusion Committee, Member College Council, Member DenYelle Kenyon President’s Council on Diversity and Inclusion, Member Diversity Health Affairs, Chair SHS Chairs Committee, Member SHS Leadership Council, Member SHS Promotion & Tenure – COHE Unit Committee, Co-Chair SHS Curriculum and Instruction Committee, Member SSOM Administrative Council, Member SSOM Administrative Staff, Member SSOM Dean’s Council, Member Racial Justice in Medicine Workgroup, Chair Susan Puumala SHS Faculty Council, Member Health Affairs Medical Informatics Committee, Member Susan Strobel USD Research Committee, Member

5) Describe how full-time and part-time faculty regularly interact with their colleagues (self- study document) and provide documentation of recent interactions, which may include minutes, attendee lists, etc.

Full-time and part-time faculty for the MPH program regularly interact with their colleagues through faculty and committee meetings. Full- and part-time faculty, as well as adjunct faculty teaching core courses, are invited to the faculty meetings and annual retreats. Many engaged faculty in the program are part-time, with half-time appointments in other departments (i.e., Kenyon, Strobel), have administrative or mentorship roles in the program (i.e., Huang, Pinto, Seefeldt), or hold full- time positions in other departments but teach core courses in the MPH program (i.e., Stenvig). Adjunct professors have regular communication with the program director or department head, to best loop them in on the program priorities and provide guidance and standardization of curricular aspects such as rubrics and grading.

Please see ERF for documentation of faculty (A1-3), curriculum and assessment (A1-4), and advisory board meetings’ minutes (A1-5), which include attendee lists.

6) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The joint MPH program demonstrates effective administrative processes to fulfill the mission and goals, as well as conditions for accreditation. Many improvements have been made recently to improve processes and standardize decision making processes. This includes establishment of standing committees (All Faculty, Advisory Board), tweaking of ongoing committees (Admissions, Curriculum and Assessment), and establishment of an ad-hoc committee for accreditation. This provides ample opportunity for interactions between faculty members and to build consensus for program improvement ideas. Additional improvements include the reorganization of SDSU’s College of Pharmacy and Allied Health Professions to include a Department of Allied and Population Health, under which the SDSU side of the MPH program falls, ensuring greater oversight of the program and input by a greater number of faculty. Additionally, the USD School of Health Sciences has a new dean who oversees the USD side of the MPH program, and who recently established the Department of Public Health and Health Sciences, where the USD side of the MPH program will fall. We will continue to strengthen our joint governance and oversight of the MPH program, with a critical eye towards continuous quality improvement.

Intentionally blank

Section A

A2. Multi-Partner Programs

Intentionally blank

Divider back

A2. Multi-Partner Programs (applicable ONLY if functioning as a “collaborative unit” as defined in CEPH procedures)

The program has a single identified leader (dean or director) and a cohesive chain of authority for all decision making relevant to the educational program that culminates with this individual.

1) Describe the major rights and responsibilities of each participating institution.

The MPH program is collaborative and builds on the strengths of both USD and SDSU. The overall direction of the program rests with the MPH Program Director, DenYelle Kenyon, and the coordination of program activities at SDSU is done by the MPH Program Coordinator, Aaron Hunt. They are the main contacts for the students enrolled at their individual universities. An MOU has been established outlining aspects such as which core courses are taught by each institution, data sharing, and committee membership. Each institution is responsible for hiring faculty and covering the courses outlined in the MOU. Spring 2020 was the inaugural launch of a joint admissions process. Degree conferral is done through the student’s home university where they initially matriculated. The fiscal arrangement is explained below in further detail; in general, the University through which each course is taught receives the tuition and fees.

2) A copy of the formal written agreement that establishes the rights and obligations of the participating universities in regard to the program’s operation.

Please see the MOU in the electronic resource file (A2-1 MOU for MPH Program, 2019).

3) Describe the role and responsibilities of the identified leader.

The leadership of the MPH program lies with the program director. Dr. Kenyon leads the organization and sets the agenda of the All Faculty Committee, Curriculum and Assessment Committee, Community Advisory Board, and ad-hoc CEPH Accreditation Committee with input from the program coordinator. Dr. Kenyon ensures aspects of CEPH accreditation are being met, structuring meetings for broader discussion and faculty/administrative input as appropriate. To assess quality of instruction, faculty evaluation scores and students’ grades and assessments on key curricular assignments are examined at the conclusion of each semester. Additionally, she is responsible for academics and curriculum assessment with support from the institutional research and assessment office.

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The institutions in the joint MPH program collaborate well and have strong organizational documents and processes established. The original MOU was updated in 2019. These formal documents help guide the day-to-day processes and decisions. Streamlining the admissions process is an area for improvement. Starting in Spring 2020, the program utilizes a joint Admissions Committee with representatives from both universities. This allows for timely sharing of information on applicants and admitted students to reduce student and faculty confusion on admission status.

Intentionally blank

Section A

A3. Student Engagement

Intentionally blank

Divider back

A3. Student Engagement

Students have formal methods to participate in policy making and decision making within the program, and the program engages students as members on decision-making bodies whenever appropriate.

1) Describe student participation in policy making and decision making at the program level, including identification of all student members of program committees over the last three years, and student organizations involved in program governance.

The joint program strives to include students in policy and decision making. Each year, faculty recruit student volunteers via email to serve on the committees outlined in the table below. These student representatives attend committee meetings via online video conferencing. They are also responsible for voicing the concerns of the class during meetings and reporting updates to the class via emails. Students on these committees serve as voting members who have full decision-making capacity. For example, students on the Curriculum and Assessment Committee regularly vote on student elective substitutions and curricular changes. We currently do not have any student organizations involved but are planning to form a student section of the South Dakota Public Health Association.

Table A3. Student Participation in Program Committees Committee Student Role MPH Curriculum & Corryn Celmer (2017-2019) Voting members, voice student Assessment Hailey Purves (2019-2020) perspective and concerns. Heidi Schultz (2019-2021) Tracy Bieber (2020-2022) Community Advisory Board Sylvia Lasley (2019-2021) Voting member, voice student perspective and concerns. Diversity and Inclusion Marissa Trosen (2020) Voting member, voice student Committee perspective and concerns.

2) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

As an online program, it is oftentimes difficult to engage students in committee work. However, we have made recent gains in this area, recruiting students to the Community Advisory Board and Curriculum and Assessment Committees. We plan for student engagement to continue to increase in the future. We intend to invite students to future faculty search committees and continue to invite them to serve on school and university wide committees. For example, SDSU added an MPH student to their College Diversity and Inclusion Committee in spring 2020. In addition, the MPH program is planning to launch a student association for public health to engage students in public health service and research projects as well as provide the association the opportunity to provide input in any programmatic or structural changes.

Intentionally blank

Section A

A4. Autonomy for Schools of Public Health Not applicable

A5. Degree Offerings in Schools of Public Health Not applicable

Intentionally blank

Divider back

A4. Autonomy for Schools of Public Health

Not applicable.

A5. Degree Offerings in Schools of Public Health

Not applicable.

Intentionally blank

Section B

B1. Guiding Statements

Intentionally blank

Divider back

B1. Guiding Statements

The program defines a vision that describes how the community/world will be different if the program achieves its aims.

The program defines a mission statement that identifies what the program will accomplish operationally in its instructional, community engagement and scholarly activities. The mission may also define the program’s setting or community and priority population(s).

The program defines goals that describe strategies to accomplish the defined mission.

The program defines a statement of values that informs stakeholders about its core principles, beliefs and priorities.

1) A one- to three-page document that, at a minimum, presents the program’s vision, mission, goals and values.

The program’s guiding statements are re-evaluated regularly to ensure relevance to and alignment with the priorities of the communities and organizations we serve. In 2019, the CEPH Accreditation Committee revised the vision, mission, and values of the MPH program with feedback from faculty, administration, students, and community members serving across MPH program committees and the Community Advisory Board. Feedback was collected by a member of the CEPH Accreditation Committee who incorporated this feedback into subsequent drafts. On November 1, 2019, these drafts were presented to the Curriculum and Assessment Committee comprising representatives of the faculty and students (see A1-4 Curriculum & Assessment Meeting Minutes in the ERF). The resulting statements were revised and finalized by the CEPH Accreditation Committee on December 30, 2019. Guiding statements were developed to be broad and inclusive, yet specific enough to allocate resources and evaluate outcomes as described in section B5. The current guiding statements are outlined below.

Vision: The program will generate resources and support for rural and underserved communities to achieve optimal and equitable health.

Mission: The University of South Dakota and South Dakota State University Master of Public Health program is committed to improving health in rural and underserved communities by providing interdisciplinary learning, scholarship, and community engagement for public health professionals.

Program Goals:

Goal 1: Academic Excellence. Promote and achieve excellence in public health teaching and learning.

Goal 2: Public Health Research & Practice. Faculty and students will engage in research and practice in collaboration with rural and underserved communities.

Goal 3: Diversity & Inclusion. Recruit and retain diverse faculty and students who embody cultural inclusiveness.

Goal 4: Service. Enhance the program’s involvement in public health service to rural and underserved communities in South Dakota.

Core Values:

The MPH program embraces the educational philosophy and values of the University of South Dakota and South Dakota State University. It is committed to the following core values: ●Academic Excellence ●Student Centered Education ●Professionalism ●Integrity ●Diversity ●Collaboration and Leadership ●Commitment to Engagement & Service in the Community ●Commitment to the Professional Development of Students and Faculty

2) If applicable, a program-specific strategic plan or other comparable document.

Not applicable

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The main tenet focusing on rural and underserved communities has underscored our vision, mission, and core values since the program’s inception in 2015. However, we continue to refine goals and objectives to better evaluate the impact of the program and alignment with our vision and mission.

For example, in a recent Alumni Survey, graduates were asked to identify how well the MPH program met goals for the program. Table B1 summarizes respondents’ assessment of these goals. A 4-point Likert scale was used to evaluate the goals (1=did not meet the goal, 2=somewhat met the goal, 3=met the goal, 4=substantially exceeded the goal).

Table B1. Alumni Perception of MPH Program Goals (N = 31) Met the Goal and Substantially Mean Program Goals Exceeded the Goal Rating % (n) Promote and achieve excellence in public health teaching 93% (28) 3.17 and learning. Faculty and students will engage in research and practice 73% (22) 3.03 in collaboration with rural and underserved communities. Recruit and retain diverse faculty and students who 73% (22) 3.03 embody cultural inclusiveness. Enhance the program’s involvement in public health service 72% (21) 3.07 to rural and underserved communities in South Dakota. Note. The alumni who completed the survey had the option not to answer every question. Response rate was 54%.

A strength of these results is that nearly three-quarters of alumni who responded perceive that the program has met established goals. While nearly all (93%) students perceive that the program met the goal for excellence in public health teaching and learning, we would like to increase perceptions of the other goals. For example, slightly fewer (72%) alumni perceive the program meeting the goal of involvement in public health service to rural and underserved communities in South Dakota. To strengthen this perception, we developed new requirements for students to participate in at least one service and one professional development activity each year. We also developed an objective in 2019 for all faculty to engage in extramural service with a focus on rural and underserved communities.

Throughout the spring and fall 2019 semesters, the program goals and objectives were reviewed in all faculty meetings, as well as in smaller accreditation workgroup meetings and the January 2020 Community Advisory Board meeting. There was slight modification of the vision and program goals, and several new objectives were created since the Alumni Survey was implemented. In addition, a Graduate Exit Survey (see B5-4 Graduate Exit Survey, 2019 in ERF) was developed and implemented at the end of fall 2019 semester to provide immediate feedback on student experiences. Therefore, we will continue to monitor metrics, including the Graduate Exit Survey and Alumni Survey.

Intentionally blank

Section B

B2. Graduation Rates

Intentionally blank

Divider back

B2. Graduation Rates

The program collects and analyzes graduation rate data for each degree offered (e.g., BS, MPH, MS, PhD, DrPH).

The program achieves graduation rates of 70% or greater for bachelor’s and master’s degrees and 60% or greater for doctoral degrees.

1) Graduation rate data for each degree in unit of accreditation. See Template B2-1.

Template B2-1. Students in MPH Degree, by Cohorts Entering Between 2014-15 and 2020-21 Cohort of 2014-15 2015-16 2016-17 2017-18 2018-19 2019-20 2020-21

Students # Students 6 entered # Students 0 withdrew, 2014 dropped, etc. -15 # Students 0 graduated Cumulative 0% graduation rate # Students 6 24 continuing at beginning of this school year (or # entering for 2015 newest cohort) -16 # Students 1 0 withdrew, dropped, etc. # Students 0 0 graduated Cumulative 0% 0% graduation rate # Students 5 24 34 continuing at beginning of this school year (or # entering for 2016 newest cohort) -17 # Students 0 0 3 withdrew, dropped, etc. # Students 2 0 0 graduated Cumulative 33% 0% 0% graduation rate # Students 3 24 31 45 2017 continuing at -18 beginning of this school

year (or # entering for newest cohort) # Students 0 0 1 5 withdrew, dropped, etc. # Students 2 15 3 0 graduated Cumulative 67% 63% 9% 0% graduation rate # Students 1 9 27 40 27 continuing at beginning of this school year (or # entering for 2018 newest cohort) -19 # Students 0 1 3 6 1 withdrew, dropped, etc. # Students 0 3 12 7 0 graduated Cumulative 67% 75% 44% 16% 0% graduation rate # Students 1 5 12 27 26 29 continuing at beginning of this school year (or # entering for 2019 newest cohort) -20 # Students 0 0 1 6 3 1 withdrew, dropped, etc. # Students 1 5 6 7 10 0 graduated Cumulative 83% 96% 62% 31% 37% 0% graduation rate # Students 5 14 13 28 39 continuing at beginning of this school year (or # entering for 2020 newest cohort) -21 # Students ------withdrew, dropped, etc. # Students -- 1 1 -- -- graduated Cumulative ------graduation rate

2) Data on doctoral student progression in the format of Template B2-2.

Not applicable

3) Explain the data presented above, including identification of factors contributing to any rates that do not meet this criterion’s expectations and plans to address these factors.

MPH students have six years to complete their degree and graduate based on CEPH standards, which is the same as the graduate policy at SDSU but more stringent than the USD graduate policy allowing for 7 years. Our program is on target for exceeding a 70% graduation rate across the first (2014-15) and second (2015-16) cohorts. All other cohorts are on track with two or more years remaining. Student retention declined during the third (2016-17) and fourth (2017-18) cohorts and improved during the fifth (2018-19) and sixth (2019-20) cohorts. The majority (78%) of our graduates completed their degree in three years. This is notable as many of our students are enrolled part-time and are employed full-time.

Our program admitted students on a rolling basis (fall, spring, summer) until the 2019-20 academic year, which contributed to variation in plans of study as well as pace and made academic advising particularly challenging. In 2018-19, our program revised and introduced a new curriculum to meet 2016 CEPH standards, of which 41% (n=40) of our active students now follow. A third revision of the curriculum was launched with our newest cohort (2020-21), of which 40% (n=39) of our active students follow. We anticipate improved graduation rates overall, as well as increasing the number of students who graduate in ≤3 years. This projection will be evaluated over time.

Factors contributing to poor retention may include admissions criteria, inconsistent academic advising, following the original MPH curriculum that was multi-disciplinary and included required courses from other departments, and issues specific to the student (e.g., financial aid, employment, family issues, medical leave for self or family).

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Our first two cohorts (2014-15 and 2015-16) have exceeded graduation rates of 70%, which demonstrates our capacity to implement a new MPH program while building a partnership between USD and SDSU and recruiting a diverse student body. While retention declined during our third and fourth cohorts (2016-17 and 2017-18), we were happy to see retention has improved in recent cohorts (2018-19 and 2019-20). In addition, the program has added additional faculty to serve as advisors and provide timely follow-up with at risk students.

Our original MPH curriculum was revised and introduced during the 2018-19 academic year. The 2nd curriculum was designed to align with 2016 CEPH standards and improve: 1) the structure and process of the Applied Practice Experience and Integrative Learning Experience, 2) PUBH course offerings for all required courses, and 3) PUBH offerings for elective courses.

There are several important weaknesses that may relate to our graduation and retention rates. During the 2017-18 cohort, 38% (n=17) of students dropped or withdrew from the program. This was the largest cohort to date and the last cohort to follow our original curriculum. As our student body increased steadily during the 2017-18 academic year, our program capacity (i.e., faculty, administration) decreased from three primary instructional faculty to two. Further compounding program capacity was the loss of an additional primary faculty in the spring of 2019, reducing primary faculty to one. This change affected the capacity of our faculty and program to respond proactively to students at risk for issues related to retention.

To address these weaknesses, our program hired an inaugural MPH director during the spring of 2019 and 2.5 primary instructional faculty during fall 2019. In addition, SDSU has created the Department of Allied and Population Health which houses the MPH program and provides additional support such as a program assistant and additional faculty time. We have also expanded the role of the program coordinator to a tenure-track faculty position requiring a PhD level candidate. These changes strengthen our capacity to provide responsive academic and career advising and early intervention services, as well as inform structural changes related to graduation and retention. A key structural change involves moving from rolling admissions (fall, spring, summer) to fall admission, which was implemented during the 2019-20 academic year. This change reduces the variation in plans of study and pace, as well as creates a pseudo-cohort in which students are more likely to take courses together and may feel more connected to their peers and the overall MPH program. We believe these changes will make a considerable impact on retention and graduation, and the data will be monitored for continuous quality improvement opportunities.

Section B

B3. Post-Graduation Outcomes

Intentionally blank

Divider back

B3. Post-Graduation Outcomes

The program collects and analyzes data on graduates’ employment or enrollment in further education post-graduation, for each degree offered (e.g., BS, MPH, MS, PhD, DrPH).

The program achieves rates of 80% or greater employment or enrollment in further education within the defined time period for each degree.

1) Data on post-graduation outcomes (employment or enrollment in further education) for each degree. See Template B3-1.

Template B3-1. Post-Graduation Outcomes (n = 46 of N=57) Post-Graduation 2016 Graduates 2017 Graduates 2018 Graduates 2019 Graduates 2020 Graduates Outcomes (FA15, SP16, (FA16, SP17, (FA17, SP18, (FA18, SP19, (FA19) SU16) SU17) SU18) SU19) Number and Number and Number and Number and Number and Percentage Percentage Percentage Percentage Percentage Employed 1 (100%) 7 (100%) 13 (100%) 15 (88%) 7 (88%) Continuing 0 0 0 1 (6%) 0 education/ training (not employed) Not seeking 0 0 0 0 1 employment or not seeking additional education by choice Actively seeking 0 0 0 1 (6%) 0 employment or enrollment in further education Unknown 0 3 4 4 0 Total 1 10 17 21 8 Of those employed, 100% (1) 86% (6) 92% (12) 73% (11) 86% (6) how many are employed in the public health or health-related field Note. Data was collected from 81% (n=46 of N=57) alumni. Survey response rate was 54% (n = 31 of N=57).

2) Explain the data presented above, including identification of factors contributing to any rates that do not meet this criterion’s expectations and plans to address these factors.

The first graduate of the MPH. program was in summer 2016. Utilizing various data collection methodologies, post-graduate outcome data was collected from 81% (n=46/N=57) of the graduates.

The electronic survey was sent to all MPH graduates between fall 2015 and fall 2019 (54% response rate). A total of 31 students responded to the Alumni Survey. A personalized follow-up of alumni who did not respond to the initial survey was conducted encouraging them to complete the survey. This prompted nine alumni to complete the survey.

For those who did not respond to the survey, post-graduation outcome data was collected through social media sites such as LinkedIn and Facebook. Of those who are employed, 84% (n=36/N=43) are employed in public health or a health-related field. The MPH program exceeds the 80% target

for post-graduation employment or enrollment in further education for 2016, 2017, 2018, 2019 and 2020.

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

It is encouraging to see that after reaching 81% of our alumni (100% of Fall 2019 graduates), their employment/continuing education rate is almost 100%. Moreover, a majority (84%) of those graduates are employed in a public health or health-related field. However, we would like to see a higher response rate to our survey. To reduce the number of unknowns for post-graduation outcomes, an Exit Survey (see B5-4 Graduate Exit Survey, 2019 in ERF) was developed and administered in fall 2019 and spring 2020 to students upon completion of the capstone course to document career plans and gather updated contact information. Of the 21 students surveyed in fall 2019 and spring 2020, 10 completed the Exit Survey resulting in a 48% response rate. Improvement plans included creating a LinkedIn and other social media accounts to engage current MPH students and sustain contact with alumni. These were promoted to the incoming 2020-2021 cohort during the Orientation session. These additional plans will enhance the current post- graduation outcomes data collection process, which includes administration of the Alumni Survey, utilizing social media, and personalized follow-up. This process will be implemented and replicated for future graduates each semester.

Section B

B4. Alumni Perceptions of Curricular Effectiveness

Intentionally blank

Divider back

B4. Alumni Perceptions of Curricular Effectiveness

For each degree offered, the program collects information on alumni perceptions of their own success in achieving defined competencies and of their ability to apply these competencies in their post-graduation placements.

The program defines qualitative and/or quantitative methods designed to maximize response rates and provide useful information. Data from recent graduates within the last five years are typically most useful, as distal graduates may not have completed the curriculum that is currently offered.

1) Summarize the findings of alumni self-assessment of success in achieving competencies and ability to apply competencies after graduation.

Alumni were asked to rate their level of competency of achieving the defined competencies and their ability to apply these competencies in post-graduation employment. A 4-point Likert scale was used to self-assess the competency level of the graduate (1=not competent, 2=somewhat competent, 3=competent, 4=very competent).

Strengths

The competencies listed below are the strengths of the MPH program (i.e. those competencies with the highest average rating (mean)). 1. 97% (n=29 of 30) of the respondents indicated they are competent or very competent in performing effectively on interprofessional teams. 84% (21 of 25) of the respondents have had the opportunity to apply this skill in their post-graduate employment. (average competency 3.73 on a 4.00 scale) 2. 90% (n=27 of 30) of the respondents indicated they are competent or very competent in communicating audience-appropriate public health content, both in writing and through oral presentation. 80% (n=20 of 25) of the respondents have had the opportunity to apply this skill in their post-graduate employment. (average competency 3.47 on a 4.00 scale) 3. 93% (n=28 of 30) of the respondents indicated they are competent or very competent in developing strategies to advance health equity through improvement of social determinants of health in rural and medically underserved communities. 42% (10 of 24) of the respondents have had the opportunity to apply this skill in their post-graduate employment. (average competency 3.40 on a 4.00 scale) 4. 90% (n=27 of 30) of the respondents indicated they are competent or very competent in discussing the means by which structural bias, social inequities and racism undermine health and create challenges to achieving health equity at organizational, community and societal levels. 64% (16 of 25) of the respondents have had the opportunity to apply this skill in their post-graduate employment. (average competency 3.40 on a 4.00 scale)

Figure B4.1: Highest Overall Mean (Competency Scale) for All Respondents, Alumni Survey (n=31)

Competency Level & Opportunity to Apply Competency

Competency Level (Competent and Very Competent) Apply Competency After Graduation (Answered Yes) Mean Competency 97% 100% 90% 93% 90% 4.00 80% 84% 3.50 80% 3.47 3.73 3.40 64% 3.40 3.00 60% 2.50 42% 2.00 40% 1.50 1.00 20% 0.50 0% 0.00 Discuss the means by Develop strategies to Communicate audience- Perform effectively on which structural bias, advance health equity appropriate public health interprofessional teams social inequities and through improvement of content, both in writing racism undermine health social determinants of and through oral and create challenges to health in rural and presentation achieving health equity at medically underserved organizational, community communities. and societal levels

Note. Responses were asked with a 4-point Likert scale: (1=not competent, 2=somewhat competent, 3=competent, 4=very competent). The alumni who completed the survey, may not have answered every question. Response rate was 54%.

Opportunities for Improvement

The competencies listed below are opportunities for improvement in the MPH program (i.e. those competencies with the lowest average [mean] rating). 1. 53% (n=16 of 30) of the respondents indicated they are competent or very competent in evaluating various risk management and risk communication approaches in relation to issues of environmental and occupational health. 16% (n=4 of 25) of the respondents have had the opportunity to apply this skill in their post-graduate employment. (average competency 2.73 on a 4.00 scale) 2. 53% (n=16 of 30) of the respondents indicated they are competent or very competent in applying negotiation and mediation skills to address organizational or community challenges. 40% (n=10 of 25) of the respondents have had the opportunity to apply this skill in their post- graduate employment. (average competency 2.73 on a 4.00 scale) 3. 53% (n=16 of 30) of the respondents indicated they are competent or very competent in explaining basic principles and tools of budget and resource management. 56% (n=14 of 25) of the respondents have had the opportunity to apply this skill in their post-graduate employment. (average competency 2.73 on a 4.00 scale) 4. 53% (n=16 of 30) of the respondents indicated they are competent or very competent in using innovative, strengths-based communication methods aimed at a broad audience to advocate for health equity in Native American populations. 24% (n=6 of 25) of the respondents have

had the opportunity to apply this skill in their post-graduate employment. (average competency 2.77 on a 4.00 scale)

Figure B4.2: Lowest Overall Mean (Competency Scale) for All Respondents, Alumni Survey (n=31)

Competency Level & Opportunity to Apply Competency

Competency Level (Competent and Very Competent) Apply Competency After Graduation (Answered Yes) Mean Competency 100% 4.00

80% 2.73 2.73 2.73 3.00 2.77 60% 53% 53% 53% 56% 53% 2.00 40% 40%

24% 1.00 20% 16%

0% 0.00 Evaluate various risk Apply negotiation and Explain basic principles Using innovative, management and risk mediation skills to and tools of budget and strengths-based communication address organizational resource management communication approaches in relation or community methods aimed at a to issues of challenges broad audience to environmental and advocate for health occupational health equity in Native American populations

2) Provide full documentation of the methodology and findings from alumni data collection.

In spring/summer 2019, the CEPH Accreditation Task Force developed a survey to be administered to graduates of the Master of Public Health (MPH) program from the University of South Dakota and South Dakota State University. The survey was developed using best practices, CEPH requirements and the needs of the program. The survey was designed to secure satisfaction ratings, information and recommendations from alumni to guide and assist faculty and staff in the quest to continuously improve the MPH degree program.

The survey software, QuestionPro, was used to distribute the survey to the MPH alumni. An email invitation and two follow-ups were sent by Ms. Chelsea Wesner, USD MPH faculty member, and Dr. Aaron Hunt, SDSU MPH Program Coordinator. Email addresses were collected from the Alumni Office, the student information system, and the master student list. These sources were utilized to request survey participation. If the email address was not valid (i.e. bounce back/filtered), an alternate email address was used to collect feedback. Alumni who completed the survey were

entered into a drawing for a $25 gift card to Walmart. The survey was open for two weeks. Only non-respondents were contacted in the reminder emails.

To enhance response rates, the survey was re-opened in the fall semester. For those who did not respond to the survey, a personalized follow-up was conducted encouraging graduates to complete the survey. This encouraged six additional alumni to complete the survey in fall 2019. In summer 2020, a similar process was followed. This methodology prompted three additional fall 2019 graduates to complete the survey.

A graduate exit survey was developed and administered in fall 2019 & spring 2020 to students registered for the capstone course to document career plans and contact information. Future plans include creating a LinkedIn and other social media accounts to engage current MPH students and sustain contact with alumni.

The full documentation of the methodology, response rates, and results from the alumni data collection can be found in the electronic resource file (B4-1 Alumni Survey, 2020; B4-2 MPH Alumni Survey Report, 2020; B4-3 Alumni Survey Results 2020; B4-4 MPH Alumni Survey comments 2020).

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The results of the alumni survey were shared with the program’s CEPH Accreditation Committee and during an All Faculty Committee meeting (see ERF B4-5 MPH Faculty Meeting Notes 10-23- 19). To address the competency level of concentration competencies, the Curriculum and Assessment Committee has approved curricular changes to address the needs of the workforce. For example, PUBH 760 Public Health and Native American Communities has become a core course in the MPH program instead of an elective. This course addresses the competency level of the concentration competency of “proposing state/tribal/local policy change to address identified gaps in rural and medically underserved communities.”

To evaluate the new and original curricula, we used survey question branching to allow certain questions to be asked of specific respondents. This tool, as well as the student’s plan of study, will allow reports to be aggregated by the new and original curricula for evaluation purposes.

Section B

B5. Defining Evaluation Practices

Intentionally blank

Divider back

B5. Defining Evaluation Practices

The program defines appropriate evaluation methods and measures that allow the program to determine its effectiveness in advancing its mission and goals. The evaluation plan is ongoing, systematic and well-documented. The chosen evaluation methods and measures must track the program’s progress in 1) advancing the field of public health (addressing instruction, scholarship and service) and 2) promoting student success.

1) Present an evaluation plan that, at a minimum, lists the program’s evaluation measures, methods and parties responsible for review. See Template B5-1.

There are three main components to our evaluation methods—the Program Goals Assessment, Student Surveys (current, exit, alumni), and Key Curricular Assessment. Formal reports from these assessments will be compiled annually at the end of each academic year in preparation for review at the annual program retreat in the fall. The program director will be responsible for the Program Goals Assessment Report and the program coordinator will be responsible for the Key Curricular Assessment Report and Student Survey Report. Template B5-1 aligns each measure with a method of data collection, brief protocol for auditing data, and the committee or body who will review and respond to the findings.

Template B5-1. Evaluation Measures, Methods, and Parties Responsibility for Evaluation measures Data collection method for measure review ACADEMIC EXCELLENCE

Goal 1. Promote and achieve excellence in public health teaching and learning. Program assistants will update the Student Database annually (each May) with current GPA. The MPH director and program coordinator will conduct an Proportion of MPH annual audit of graduates at the beginning of each Curriculum and students who graduate academic year (late May). Results will be compiled into Assessment with an overall GPA of an aggregate Program Goals Assessment Report that Committee 3.7 or higher is disseminated to the Curricular and Assessment Committee (each July) for consideration in decision making. The report is also shared during the MPH All Faculty Retreat (each August) for additional feedback. All faculty teaching a PUBH course will submit an annual evaluation report by October 1 each year, which includes a summary of all progress on research, Percent of MPH teaching, and service over the past year. USD faculty primary faculty who submit data via Digital Measures and SDSU faculty Curriculum and perform above or submit data via a Faculty Annual Review form. With Assessment exceed expectations these reports, the MPH director will audit annually Committee on annual reviews for (each December), compile overall teaching evaluation teaching scores across all PUBH courses taught by primary faculty, and update outcomes in the Program Goals Assessment Report. The report will be reviewed by the Curriculum and Assessment Committee (each July). The MPH program director and program coordinator Percent of MPH will obtain IDEA course evaluation results for all courses receiving courses taught by MPH primary faculty from the Office Curriculum and ratings for excellent of Institutional Research and Assessment at the end of Assessment teaching and excellent each academic year (May). Results will be compiled Committee course on student into the aggregate Program Goals Assessment Report course evaluations that is disseminated to the Curricular and Assessment

Committee (each July) for consideration in decision making.

The program coordinator will gather course outcomes and qualitative feedback from all PUBH courses at the end of each semester. Results will be compiled in a Key Curricular Assessment at the end of each Percent of graduates academic year (late May). The report and key findings who earn ≥90% will be presented to the Curriculum and Assessment Curriculum and across key Committee (July) for feedback in advance of reviewing Assessment assessments for all all PUBH courses for currency and continuous Committee; All MPH foundational and improvement. The program coordinator will also update Faculty Committee concentration the Program Goals and Assessment Report with competencies relevant outcomes data from the Key Curricular Assessment, which will be shared with the All Faculty Committee during an annual retreat (each August).

Percent of students The MPH program director and program coordinator Curriculum and who are satisfied or will conduct a Student Survey (annually in July), Annual Assessment very satisfied with Alumni Survey (6-12 months post-graduation) and Exit Committee; All academic advising Survey (upon graduation). Program assistants will audit Faculty Committee and compile results at the end of each academic year (May). A summary of results from each type of survey will be compiled in an annual report and shared with the Curriculum and Assessment and All Faculty Committees (each July). Student satisfaction outcome data will be updated in the Program Goals Assessment Report by the MPH director, which will be reviewed during the annual Faculty Retreat (August).

Percent of Applied APE preceptors (and students) will complete an Curriculum and Practice Experience evaluation at the end of the APE based on the overall Assessment (APE) preceptor experience across 3 semesters. The preceptors will Committee; All evaluations that assess each student’s professional skills, performance, Faculty Committee exceed expectations and attainment of key competencies and learning objectives related to their APE. APE faculty (from USD and SDSU) will review the Preceptor Evaluation results each December to determine completion of APE requirements and assess overall trends. Items will be placed on All Faculty Committee meeting agendas in the Spring to review opportunities for continuous quality improvement. Preceptor evaluation data will be updated in the Program Goals Assessment Report by the MPH director.

PUBLIC HEALTH RESEARCH & PRACTICE

Goal 2. Faculty and students will engage in research and practice in collaboration with rural and underserved communities.

Percent of primary and All faculty teaching a PUBH course will submit an non-primary annual self-evaluation report by October 1 of each All Faculty instructional faculty year, which includes a summary of all progress on Committee who publish scholarly research, teaching, and service over the past year.

articles or USD faculty submit data via Digital Measures and commissioned report SDSU faculty submit data via a Faculty Annual Review on topics in rural form. With these reports, the MPH director will audit and/or underserved annually (each December) and update outcomes in the communities Program Goals Assessment Report. Items will be placed on All Faculty Committee meeting agendas in the Spring to review professional development options. The report will be reviewed by the All Faculty Committee during the annual MPH Faculty Retreat (August) . The program director will request data at the end of Percent of primary and each academic year (late May) from the USD & SDSU non-primary Offices of Research & Sponsored Programs for all instructional faculty proposals involving PUBH primary and non-primary Curriculum & who submit a grant instructional faculty. Data will also be gathered directly Assessment proposal on topics in from faculty in the Faculty Database. Aggregate data Committee rural and/or will be compiled and updated in the Program Goals underserved and Assessment Report, which will be shared with the communities Curriculum and Assessment Committee (July) for feedback and decision-making. APE and ILE faculty will update the APE and ILE Percentage of APE database each semester. Data will be transferred to the and Integrative Master Student Database. The program director will Learning Experience audit the Master Student Database annually and Curriculum and (ILE) projects that compile outcomes related to projects in rural and/or Assessment focus on public health underserved communities. Relevant outcomes will be Committee; All issues in rural and/or updated in the annual Program Goals Assessment Faculty Committee underserved Report and shared with the Curriculum and communities Assessment Committee (each July) and All Faculty Committee (each August) during the faculty retreat. APE faculty will update the APE and ILE database each semester. Data will be transferred to the Master Percent of APE that Student Database. The program director will audit the involve partnership Master Student Database annually and compile Curriculum & with a local, state, outcomes related to projects in rural and/or Assessment federal, or tribal health underserved communities. Relevant outcomes will be Committee department updated in the annual Program Goals Assessment Report and shared with the Curriculum and Assessment Committee (each July). ILE faculty will update the APE and ILE database each semester. Data will be transferred to the Master Student Database. The program director will audit the Percent of ILE that Master Student Database annually and compile involve a partnership Curriculum & outcomes related to projects in rural and/or with a local, state, Assessment underserved communities. Relevant outcomes will be federal, or tribal health Committee updated in the annual Program Goals Assessment department Report and shared with the Curriculum and Assessment Committee (each July).

DIVERSITY & INCLUSION Goal 3. Recruit and retain diverse faculty and students who embody cultural inclusiveness.

Program assistants will update the Master Student Database annually (each August) upon matriculation of Student composition new students. The MPH director and program of priority coordinator will conduct an annual audit of students at Curriculum & underrepresented the end of each academic year (late May). Results will Assessment minority (URM) be compiled into an aggregate Program Goals Committee, populations Assessment Report that is disseminated to the Admissions Curricular and Assessment Committee (each July) for Committee, All consideration in decision making. The report is also Faculty Meeting, shared during the MPH All Faculty Retreat (each Advisory Board Percent of URM August) and Advisory Board for additional feedback. students graduating Results and final recommendations will be shared with within 3 years. the Admissions Committee in advance of reviewing applicants for the next academic year (each March). The MPH program director and program coordinator will conduct a Student Survey (annually in July), Alumni Survey (6-months post-graduation) and Exit Survey (upon graduation). Program assistants will audit and compile results at the end of each academic year (May). A summary of results from each type of survey will be compiled in an annual report and shared with Curriculum & Percent of students the Curriculum and Assessment and All Faculty Assessment rating the climate of Committees (each July). Student ratings and Committee, All the MPH program as perceptions of the overall climate, diversity, and Faculty Meeting, diverse and inclusive inclusion of the program will be updated in the Program Dean’s Council, Goals Assessment Report by the MPH director, which Advisory Board will be reviewed by the Curriculum and Assessment Committee (each July) and during the annual Faculty Retreat (August) for feedback and recommendations for improvement. Results will also be shared with the Dean’s Council (August) and Community Advisory Board (Fall) for feedback and decision-making. Proportion of URM candidates that The program director will request aggregate faculty comprise faculty applicant data from USD/SDSU Human Resources at Curriculum & applicant pool. the end of each academic year (late May) for any years Assessment in which an open search has occurred. The program Committee, All Percent of URM director will compile data in the Program Goals Faculty Committee, candidates invited to Assessment Report and share with the Curriculum and Dean’s Council, interview. Assessment Committee (each July), All Faculty Advisory Board Composition of URM Committee (each August), Dean’s Council (August), faculty in the MPH and Community Advisory Board (Fall). program (all faculty) MPH primary faculty will submit an annual evaluation Percent of MPH report by October 1, which includes a summary of all primary and non- progress on research, teaching, and service over the primary faculty will past year. USD faculty submit data via Digital Curriculum & participate in at least 1 Measures and SDSU faculty submit data via a Faculty Assessment rigorous training (e.g., Annual Review form. With these reports, the MPH Committee, All certificate, continuing director will audit annually (each December) and Faculty Committee, education) on topics update outcomes in the Program Goals Assessment Dean’s Council, related to health equity Report. Items will be placed on All Faculty Committee Advisory Board in culturally and meeting agendas in the Spring to review professional socioeconomically development options. Outcomes will be shared with diverse communities the: Curricular & Assessment Committee (each July),

All Faculty Committee during the annual MPH Faculty Retreat (each August), with the Dean’s Council (every August), and Advisory Board (each Fall).

All faculty submit an annual self-evaluation report by October 1 of each year, which includes a summary of all progress on research, teaching, and service over Percent of MPH the past year. With these reports, the MPH director will primary and non- audit annually (each December) and update outcomes Curriculum & primary instructional in the Program Goals Assessment Report. Items will be Assessment faculty participating in placed on All Faculty Committee meeting agendas in Committee, All diversity and inclusion the Spring to review professional development options. Faculty Committee training The report will be reviewed by Curriculum and Assessment Committee (each July) and shared during the annual faculty retreat (August).

SERVICE Goal 4. Enhance the program’s involvement in public health service to rural and underserved communities in South Dakota. Students will be required to upload annual resumes to their student portfolio during the spring semester via online learning platform. In addition, students will Number of MPH complete an annual survey specifically identify the Curriculum & students engaging in community service and professional development Assessment community service events completed during the academic year. The Committee, All and professional program coordinator will audit annually (May) and Faculty Committee development update outcomes in the Program Goals Assessment Report The report will be reviewed by Curriculum and Assessment Committee (each July) and shared during the annual faculty retreat (August). All faculty submit an annual self-evaluation report by Number of MPH October 1 of each year, which includes a summary of primary and non- all progress on research, teaching, and service over primary instructional Curriculum & the past year. With these reports, the MPH director will faculty engaging in Assessment audit annually (each December) and update outcomes extramural service Committee, All in the Program Goals Assessment Report. The report with a focus on rural Faculty Committee will be reviewed by Curriculum and Assessment and underserved Committee (each July) and shared during the annual populations faculty retreat (August). Faculty identify and discuss training opportunities during faculty meetings to plan for each year. To track Number of completion, faculty. will complete a faculty survey each professional semester identifying any professional development development activities activities developed by the program. In addition, the Curriculum & (e.g., webinars, survey will confirm completed trainings, publications, Assessment collaboration and presentations by faculty. This information will be Committee, All meetings) developed collected by the program assistant and reviewed by the Faculty Committee by the MPH program Program Coordinator annually (May). The report will be for the public health reviewed by Curriculum and Assessment Committee workforce (each July) and shared during the annual faculty retreat (August).

2) Briefly describe how the chosen evaluation methods and measures track the program’s progress in advancing the field of public health (including instruction, scholarship and service) and promoting student success.

The evaluation methods and measures outlined in Template B5-1 provide a framework for our program to track progress, identify gaps and areas of strength, and respond strategically to meet established goals. The evaluation plan and measures align with our program’s four overarching goals including academic excellence, public health research and practice, diversity and inclusion, and service. Objectives that promote student success are embedded within each of the four goals ranging from student performance across MPH competencies to recruiting and retaining a culturally diverse student body and faculty. Furthermore, our program’s commitment to rural and underserved communities is demonstrated through objectives and measures that track student and faculty engagement in research and service to rural and Medically Underserved Areas and Populations (MUA/P).

Additional evaluation methods, measures, and protocols that inform our program’s goals and objectives are outlined in Table B5.

Table B5. Program and Institutional Survey and Evaluation Methods Survey/Method Implementation Frequency Protocol Alumni Survey All alumni 6-12 Annual • Results will be compiled annually months post- by program director and graduation coordinator and shared in Student Survey All active students Annual Program Goals Assessment Exit Survey Delivered upon One-time completion of the per student ILE and/or upon at point of graduation graduation Course IDEA All students enrolled Semester • Individual results will be reviewed Evaluation in PUBH courses at each semester by faculty member the end of each and program director semester • Aggregate results will be compiled annually and shared in Program Goals Assessment Key Curricular All faculty teaching Semester • Program coordinator will conduct Assessment PUBH courses at at end of each semester the end of each • Results will be compiled annually semester and shared in Key Curricular Assessment report Student Entry Delivered upon Annual • Results will be added to the Survey acceptance into access database to ensure program updated information • Information on resource needs and career interests of cohort will be shared in All Faculty committee meeting. Portfolio (Fall All students Annual • Program coordinator will review at 2020) end of each academic year • Results will be compiled annually and shared in Program Goals Assessment

APE & ILE All students Annual • Program coordinator and course Preceptor faculty will review at end of each Evaluation academic year • Results will be compiled annually and shared in Program Goals Assessment Faculty All faculty Annual • Individual results will be reviewed Performance with faculty member and program Evaluation director • Aggregate results will be compiled annually and shared in Program Goals Assessment Faculty and Staff All faculty and staff Annual • Program director will review Survey (Fall results and share in all faculty 2020) meeting Human All faculty Annual • Data on URM faculty composition Resources and faculty applicant pools will be reviewed annually by program director • Aggregate results will be compiled annually and shared in Program Goals Assessment Public Health Members of South Annual • Results will be collected from the Workforce Survey Dakota Public online survey and reviewed by Health Association program coordinator and program (SDPHA), South director Dakota Department • Results will be compiled annually of Health, and and disseminated to SDPHA Partners members, faculty, alumni and students. Key Informant Selected members As needed • To gather follow-up, in-depth Interviews of Advisory Board insight on specific issues and alumni

3) Provide evidence of implementation of the plan described in Template B5-1. Evidence may include reports or data summaries prepared for review, minutes of meetings at which results were discussed, etc. Evidence must document examination of progress and impact on both public health as a field and student success.

The Annual Program Goals Assessment Report for academic years 2016 to 2019 (summer and fall only) is available in the ERF (see B5-1 MPH Program Goals Assessment Report, AY 16-19), as well as minutes from committee meetings at which the report was discussed (B5-2 MPH Curriculum & Assessment Meeting Notes 11-01-19 and B5-3 MPH Curriculum and Assessment Meeting Notes 7-31-2020). The report includes a summary of progress toward meeting programmatic goals in areas of teaching, scholarship and research, diversity and inclusion, and service. The report is compiled annually and reviewed by bodies outlined in Template B5-1. The Program Director and Program Coordinator will identify gaps during the review process and develop and share a plan for improvement with the ad hoc CEPH Accreditation Committee.

The Key Curricular Assessment Report for 2019-2020 is available in the ERF (D2-5 Key Curricular Assessment Report 2019-2020), as well as minutes from the committee meetings in which it was discussed.

Reports of the Student Survey (H1-6 Current Student Survey in ERF) and Alumni Survey (B4-2 MPH Alumni Survey Report, 2020 in ERF), are available, as well as minutes from the committee

meetings in which it was discussed. Reports of the Exit Survey (see B5-5 MPH Graduate Exit Survey Report 2020 and B5-6 MPH Graduate Exit Results 2019-2020) and Public Health Workforce Survey (B5-7 Public Health Workforce Survey Results, August 2020) are also included.

Other data collection tools (e.g., Faculty Database) will be updated and administered during the upcoming academic year (2020). Data will be collated and shared with the parties as noted above.

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

A major weakness of our evaluation process was the delay in finalizing measures and reporting. Original programmatic goals, objectives, and measures were tweaked across summer and fall 2019. A substantial revision was necessary given the hiring of our inaugural director and three primary instructional faculty during the 2019 academic year, as well as the need to incorporate feedback from our Community Advisory Board. While the revisions strengthen our evaluation methods and practices, the process delayed our annual reporting and dissemination of some findings. To date, we have shared evaluation results with the MPH Curriculum and Assessment Committee. We have shared results at multiple All Faculty Committee meetings and during the Community Advisory Board meeting in January 2020.

Other weaknesses include the measures that are newer, including the Exit Survey and Key Curricular Assessment. We are providing data from fall 2019 and spring 2020 for the Exit Survey and the 2019-2020 Key Curricular Assessment report, and plan to finalize and implement the Student Portfolio during the fall 2020 semester. Plans for addressing these issues are described in B6.

Section B

B6. Use of Evaluation Data

Intentionally blank

Divider back

B6. Use of Evaluation Data

The program engages in regular, substantive review of all evaluation findings, as well as strategic discussions about the implications of evaluation findings.

The program implements an explicit process for translating evaluation findings into programmatic plans and changes and provides evidence of changes implemented based on evaluation findings.

1) Provide two to four specific examples of programmatic changes undertaken in the last three years based on evaluation results. For each example, describe the specific evaluation finding and the groups or individuals responsible for determining the planned change, as well as identifying the change itself.

Example 1: Our program is committed to supporting local, state, federal, and tribal health departments. However, results from our evaluation report indicate on some indicators we fall below our target of 50% of APE and 33% of ILE projects involving a partnership with one of these agencies, partially due to the impact of covid-19 on preceptors from these organizations to have time to mentor. The public health infrastructure in South Dakota is limited, as the South Dakota State Department of Health is centralized with no county health departments and only one public health nurse to serve at the county-level. In recent years, our students have completed the APE and ILE in partnership with state health departments in South Dakota, Iowa, and Nebraska; county health departments in Colorado and Washington; a city health department in South Dakota; and a tribal health department on the White Earth Nation. In the next year, we plan to strengthen our relationships with these agencies and streamline placements and preceptors for APE and ILE projects. This process involves curating an annual list, for which our program solicits APE and ILE project opportunities with local, state, federal, and tribal health departments in South Dakota. Advanced planning and coordination will ensure a select number of APE and ILE projects will be available each year and will become an evergreen model to further strengthen and build the public health infrastructure in South Dakota.

Example 2: While our program faculty are actively engaged in the dissemination of public health practice and research, historically the program has not developed exclusive professional development activities for the public health workforce as shown in the annual evaluation report (see ERF, B5). To address this gap, we plan to use data from a 2018 and 2020 public health workforce survey conducted by our MPH faculty in partnership with the South Dakota Public Health Association and information from the advisory board meetings to inform the development of at least two professional development activities (e.g., webinar, workshop) each year. The public health workforce survey will be revised and administered annually prior to the South Dakota Public Health Association (SDPHA) conference, which draws a diverse group of public health, clinical, and community-based practitioners and researchers. Survey results will also inform professional development activities for our students, all of whom are expected to participate in at least one professional development activity per year. Survey results will be presented during the SDPHA conference and additional feedback gathered. We are happy to report these changes resulted in several professional development activities offered in Spring 2020.

Example 3: Several areas of our evaluation plan are missing one or more years of data. To address these gaps, we launched an Exit Survey in fall 2019 and a Key Curricular Assessment report, and plan to develop a Student Portfolio in fall 2020. The Exit Survey will help us confirm post-graduation contact information, immediate plans for employment or continuing education, and a brief survey of satisfaction with the overall program. The Key Curricular Assessment report shows outcome data for key assessments aligned with each MPH competency. This information will be used to track progress toward our program goals. Results will be shared with the MPH Curriculum and Assessment Committee and All Faculty Committee on an annual basis. Finally, the Student Portfolio will include key deliverables from the APE, final products from the ILE, and documentation of professional development and service learning. The portfolio will be audited annually by the

Program Coordinator and reviewed by a committee of faculty upon each student’s completion of the ILE.

2) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Please see response to B6.1.

Section C

C1. Fiscal Resources

Intentionally blank

Divider back

C1. Fiscal Resources

The program has financial resources adequate to fulfill its stated mission and goals. Financial support is adequate to sustain all core functions, including offering coursework and other elements necessary to support the full array of degrees and ongoing operations.

1) Describe the program’s budget processes, including all sources of funding. This description addresses the following, as applicable:

a) Briefly describe how the program pays for faculty salaries. If this varies by individual or appointment type, indicate this and provide examples. If faculty salaries are paid by an entity other than the program (such as a department or college), explain.

The MPH program budget, including faculty salaries, is supported through tuition and fee revenues generated by the program as well as through institutional support. Additionally, extramural funds supplement the MPH program’s resources through the grants and contracts awarded to MPH faculty and interdisciplinary grants in which MPH faculty participate.

MPH faculty salaries are “guaranteed,” meaning they are fully budgeted on MPH tuition and fees which allows for an appropriate balance between teaching, research and other scholarly activities. When faculty secure extramural support for salaries and benefits, the tuition and fee savings become available for reinvestment in the program.

b) Briefly describe how the program requests and/or obtains additional faculty or staff (additional = not replacements for individuals who left). If multiple models are possible, indicate this and provide examples.

Program budgets are adjusted annually based on state, SDBOR, and institutionally approved salary and benefit increases. MPH program requests for additional FTE or operating support must first be requested through the respective deans (USD’s School of Health Sciences for USD’s MPH program budget or SDSU’s College of Pharmacy & Allied Health Professions for SDSU’s MPH program budget). Increases to program budgets are considered within the framework of the university and school’s strategic priorities and funding availability. To request additional faculty or staff, specific policies and procedures must be followed. GR, which returns tuition and fees and indirects to the schools and colleges generating the revenues. As such, the ability to expand MPH program faculty and staff is considered within the framework of each school/college strategic priorities and funding availability, whether through new resources or reallocation of existing resources. If the school/college approves the budget increase, appropriate institutional procedures are followed to expand or realign the budget as requested.

If the program request for faculty/staff expansion is determined to be a school/college priority, but school/college resources are not available to support, institutional level support may be sought within each institution’s defined procedures and in consideration of institutional priorities and funding availability.

c) Describe how the program funds the following:

a. operational costs (programs define “operational” in their own contexts; definition must be included in response)

Direct operational costs are funded through the tuition and fees generated by the MPH program as well as through institutional support. Direct operational costs include MPH faculty and staff salaries and benefits and MPH operating expenses (OE) categorized under travel, supplies and materials, contractual services and capital assets. MPH OE includes support for expenses such as professional development for the MPH program

director, MPH program coordinator and faculty, accreditation related costs and annual dues, travel, new course development costs, general supplies and materials, office furnishings, and technology. Direct operating expenses may also be supplemented through extramural funding from MPH faculty grants and contracts.

Indirect operational costs, such as university overhead for administrative services, facilities, information technology support, etc., are funded at the college/school level based on cost allocation methodologies adopted under RCM.

b. student support, including scholarships, support for student conference travel, support for student activities, etc.

In addition to tuition, MPH students pay an MPH delivery fee per credit hour for all MPH courses. MPH delivery fee revenues can be used to support student travel expenses. The MPH program covers membership and registration expenses for all MPH students who participate or present at the South Dakota Public Health Association Conference and those selected to present at the Annual American Public Health Association Meeting.

c. faculty development expenses, including travel support. If this varies by individual or appointment type, indicate this and provide examples

The MPH program budget supports $2,000 per year for professional development costs for each MPH primary faculty member as well as the program director. Additionally, school/college level support may be solicited/awarded for exceptional or unique opportunities, such as leadership development programs. Extramural funds may provide additional support for travel and development. d) In general terms, describe how the program requests and/or obtains additional funds for operational costs, student support and faculty development expenses.

As with request for additional faculty/staff, budget expansion in support of new or increased direct OE must first be requested through the respective school/college deans. The request is considered within the framework of each school/college strategic priorities and funding availability, whether through new resources or reallocation of existing resources. If the school/college approves the budget increase, appropriate institutional procedures are followed to expand or realign the budget as requested.

If the MPH program is seeking funds to support a “one-time” or unique/exceptional cost in a fiscal year, support is requested through the respective school or college dean and can be elevated to a request for institutional support if appropriate. e) Explain how tuition and fees paid by students are returned to the program. If the program receives a share rather than the full amount, explain, in general terms, how the share returned is determined. If the program’s funding is allocated in a way that does not bear a relationship to tuition and fees generated, indicate this and explain.

Both USD and SDSU operate under the responsibility centered management (RCM) financial model which returns revenues generated by programs to the schools and colleges to support both direct program costs as well as contributions for university administrative overhead. A portion of tuition revenues are retained as they are net of certain mandatory assessments, such as fixed percentages for higher education facilities fund contributions or bad debt allowances. The percent of tuition returned can vary depending on the tuition rate (i.e., state-support vs self-support, resident vs non-resident). The school/college receives 100% of program level fee revenue, including the MPH delivery fee.

Each school/college is responsible for ensuring proper alignment of program revenue and budget and to balance the school/college collective resources so that programmatic academic, research and service missions and contributions to university overhead are met. As with program budget expansion, any budget contraction is considered within the framework of strategic priorities and funding availability. Typically, a program budget remains steady even through minor enrollment fluctuations. Annually, if the retained tuition and fees on MPH program courses are less than the program budget, the school/college level support fills the gap. If a decrease in enrollment appears to be long-term in nature and produces a material change in school/college revenues and operating margins, MPH program leadership and the deans work collaboratively to assess how to balance the change in revenues while maintaining the ability to deliver programming within accreditation and the university’s academic standards.

f) Explain how indirect costs associated with grants and contracts are returned to the program and/or individual faculty members. If the program and its faculty do not receive funding through this mechanism, explain.

Under RCM, the school/college retains a percentage of the indirect costs generated on grants and contracts (90% USD and 80% SDSU). As each school/college owns responsibility for balancing the collective resources with programmatic budget needs and contribution to university overhead, the deans have discretion in establishing or negotiating the share of indirect costs returned to a program or faculty. For USD, the dean of the School of Health Sciences returns 16% of indirect costs to the program generating the award. For SDSU, 40% is allocated to overhead costs related to research and 40% is returned to the College for reinvestment in research related activities.

If the program is a multi-partner unit sponsored by two or more universities (as defined in Criterion A2), the responses must make clear the financial contributions of each sponsoring university to the overall program budget. The description must explain how tuition and other income is shared, including indirect cost returns for research generated by the public health program faculty appointed at any institution.

Each institution is subject to the policies and procedures outlined by their institution and the South Dakota Board of Regents. The budget of the MPH program at one institution is independent of the budget of the MPH program at the other institution. MPH’s Dean’s Council includes a Dean from each institution ensuring the institutions’ policies and procedures are implemented as well as identifying and discussing resource needs within the program.

2) A clearly formulated program budget statement in the format of Template C1-1, showing sources of all available funds and expenditures by major categories, for the last five years. Joint USD – SDSU MPH Program Sources of Funds and Expenditures by Major Category, 2015 to 2020 FY 15 FY 16 FY 17 FY 18 FY 19 FY 20 Source of Funds Tuition & Fees 13,941 102,149 188,849 182,663 240,206 271,988 State Appropriation 0 0 0 0 0 0 University Funds 103,494 60,051 15,742 0 0 0 Contractual 0 0 0 3,504 17,518 0 Grants/Contracts 0 13,127 20,339 35,489 91,673 181,371 Indirect Cost Recovery 0 0 0 842 2,133 27,530 Endowment 0 0 0 0 0 0 Gifts 0 0 0 0 0 0

Joint USD – SDSU MPH Program Sources of Funds and Expenditures by Major Category, 2015 to 2020 FY 15 FY 16 FY 17 FY 18 FY 19 FY 20 MPH Delivery Fee 11,580 69,919 123,366 156,083 174,480 136,616 Total 129,015 245,246 348,296 378,581 526,010 617,504

Expenditures Faculty Salaries & Benefits 45,040 108,038 147,260 212,360 242,504 463,135 Staff Salaries & Benefits 41,247 72,448 72,895 91,484 110,537 55,766 Operations 0 0 0 0 0 0 Travel 2,857 0 6,952 4,423 9,763 11,509 Student Support 0 0 0 0 12,722 8,899 University Tax 0 0 0 0 0 0 Contractual 14,738 26,636 7,321 9,794 49,631 33,997 Supplies 786 1,689 877 879 124 1,451 Grants & Subsidies 0 0 0 0 10,962 68,599 Capital Assets 2,596 2,431 343 875 6,172 4,705 Total 107,264 211,242 235,648 319,815 442,415 648,062

If the program is a multi-partner unit sponsored by two or more universities (as defined in Criterion A2), the budget statement must make clear the financial contributions of each sponsoring university to the overall program budget.

The financial contributions from each university are outlined in the budget table below. While the institutions work collaboratively on the MPH program, the tuition and other income is not shared between the institutions. Each institution operates independently from a financial perspective.

Source of Funds and Expenditures by Major Category, 2015 to 2020 FY 15 FY 16 FY 17 FY 18 FY 19 FY 20 SDSU USD SDSU USD SDSU USD SDSU USD SDSU USD SDSU USD Source of Funds Tuition & Fees 2,620 11,321 39,440 62,709 69,183 119,666 71,053 111,610 70,876 169,330 66,879 205,109 State Appropriation University Funds 57,810 45,684 60,051 15,742 Contractual 3,504 17,518 Grants/Contracts 13127 20339 35,489 91,673 181,371 Indirect Cost Recovery 842 2,133 27,530 Endowment Gifts MPH Delivery Fee 2,470 9,110 12,348 57,571 30,946 92,420 36,855 119,228 59,281 115,199 43,996 92,620 Total 62,900 66,115 111,839 133,407 115,871 232,425 111,412 267,169 147,675 378,335 110,875 506,629

Expenditures Faculty Salaries & 45,040 19,622 88,416 42,198 105,062 57,648 154,712 63,392 179,112 96,149 366,986 Benefits Staff Salaries & Benefits 35,900 5,347 67,123 5,325 67,463 5,432 70,395 21,089 77,192 33,345 22,148 33,618 Operations Travel 996 1,861 1,840 5,112 4,423 9,763 3,750 7,759 Student Support 12,722 8,899 University Tax Contractual 1,099 13,639 4,250 22,386 2,972 4,349 630 9,164 2,456 47,175 5,043 28,954 Supplies 774 12 785 904 12 865 879 124 593 858 Grants & Subsidies 10,962 68,599 Capital Assets 2,381 215 1,151 1,280 343 875 6,172 4,705 Total 41,150 66,114 92,931 118,311 114,485 121,163 128,673 191,142 143,040 299,375 127,683 520,379

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The MPH program has established itself as a financially healthy program in a short time. Addition of faculty and staff FTEs will be needed to continue expansion. Therefore, in the upcoming years, a close eye on the financial status of the MPH program will be needed to forecast where to best invest and grow the program.

Intentionally blank

Section C

C2. Faculty Resources

Intentionally blank

Divider back

C2. Faculty Resources

The program has adequate faculty, including primary instructional faculty and non-primary instructional faculty, to fulfill its stated mission and goals. This support is adequate to sustain all core functions, including offering coursework and advising students. The stability of resources is a factor in evaluating resource adequacy.

Students’ access to a range of intellectual perspectives and to breadth of thought in their chosen fields of study is an important component of quality, as is faculty access to colleagues with shared interests and expertise.

All identified faculty must have regular instructional responsibility in the area. Individuals who perform research in a given area but do not have some regular expectations for instruction cannot serve as one of the three to five listed members.

1) A table demonstrating the adequacy of the program’s instructional faculty resources in the format of Template C2-1.

Template C2-1. Program’s Instructional Faculty ADDITIONAL MASTER'S DOCTORAL BACHELOR'S + FACULTY FACULTY CONCENTRATION PIF 1* PIF 2* 3^ PIF 4* PIF 5*

Generalist Chelsea Susan Aaron PIF: 1 MPH Wesner Puumala Hunt NA NA 1.0 1.0 1.0 Non-PIF: 10

TOTALS: Named PIF 3 Total PIF 4 Non-PIF 10

2) Explain the method for calculating FTE for faculty in the templates and evidence of the calculation method’s implementation. Programs must present calculation methods for primary instructional and non-primary instructional faculty.

Table C2.1. Method for Calculating Primary Instructional and Non-Primary Instructional Faculty Primary Instructional Months/FTE FTE Calculation Faculty Hunt, Aaron 12 months, 100% 100% of Dr. Hunt’s position is administrative, teaching, research, and service for MPH program Puumala, Susan 12 months, 100% 100% of Dr. Puumala’s position is teaching and service for MPH program Strobel, Susan 9 months, 50% 50% of Dr. Strobel’s position is teaching, research, and service for the MPH program. She

also spends 50% with the USD SHS Nursing program. TBH – USD Assistant 12 months, 50% 50% of the TBH Assistant Professor position will Professor be teaching and service for MPH program. They will spend the other 50% in policy and outreach activities for the SHS Dean’s office. Non-Primary Instructional Faculty Huang, Yen-Ming 12 months, 36% 36% of Dr. Huang’s position is teaching, research, advising, and service for MPH program. His primary appointment is in the Department of Allied and Population Health. Jepperson, Shelby Fall semester, 20% of Ms. Jepperson’s time Fall 2020 is spent 20% teaching for the MPH program. Her primary role is a Project Coordinator for the SHS Dean’s office. Kenyon, DenYelle 12 months, 50% 50% of Dr. Kenyon’s position is administrative, teaching, research, and service for MPH program. She is shared 50% with the SSOM for Diversity and Inclusion efforts. Killian, Scott Adjunct N/A Lucas-Molitor, Fall semester, Dr. Lucas-Molitor’s is teaching for the MPH Whitney 20% program on overload Fall 2020. Her primary role is an Assistant Professor in the Department of Occupational Therapy. Melstad, Sandra Adjunct N/A Pinto, Sharrel 12 months, 13.3% of Dr. Pinto’s time is administrative, 13.3% teaching, and service for MPH program. She also has administrative duties for the other parts of the Department of Allied and Population Health. Stenvig, Thomas 12 months, 26% 26% of Dr. Stenvig’s time is dedicated to teaching and service for MPH program. His main appointment is in the SDSU School of Nursing. Miller, Erin 12 months, 36% 36% of Dr. Miller’s position is teaching, research, advising, and service for MPH program. Her primary appointment is in the Department of Allied and Population Health. Wesner, Chelsea 12 months, 75% 35% of Ms. Wesner’s position is research for the OD2A project which includes supervising ILE students. She has an Intergovernmental Personnel Agreement with the CDC for 35% of her time, and 5% of her time is service for the MPH program.

3) If applicable, provide a narrative explanation that supplements reviewers’ understanding of data in the templates.

The MPH Program has 4 primary instructional faculty and 10 non-primary instructional faculty. Several of these faculty (although not all) are engaged in general advising and career counseling and advising in the Integrative Learning Experience. We have administrative/staff support for the program which includes the MPH program director, department chair at SDSU, associate dean for academic programs at SDSU, and finance coordinators and administrative assistants at both

universities. As many faculty and staff wear several hats in their roles, many of them split time with other departments, so we have indicated % FTE dedicated to the MPH program.

The data for current advising ratios in C2-2 was calculated based on data as of 8/19/20. Faculty who do not have any students were not included in the ratio calculations. For the general advising and career counseling, students on leave of absence were counted with active students in the ratio calculations. For the Integrated Learning Experience, both primary and secondary research advisors were counted in the ratios. See back up documentation in the electronic resource file (ERF C2-1 Advising Ratio Documentation; workload examples in C2-2 through C2-6).

4) Data on the following for the most recent year in the format of Template C2-2. See Template C22 for additional definitions and parameters.

Template C2-2. General advising and career counseling Degree level Average Min Max Master’s 1:12 1:2 1:27 (primary instructional faculty & non-primary instructional faculty)

Advising in MPH integrative experience Average Min Max 1:2 1: 0.5 1:3.5

5) Quantitative data on student perceptions of the following for the most recent year:

a. Class size and its relation to quality of learning (e.g., The class size was conducive to my learning)

In spring/summer 2019, the CEPH Accreditation Task Force developed a survey to be administered to current Master of Public Health students at the University of South Dakota and South Dakota State University. The survey was developed using best practices, CEPH requirements and the needs of the program. The primary purpose of the survey was to collect feedback from current students to help improve the MPH program and help current and future students complete their MPH degrees in a timely fashion. In spring, climate and diversity related questions were added to the survey.

The survey software, QuestionPro, was used to distribute the survey to the current students. An email invitation and two follow-ups were sent by Ms. Chelsea Wesner, USD MPH faculty member and Dr. Aaron Hunt, SDSU MPH faculty member. The student’s home institution email address was used to request survey participation. The survey was open for two weeks. Only non-respondents were contacted in the reminder emails. If a bounce back/filter were received, the email would be sent to an alternate email address, if applicable. The response rate for the Student Survey in 2019 was 45%. The response rate for the Student Survey in 2020 was 52%.

Current MPH students were asked to rate their perceptions on the class size and its relation to quality of learning. 97% of the survey respondents from the MPH current student survey agreed or strongly agreed that the class size is conducive to learning in 2019. 93% of the survey respondents from the MPH current student survey agreed or strongly agreed that the class size is conducive to learning in 2020. The difference between the means is not statistically significant.

2019 2019 2019 2020 2020 2020 Statistically Overall Overall Mean Overall Overall Mean Significant Question # % Overall # % Overall (p<0.05) C2-C (5): The class size is conducive to my learning. 1=Strongly Disagree 0 0% 1 3% 2=Disagree 1 3% 1 3% 3.39 3.40 No 3=Agree 17 55% 13 43% 4=Strongly Agree 13 42% 15 50%

b. Availability of faculty (i.e., Likert scale of 1-5, with 5 as very satisfied)

MPH students were asked to rate their perceptions on the availability of faculty. The percent of respondents satisfied or very satisfied with the availability of faculty increased from 80% in 2019 to 97% in 2020.

2019 2019 2019 2020 2020 2020 Statistically Overall Overall Mean Overall Overall Mean Significant Question # % Overall # % Overall (p<0.05) C2-C (5): How satisfied were you with the availability of faculty? 1=Very Dissatisfied 3 10% 1 3% 2=Dissatisfied 3 10% 0 0% 3.03 3.37 No 3=Satisfied 14 47% 16 53% 4=Very Satisfied 10 33% 13 43%

6) Qualitative data on student perceptions of class size and availability of faculty.

Comments from the current students align with the quantitative data presented above. For example, students shared: • “I like when the class size is under 30 individuals.” • “I have had no issues or concerns with the class sizes. I thought they have worked very well for me.”

In 2019, 80% of the survey respondents were satisfied or very satisfied with the availability of faculty, and although high, this appeared to be an opportunity for improvement. SDSU’s first program coordinator signed a contract for FY20 but then voluntarily reduced their time to 45% to begin transitioning out of the position in June 2019. As outlined in several of the student’s comments (see below), some students were frustrated by the lack of availability of certain faculty and the response times to emails. • “None at this time- everyone has always been timely with their responses.” • “Responding to students’ questions via email, phone calls and posts on discussion boards. There is one instructor in particular that 'ghosted' students and would not respond to any of the above. This class also lacked rubric for assignments or any type of guidelines of what was expected out of the assignment. I felt that class to be a waste of my time and money. I am currently avoiding any other classes led by this instructor.” • “There needs to be someone in charge of the program and multiple faculty appointed as advisers to the students. It is a distance learning program so communication is essential to the success of the students. I have had a decent experience with course instructors

communicating and making assignments clear, but terrible experience with career guidance or any guidance from my advisor.” • “I think one thing that was frustrating with my courses is that both times my Professor lacked professional communication. She did not reply to email for weeks or even up to a month. Additionally, she didn't have clear expectations with grades and the syllabus.” • “They have been very available and helpful with answering questions.” • “When an instructor, or adviser plans on leaving the organization, it would be beneficial to have their emails forwarded to other committed staff so students don't feel lost, especially in the online mph program. In the summer, I was enrolled in Public Mental Health with {faculty/staff name} and they left the class unattended for 7 weeks, causing frustrations, uncertainty and overall negative feelings towards them and the university.” • “Stick to response times as indicated in the syllabus. Provide more interaction with course materials such as lectures, responses to discussions (other than grades), or checking in more frequently. Sometimes I feel as in the professors just provide materials and then expect it to be completed with little to no guidance.”

Recognizing this as an area of improvement, the department head, program coordinator, and program director quickly devised a plan to increase faculty availability and emphasized student- centered mentoring practices. In 2020, 97% of the survey respondents were satisfied or very satisfied with the availability of faculty. Example comments from the current students from 2020 regarding faculty availability are included below.

• “There is no improvement needed. Faculty has time for students and are readily available to discuss students' needs.” • “I felt that staff were available whenever I would reach out and request meetings or ask questions. However, with some staff it did feel like they were being bothered by reaching out to them and some didn't really encourage reaching out to them. Understandably, faculty are pulled in many directions but I often felt isolated or forget by faculty.” • “Office hours (virtual is fine) outside of 8-5, which I understand is inconvenient but helpful for students employed full time. Most faculty have been responsive to questions and communications. A couple were slow to respond (beyond syllabus established time frames) and one, in particular, was unresponsive for weeks.” • “Generally all were very responsive to emails.”

A full report of results from the student survey can be found in the Electronic Resource File (C2-7 Current Student Survey Report 2019 and 2020).

7) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The MPH program has faced some challenges. During the early years of the program, the interim program director was Michael Lawler, Dean of the USD School of Health Sciences. Some faculty have left for other positions, which left a greater load for the remaining faculty/staff. This was compounded by the rate of growth of the program, as well as large faculty/student ratios for advising. Much recent stability has come with the hiring of a permanent program director at USD, a new program coordinator/assistant professor at SDSU, conversion of the program coordinator to a faculty position, and replacing previous faculty.

Additional measures were put into place to provide a better experience for MPH students. A new oversight structure was put into place for the program at SDSU, which ensures greater support for the program coordinator, as well as the MPH program in general. Similarly, USD’s new dean of the School of Health Sciences was installed June 2019, providing guidance to the program director. At SDSU, the Department of Allied and Population Health was implemented in July 2019. This structural change provides support for the MPH program, builds a strong foundation for

collaboration among disciplines, and creates research synergy. With the new program director in place March 2019 and the MPH coordinator/assistant professor in place as of September 2019, the MPH program has human resources in place to provide exceptional customer service to the students. Additionally, there is dedicated administrative support both at the USD Department of Public Health and Health Sciences and SDSU Department of Allied and Population Health to provide staff support to the programs’ faculty and students in the department.

Section C

C3. Staff and Other Personnel Resources

Intentionally blank

Divider back

C3. Staff and Other Personnel Resources

The program has staff and other personnel adequate to fulfill its stated mission and goals. The stability of resources is a factor in evaluating resource adequacy.

1) A table defining the number of the program’s staff support for the year in which the site visit will take place by role or function in the format of Template C3-1. Designate any staff resources that are shared with other units outside the unit of accreditation.

Template C3.1. Role and FTE of Staff Role/function FTE Academic Programs/ Curriculum/Assessment (SDSU) 0.083 Finance Coordinator (SDSU) 0.05 Finance Coordinator (USD) 0.05 Administrative Assistant (SDSU) 0.25 Administrative Assistant (USD) 0.25

2) Provide a narrative description, which may be supported by data if applicable, of the contributions of other personnel.

Table C3.1. Contributions of Other Personnel Role/function FTE Narrative Associate Dean for 0.083 Oversees MPH program curricular and assessment Academic Programs outcomes. Member of Accreditation and Curriculum & (SDSU) Assessment committees. Finance Coordinator 0.05 Maintains oversight of budget and financial resources (SDSU) for USD side of MPH program. Finance Coordinator 0.05 Maintains oversight of budget and financial resources (SDSU) for SDSU side of MPH program. Administrative Assistant 0.25 Provides administrative program support, assists in (SDSU) tracking student data, manages admissions process (e.g., background checks), and fiscal needs (e.g., paying invoices). Administrative Assistant 0.25 Provides administrative program support, assists in (USD) tracking student data, manages admissions process (e.g., background checks), and fiscal needs (e.g., paying invoices).

3) Provide narrative and/or data that support the assertion that the program’s staff and other personnel support is sufficient or not sufficient.

As detailed above, much of the program’s support has been established more recently. We are hopeful that once everyone is fully onboarded and up-to-speed with the various programs and university systems, that we are fully supported with this level of student enrollment. A growing student enrollment may require greater program administrative support.

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The program is fortunate to have much experience and energy from faculty and administration. We can continue to find new ways to find new efficiencies in the program management. For example, we are still finalizing a new database for student tracking. Program administration are always looking for methods to help with efficiency and more fully utilize technology that our students are comfortable with and to provide the optimal educational experience.

Section C

C4. Physical Resources

Intentionally blank

Divider back

C4. Physical Resources

The program has physical resources adequate to fulfill its stated mission and goals and to support instructional programs. Physical resources include faculty and staff office space, classroom space, student shared space and laboratories, as applicable.

1) Briefly describe, with data as applicable, the following. (Note: square footage is not required unless specifically relevant to the program’s narrative.)

• Faculty office space

Faculty who support the MPH program have dedicated office space. The MPH program director has an office space in both Sioux Falls, SD, Health Sciences Building and Lee Medical Building in Vermillion, SD. Space for the MPH faculty offices is in the Lee Medical Building in Vermillion, SD. SDSU’s MPH program coordinator and the rest of the Department of Allied and Population Health faculty have an office space within the Avera Health & Science Center Building of the College of Pharmacy and Allied Health Professions in Brookings, SD. SDSU faculty who teach and advise in the MPH program have dedicated office space in one of three locations: the Avera Health & Science Center building in Brookings, SD (Department of Allied and Population Health), the Community College for Sioux Falls in Sioux Falls, SD (Department of Allied and Population Health), and the Wagner Building in Brookings, SD (College of Nursing).

• Staff office space

Office space for a support staff member at USD is provided in the Lee Medical Building in Vermillion. With the growth of the MPH program, a full-time program assistant was hired in summer 2019 at SDSU to support the MPH program as well as the other programs and needs of the Department of Allied and Population Health. The full-time support staff has an office within the Avera Building.

• Classrooms

The MPH program is completely online, with an internship/practicum component, that does not require classroom usage.

• Shared student space

Conference rooms and academic support spaces are located throughout the USD Lee Medical Building and Sioux Falls Health Sciences Building and are accessible to faculty, staff and students. The SDSU Avera Health and Science Center provides a common area on each floor which is a shared student space for collaboration, a quiet study area, and/or relaxation.

• Laboratories, if applicable to public health degree program offerings

Not applicable.

2) Provide narrative and/or data that support the assertion that the physical space is sufficient or not sufficient.

As described in C5, the information and technology resources are essential for the successful delivery of the MPH program. Both institutions provide a learning management system for course delivery and communication technology to allow online students to complete the program without

physically stepping on the campus. The physical resources provided by both institutions are sufficient to support the needs of the faculty, staff and students.

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

As an online program, physical resources are somewhat less critical and are sufficient at this time. The only limitation foreseen is in office space if additional faculty are added, and if those faculty require specialized research lab space.

Section C

C5. Information and Technology Resources

Intentionally blank

Divider back

C5. Information and Technology Resources

The program has information and technology resources adequate to fulfill its stated mission and goals and to support instructional programs. Information and technology resources include library resources, student access to hardware and software (including access to specific software or other technology required for instructional programs), faculty access to hardware and software (including access to specific software required for the instructional programs offered) and technical assistance for students and faculty.

1) Briefly describe, with data if applicable, the following: • library resources and support available for students and faculty

MPH students have access to the library resources through SDSU’s Briggs Library and USD’s I.D. Weeks and Wegner Health Science Library. All students (distance or on-campus) are provided the same services, except the library will mail books and other physical materials to distance students upon request. The library provides the following resources and services for MPH students: website access, research guide for evidence-based medicine and public health research projects, off-campus access to resources, and research assistance. In addition, both USD and SDSU libraries offers research instruction/orientation upon request from faculty.

Briggs Library has the following public health related books and journals for MPH students (see ERF C5-1 SDSU Public Health Book Titles and C5-2 SDSU Public Health Journal Titles). The books and journals include the following formats: print, online (ebooks and ejournals), and microfilm/microfiche. However, most of the titles are print or online.

Format Number

Books 6,899

Journals 960

AV Items 59

The Wegner Health Science Library has the following public health related journals for MPH students (see C5-3 USD Public Health Book Titles and C5-4 USD Public Health Journal Titles):

# Wegner Research Wegner # Core Access Level Core Access Topic Journals Available Journals Available

Health Services 16 13 37 26 Administration

Epidemiology 11 11 37 31

Biostatistics 12 12 16 15

Health Behavior & Education 16 13 37 32

• student access to hardware and software (including access to specific software or other technology required for instructional programs)

Students can access software at the library, through the D2L course shell, and through a virtual machine software which students can log into from off campus. Depending on the programs some are available for personal download, as well such as the R-Studio software. When software is required for a course, students are responsible for purchasing a license (e.g., SAS) and referring to the software company for questions.

Students have access to the following platforms: email (Microsoft Outlook), Desire2Learn (learning management system), WebAdvisor/Banner Self-Service (registration and course planning software to drop/add classes), and student portals with numerous links to resources (MyState - SDSU student portal and myUSD - USD student portal).

• faculty access to hardware and software (including access to specific software or other technology required for instructional programs)

Wireless access is provided in various areas throughout the campuses. All university computers access the network through a secure connection. Faculty can access software at the library, in class/labs, and through a virtual machine software which faculty can log into from off campus.

Faculty have access to the following platforms: Microsoft 365 email, Desire2Learn (learning management system) and WebAdvisor/Banner Self-Service (registration and course planning software). The technology centers provide the universities with the necessary technology for classroom presentations, conference rooms and videoconferencing needs which connect the campus with its off-campus sites through its management of the University’s network and its connection to the statewide network (Classroom Technology Services at SDSU and Information Technology Services at USD). Faculty also have access to the portals with numerous links to resources and SDSU has InsideState (a collaborative tool utilizing file, document, and calendar sharing, message boards and other cooperative features. It also promotes announcements and events, as well as information sharing on a University-wide or community-specific basis).

• technical assistance available for students and faculty

MPH faculty and staff have technical assistance available through the help desk. Support is available through email, phone, live chat, and face-to-face interaction. For statistical packages, technical assistance support is limited to installation issues, and students are directed to the software company for questions.

To determine which help desk the student should contact depends on which institution is teaching the course. For example, the SDSU help desk is responsible for providing assistance to students registered for any SDSU courses. This process is the same for USD taught courses.

2) Provide narrative and/or data that support the assertion that information and technology resources are sufficient or not sufficient.

On a recent survey of current MPH students, 93% of respondents indicated they are satisfied or very satisfied with the ease of collaboration through different technology platforms compared to 97% in 2019. With a 4-point Likert scale (1=very dissatisfied and 4=very satisfied), the average satisfaction rating was 3.40 for both 2019 and 2020. SDSU and USD has embraced the technological changes and adapted well to integrating technology into teaching, advising, and the entire program itself to allow students to stay current with the technology landscape.

2019 2019 2019 2020 2020 2020 Statistically Overall Overall Mean Overall Overall Mean Significant Question # % Overall # % Overall (p<0.05) How satisfied were you with the ease of collaboration through different technology platforms? 1=Very Dissatisfied 0 0% 0 0% 2=Dissatisfied 1 3% 2 7% 3.40 3.40 No 3=Satisfied 16 53% 14 47% 4=Very Satisfied 13 43% 14 47% The response rate for the Student Survey in 2019 was 45%. The response rate for the Student Survey in 2020 was 52%.

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Currently, the program has strong resources in information and technology resources, and alumni are quite satisfied with the online platform learning. We aim to maintain this through continuous adoption of technology and a student-centered approach to learning.

Intentionally blank

Section D

D1. MPH & DrPH Foundational Public Health Knowledge

Intentionally blank

Divider back

D1. MPH & DrPH Foundational Public Health Knowledge

The program ensures that all MPH and DrPH graduates are grounded in foundational public health knowledge.

The program validates MPH and DrPH students’ foundational public health knowledge through appropriate methods.

1) Provide a matrix, in the format of Template D1-1, that indicates how all MPH and DrPH students are grounded in each of the defined foundational public health learning objectives (1-12). The matrix must identify all options for MPH and DrPH students used by the program.

The foundational public health knowledge outcomes are taught and assessed in PUBH 702 Public Health Theory and Practice. Each week of the course focuses on a foundational public health content area. Instructional methods used include readings, videos, and recorded lectures. Students discuss the week’s content through small group discussion posts and take a quiz to assess student learning.

Template D1-1. Content Coverage for MPH (and DrPH degrees, if applicable) (SPH and PHP) Course number(s) & name(s) or Content other educational requirements 1. Explain public health history, philosophy and values PUBH 702 Public Health Theory and Practice 2. Identify the core functions of public health and the 10 PUBH 702 Public Health Theory and Essential Services Practice 3. Explain the role of quantitative and qualitative PUBH 702 Public Health Theory and methods and sciences in describing and assessing a Practice population’s health 4. List major causes and trends of morbidity and PUBH 702 Public Health Theory and mortality in the US or other community relevant to the Practice school or program 5. Discuss the science of primary, secondary and PUBH 702 Public Health Theory and tertiary prevention in population health, including health Practice promotion, screening, etc. 6. Explain the critical importance of evidence in PUBH 702 Public Health Theory and advancing public health knowledge Practice 7. Explain effects of environmental factors on a PUBH 702 Public Health Theory and population’s health Practice 8. Explain biological and genetic factors that affect a PUBH 702 Public Health Theory and population’s health Practice 9. Explain behavioral and psychological factors that PUBH 702 Public Health Theory and affect a population’s health Practice 10. Explain the social, political and economic PUBH 702 Public Health Theory and determinants of health and how they contribute to Practice population health and health inequities 11. Explain how globalization affects global burdens of PUBH 702 Public Health Theory and disease Practice

12. Explain an ecological perspective on the PUBH 702 Public Health Theory and connections among human health, animal health and Practice ecosystem health (e.g., One Health)

2) Document the methods described above. This documentation must include all referenced syllabi, samples of tests or other assessments and web links or handbook excerpts that describe admissions prerequisites, as applicable.

The syllabus and course schedule PUBH 702 in the ERF (D1-1 PUBH 702 Syllabus Fall 2020 and D1-2 PUBH 702 Course Schedule Fall 2020). Assessment of student learning for the D1 objectives is conducted using quizzes and discussion posts. The quizzes (see D1-3), discussion prompts, and discussion grading rubric (see D1-4) can also be found in the electronic resource file.

3) If applicable, assessment of strengths and weaknesses related to this criterion and plans for improvement in this area.

A strength related to this criterion is that all the foundational competencies are included in one core course. This approach ensures that all students entering the MPH program have the foundational knowledge necessary for success in later courses.

Section D

D2. MPH Foundational Competencies

Intentionally blank

Divider back

100

D2. MPH Foundational Competencies

The program documents at least one specific, required assessment activity (e.g., component of existing course, paper, presentation, test) for each competency, during which faculty or other qualified individuals (e.g., preceptors) validate the student’s ability to perform the competency.

Assessment opportunities may occur in foundational courses that are common to all students, in courses that are required for a concentration or in other educational requirements outside of designated coursework, but the program must assess all MPH students, at least once, on each competency. Assessment may occur in simulations, group projects, presentations, written products, etc. This requirement also applies to students completing an MPH in combination with another degree (e.g., joint, dual, concurrent degrees). For combined degree students, assessment may take place in either degree program.

1) List the coursework and other learning experiences required for the program’s MPH degrees, including the required curriculum for each concentration and combined degree option. Information may be provided in the format of Template D2-1 or in hyperlinks to student handbooks or webpages, but the documentation must present a clear depiction of the requirements for each MPH degree.

The MPH program is currently in the process of revising the curriculum for the generalist concentration. The curriculum can be found in template D2-1 below. The modifications in this curriculum include the addition of PUBH 760 Public Health and Native American Communities to the core curriculum and separation of PUBH 730 Integrative Learning Experience into two 3-credit courses. This curriculum revision has been approved by the MPH program faculty and the universities; it was approved by the SDBOR for implementation in fall 2020.

The addition of Public Health and Native American Communities (PUBH 760) to the core course list is important to achieving educational competencies related to diversity and cultural competence in public health. This addition will also facilitate achievement of the mission of the MPH program which is to advance health for rural and underserved communities. In the previous curriculum (see ERF D2-1 Previous MPH Curricula), the integrative learning experience was a 6-credit course completed in one semester. Students and faculty indicated that completion of the requirements of the integrative learning experience in one semester is difficult. This curriculum revision splits the 6-credit course into two 3-credit courses. The first course, PUBH 729 Leadership and Project Management in Public Health, will cover fundamental concepts in leadership and project management needed by public health professionals; students will also draft a proposal for their integrative learning experience project in this course. Then, students will complete the project in PUBH 730 Integrative Learning Experience.

Template D2-1. Requirements for MPH degree Course Credits number Course name (if applicable) PUBH 701 Biostatistics for Public Health 3 PUBH 702 Public Health Theory and Practice 3 PUBH 710 Epidemiology 3 PUBH 733 Environmental Health 3 PUBH 740 Intro to US Health Systems & Policy 3 PUBH 750 Social & Behavioral Sciences in Public Health 3 PUBH 755 Program Planning and Evaluation 3 PUBH 760 Public Health and Native American Communities 3 PUBH 721 Applied Practice Experience I 1

101

PUBH 722 Applied Practice Experience II 1 PUBH 723 Applied Practice Experience III 1 PUBH 729 Leadership and Project Management in Public Health 3 PUBH 730 Integrative Learning Experience 3 Elective Options 9 Total 42

The two previous MPH curricula can be found in the ERF (D2-1 Previous MPH Curricula). A table showing the number of students currently in the program and number of graduates from each curriculum can be found below.

Number of current students* Number of graduates Fall 2020 curriculum 39 0 Fall 2018 curriculum 40 11 Fall 2014 curriculum 18 64 Note. *Includes students on leave of absence

The coursework required for completion of the MPH program is listed in the graduate academic catalogs for USD and SDSU:

USD http://catalog.usd.edu/preview_program.php?catoid=30&poid=5818&returnto=1727 SDSU https://catalog.sdstate.edu/preview_program.php?catoid=39&poid=9106&returnto=6226

Available elective courses for the MPH degree are also listed in the academic catalogs.

2) Provide a matrix, in the format of Template D2-2 that indicates the assessment activity for each of the foundational competencies. If the program addresses all of the listed foundational competencies in a single, common core curriculum, the program need only present a single matrix. If combined degree students do not complete the same core curriculum as students in the standalone MPH program, the program must present a separate matrix for each combined degree. If the program relies on concentration-specific courses to assess some of the foundational competencies listed above, the program must present a separate matrix for each concentration.

Template D2-2. Assessment of Competencies for MPH (all concentrations) Competency Course number(s) Describe specific and name(s) assessment opportunity Evidence-based Approaches to Public Health 1. Apply epidemiological methods to PUBH 710 - Case Study Assignment: Students the breadth of settings and situations Epidemiology develop a plan for an epidemiological in public health practice study including study design and identification of the target population. 2. Select quantitative and qualitative PUBH 710 – Homework Assignment: Students had data collection methods appropriate Epidemiology to identify data collection methods as for a given public health context well as possible concerns and biases in the way the data was collected.

PUBH 755 – Program Qualitative Evaluation Assignment: Planning and Students identify a specific public health Evaluation program and collect and analyze qualitative data to evaluate the program.

102

3. Analyze quantitative and PUBH 701 – Homework assignment: Students are qualitative data using biostatistics, Biostatistics for Public asked to evaluate several aspects of a informatics, computer-based Health dataset using statistical software and programming and software, as make decisions about how best to appropriate analyze the data and draw statistical conclusions from the analysis.

PUBH 755 – Program Qualitative Evaluation Assignment: Planning and Students identify a specific public health Evaluation program and collect and analyze qualitative data to evaluate the program. 4. Interpret results of data analysis PUBH 701 – Homework assignment: The homework for public health research, policy or Biostatistics for Public asks the students to consider different practice Health types of analysis and to assess what the result meant in terms of recommendations. Public Health & Health Care Systems 5. Compare the organization, PUBH 740 – Health System Comparison Project: structure and function of health care, Introduction to US Students complete a presentation and public health and regulatory systems Health Systems and paper describing another nation’s health across national and international Policy care system and comparing its settings strengths and weaknesses to other nations. 6. Discuss the means by which PUBH 750 – Social Case Study Analysis: Students analyze structural bias, social inequities and and Behavioral a case study on the syndemic of opioid racism undermine health and create Sciences in Public overdose, HIV, and Hepatitis C in rural challenges to achieving health equity Health and medically underserved at organizational, community and communities in South Dakota. societal levels Planning & Management to Promote Health 7. Assess population needs, assets PUBH 755 – Program Community Assessment and Program and capacities that affect Planning and Planning Assignment: Students select a communities’ health Evaluation community and evaluate the health needs, assets, and capacities of that community. 8. Apply awareness of cultural PUBH 755 – Program Community Assessment and Program values and practices to the design or Planning and Planning Assignment: Students design implementation of public health Evaluation a public health intervention and policies or programs specifically include a discussion of cultural considerations and implications. 9. Design a population-based policy, PUBH 755 – Program Community Assessment and Program program, project or intervention Planning and Planning Assignment: Students use a Evaluation planning model to design a population- based treatment, policy, program, project, or intervention to address a real or potential health problem. 10. Explain basic principles and tools PUBH 729 Leadership Project Charter Paper: Students will of budget and resource management & Project develop a project charter for a public Management health organization with budget and resource management as a primary requirement. Students will develop a 5- year budget and project return on investment. This will be accompanied by readings and recorded lecture.

103

11. Select methods to evaluate PUBH 755 – Program Program Evaluation Proposal: Students public health programs Planning and select an evaluation framework and Evaluation draft a written proposal to evaluate a public health program. Policy in Public Health 12. Discuss multiple dimensions of PUBH 740 – Policy Analysis Project: In Domain 3 of the policy-making process, including Introduction to US this project, students utilize knowledge the roles of ethics and evidence Health Systems and of the policy-making process to develop Policy a strategy for furthering adoption of a policy solution. 13. Propose strategies to identify PUBH 733 – Environmental Health Impact stakeholders and build coalitions and Environmental Health Assessment Paper: Students identify partnerships for influencing public stakeholders and draft strategies to health outcomes build coalitions and partnerships needed for completion of an environmental health impact assessment. 14. Advocate for political, social or PUBH 740 – Media Advocacy Project: Students write economic policies and programs that Introduction to US an op-ed on a policy or public health will improve health in diverse Health Systems and issue to promote policy change. populations Policy 15. Evaluate policies for their impact PUBH 740 – Policy Analysis Project: Students apply on public health and health equity Introduction to US the CDC health policy analytical Health Systems and framework to analyze four current Policy federal, state, or tribal bills that aim to improve health outcomes and/or health equity. Leadership 16. Apply principles of leadership, PUBH 729 – Case Study: Review Healthcare.gov governance and management, which Leadership and launch case study and write a paper include creating a vision, Project Management identifying effective leadership and empowering others, fostering in Public Health management approaches that could collaboration and guiding decision have prevented the failed launch. making 17. Apply negotiation and mediation PUBH 729 – Negotiation and Mediation Paper: skills to address organizational or Leadership and Students will write a paper to assess a community challenges Project Management controversial public health intervention in Public Health from the pro and con perspective. Once complete students will be assigned one side to debate in a recorded video with a partner. Communication 18. Select communication strategies PUBH 702 – Public Health Communication Plan: Students for different audiences and sectors Health Theory and develop a communication plan to Practice address a public health issue identified through a community needs assessment. 19. Communicate audience- PUBH 750 – Social Health Promotion Project: Students appropriate public health content, and Behavioral develop a written proposal and oral both in writing and through oral Sciences in Public presentation appropriate for an presentation Health audience of community leaders and stakeholders.

104

20. Describe the importance of PUBH 760 – Public Reflective Writing Assignment #2: cultural competence in Health and Native Students reflect on the importance of communicating public health content American practicing cultural humility and Communities competence in communicating public health content with tribal communities. Interprofessional Practice 21. Perform effectively on PUBH 723 – Applied Interprofessional Team Activity – MPH interprofessional teams Practice Experience III students participate in interprofessional case discussions with students from several health professions including medicine, nursing, pharmacy, physical therapy, and occupational therapy. Students complete pre-reading and lecture on IPE competencies. Following completion of the IPE activity, students complete a reflection paper on the experience and lessons learned. Systems Thinking 22. Apply systems thinking tools to a PUBH 710 – Systems Thinking Assignment – public health issue Epidemiology Students read an article about using data for policy decisions and write about how the data were used, possible limitations to the use of the research for policy, factors that could account for relationships, and the potential for unintended consequences of policy on health.

3) Include the most recent syllabus from each course listed in Template D2-1, or written guidelines, such as a handbook, for any required elements listed in Template D2-1 that do not have a syllabus.

Course syllabi (ERF D2-2), schedules (ERF D2-3), and key assessment details (ERF D2-4) are provided in the ERF.

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The program continues to improve the curriculum and key assessments for the D2 competencies. The addition of two courses PUBH 760 (existing elective) and PUBH 729 (new course) strengthen the curriculum in key areas. Faculty have also recently revised key assessments to better assess student achievement of the competencies. For data collection, the data collection process for the Key Curricular Assessments was piloted with data from two courses in spring and summer 2019. Based on this experience, an assessment data collection form was developed and implemented for all courses starting in fall 2019. The 2019-2020 Key Curricular Assessment report can be found in the ERF (D2-5 Key Curricular Assessment Report 2019-2020). Meeting minutes that include discussion of the assessment plan and data can be found in ERF D2-6.

105

Intentionally blank

106

Section D

D3. DrPH Foundational Competencies Not applicable

107

Intentionally blank

Divider back

108

D3. DrPH Foundational Competencies

Not applicable.

109

Intentionally blank

110

Section D

D4. MPH & DrPH Concentration Competencies

111

Intentionally blank

Divider back

112

D4. MPH & DrPH Concentration Competencies

The program defines at least five distinct competencies for each concentration or generalist degree at each degree level in addition to those listed in Criterion D2 or D3.

The program documents at least one specific, required assessment activity (e.g., component of existing course, paper, presentation, test) for each defined competency, during which faculty or other qualified individuals (e.g., preceptors) validate the student’s ability to perform the competency.

If the program intends to prepare students for a specific credential (e.g., CHES/MCHES) that has defined competencies, the program documents coverage and assessment of those competencies throughout the curriculum.

1) Provide a matrix, in the format of Template D4-1, that lists at least five competencies in addition to those defined in Criterion D2 or D3 for each MPH or DrPH concentration or generalist degree, including combined degree options, and indicates at least one assessment activity for each of the listed competencies. Typically, the program will present a separate matrix for each concentration.

Template D4-1. Assessment of Competencies for MPH in Generalist Concentration Course number(s) or other Specific assignment(s) that Competency educational requirements allow assessment 1. Environmental Health: Utilize PUBH 733 – Environmental Environmental Health Impact risk management and risk Health Assessment Paper: Students communication approaches in conduct an environmental evaluation of environmental health impact assessment on a health issues. contemporary physical, chemical, or biological environmental health problem. 2. Project Management: Apply PUBH 729 - Leadership and Project Charter: Students project management principles to Project Management in create a project charter for a public health programs. Public Health non-profit organization addressing youth related public health issues in their community. Students utilize understanding of project management topics to address the scope, cost, and time which are the key constraints of any project. 3. Community Collaboration: PUBH 750 – Social and Health Promotion Project: Develop strategies using a Behavioral Sciences in Students apply principles of community-informed approach to Public Health community-based participatory address health equity in rural research to identify and adapt and/or medically underserved an evidence-based social and communities. behavioral intervention to address health equity and improve a specific public health issue in a rural and/or underserved community. 4. Advocacy: Propose PUBH 740 – Introduction to Policy Change Project: state/tribal/local policy change to US Health Systems and Students write a paper and address an identified gap in rural Policy develop a presentation

113

and/or medically underserved outlining a policy change to communities. address a public health issue including the rationale for the proposed change and action steps needed to move this change forward. 5. Communication: Create a PUBH 760 - Public Health Digital Storytelling Project: compelling health equity story and Native American Students create a digital story using innovative multimedia Communities to inform policymakers about communication methods. public health issues affecting American Indian and Alaska Native communities.

2) For degrees that allow students to tailor competencies at an individual level in consultation with an advisor, the program must present evidence, including policies and sample documents, that demonstrate that each student and advisor create a matrix in the format of Template D4-1 for the plan of study. Include a description of policies in the self-study document and at least five sample matrices in the electronic resource file.

The program does not currently allow students to tailor the generalist concentration competencies.

3) Include the most recent syllabus for each course listed in Template D4-1, or written guidelines for any required elements listed in Template D4-1 that do not have a syllabus.

Course syllabi (D4-1), course schedules (D4-2), and key assessment descriptions (D4-3) are provided in the ERF.

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Through the curriculum mapping process, weaknesses were addressed through the revision of the concentration competencies. Further revision was made after input from the Community Advisory Board, which informed the creation of a concentration competency on Project Management. We will continue to evaluate student outcomes and make further changes in the future if needed.

114

Section D

D5. MPH Applied Practice Experiences

115

Intentionally blank

Divider back

116

D5. MPH Applied Practice Experiences

MPH students demonstrate competency attainment through applied practice experiences.

The applied practice experiences allow each student to demonstrate attainment of at least five competencies, of which at least three must be foundational competencies (as defined in Criterion D2). The competencies need not be identical from student to student, but the applied experiences must be structured to ensure that all students complete experiences addressing at least five competencies, as specified above. The applied experiences may also address additional foundational or concentration-specific competencies, if appropriate.

The program assesses each student’s competency attainment in practical and applied settings through a portfolio approach, which demonstrates and allows assessment of competency attainment. It must include at least two products. Examples include written assignments, projects, videos, multi-media presentations, spreadsheets, websites, posters, photos or other digital artifacts of learning. Materials may be produced and maintained (either by the program or by individual students) in any physical or electronic form chosen by the program.

1) Briefly describe how the program identifies competencies attained in applied practice experiences for each MPH student, including a description of any relevant policies.

The MPH program has identified five competencies that must be covered in all Applied Practice Experiences (APE). Four competencies were selected from the MPH foundational competencies. These competencies were selected because they are higher level, application focused competencies that can be achieved across all public health practice sites. D2-4: Interpret results of data analysis for public health research, policy, or practice. D2-7: Assess population needs, assets and capacities that affect communities’ health. D2-13: Propose strategies to identify stakeholders and build coalitions and partnerships for influencing public health outcomes. D2-19: Communicate audience appropriate public health content, both in writing and through oral presentation.

The final competency is one of the program specific generalist competencies. This competency focuses on advancing health equity in rural and medically underserved communities, and it was selected because of its close connection to the mission of the MPH program.

D4-3: Develop strategies using a community-informed approach to address health equity in rural and/or medically underserved communities.

At the beginning of the APE, students submit an Applied Practice Experience Learning Plan Proposal (D5-1). The learning plan defines the goals, objectives, and deliverables of the APE and outlines the responsibilities of the student, the preceptor, and the MPH program faculty. The student needs to write specific learning objectives for the APE that are aligned to the APE competencies described above. The students provide regular progress reports throughout the course sequence to describe progress toward meeting the objectives and competencies outlined in the learning plan. During the final semester, the students submit the deliverables along with a final report for evaluation. The two deliverables should represent attainment and synthesis of the competencies by the student. The final report provides an overview of the project, a description of each deliverable, the student’s reflection on how the competencies were met, and a summary conclusion of the overall APE experience. The series concludes with a final presentation of the APE deliverables to faculty, the preceptor, and other stakeholders. The preceptor also completes a final evaluation of the student’s performance on the APE (see ERF D5-2 Final Preceptor Evaluation of Student Performance).

117

Prior to Fall 2019, students in the APE selected their own competencies. The practicum learning plan asked the students to select program competencies to be assessed during the APE. Students were asked to describe how the practice experience related to the mission of the MPH program and the student’s area of interest in public health. They also needed to describe how successful completion of the applied practice experience would demonstrate proficiency in the selected competencies.

2) Provide documentation, including syllabi and handbooks, of the official requirements through which students complete the applied practice experience.

Syllabi for the APE courses can be found in the ERF (D5-3 APE Syllabi). The APE handbook (see ERF D5-4) provides additional detail for students on policies and procedures for the practicum experience. Template affiliation agreements can be found in the ERF (D5-5 MPH Affiliation Agreement – USD; D5-6 MPH Affiliation Agreement – SDSU). Rubrics for evaluation of the learning plan (ERF D5-7 PUBH 721 Written Learning Plan Rubric) and the final report (ERF D5-8 PUBH 723 Rubric-final) can be found in the electronic resource file.

3) Provide samples of practice-related materials for individual students from each concentration or generalist degree. The samples must also include materials from students completing combined degree programs, if applicable. The program must provide samples of complete sets of materials (i.e., Template D5-1 and the work products/documents that demonstrate at least five competencies) from at least five students in the last three years for each concentration or generalist degree. If the program has not produced five students for which complete samples are available, note this and provide all available samples.

Samples of student learning in the applied practice experiences can be found in D5-9 of the ERF.

Template D5-1. Practice-based products that demonstrate MPH competency achievement: Generalist Concentration Specific assignment(s) that demonstrate application or Competency as defined in Criteria D2 and D4* practice The prevalence of Shiga Toxin- 4. Interpret results of data analysis for public health research, producing Escherichia coli at policy and practice state and county fairs in South 2. Select quantitative and qualitative data collection methods Dakota appropriate for a given public health context • Deliverables: sampling 7. Assess population needs, assets, and capacities that protocol, infographic affect communities’ health • Spring – Fall 2019 19. Communicate audience-appropriate public health content, both in writing and through oral presentation 21. Perform effectively on interprofessional teams Jurisdictional Vulnerability 4. Interpret results of data analysis for public health research, Assessment: Opioid Overdose, policy and practice HIV, and Hepatitis C (HCV) in 7. Assess population needs, assets, and capacities that South Dakota affect communities’ health • Deliverables: communication plan, executive summary and 18. Select communication strategies for different audiences county-level reports, and and sectors advertisement for the South 19. Communicate audience-appropriate public health Dakota Medicine Journal content, both in writing and through oral presentation • Summer 2019 21. Perform effectively on interprofessional teams 1. Apply epidemiological methods to the breadth of settings and situations in public health practice

118

Point of Sale for Tobacco 2. Select quantitative and qualitative data collection methods Retailers, ArcGIS StoryMaps appropriate for a given public health context Project 3. Analyze quantitative and qualitative data using • Deliverables: ArcGIS Story biostatistics, informatics, computer-based programming and Map, Story Map training guide software, as appropriate • Spring – Fall 2019 4. Interpret results of data analysis for public health research, policy or practice 19. Communicate audience-appropriate public health content, both in writing and through oral presentation Health Status of American 1. Apply epidemiological methods to the breadth of settings Indians/Alaska Natives in South and situations in public health practice Dakota: 2017 BRFSS 3. Analyze quantitative and qualitative data using • Deliverables: Annotated biostatistics, informatics, computer-based programming and Bibliography, Table Shells software, as appropriate and Data Analysis Plans 6. Discuss the means by which structural bias, social • Spring – Fall 2019 inequities and racism undermine health and create challenges to achieving health equity at organizational, community and societal levels 19. Communicate audience-appropriate public health content, both in writing and through oral presentation 20. Describe the importance of cultural competence in communicating public health content Prenatal Oral Health Program 5. Compare the organization, structure and function of health Needs Assessment for South care, public health and regulatory systems across national Dakota and international settings • Deliverables: survey of South 7. Assess population needs, assets and capacities that affect Dakota dentists, communities’ health stakeholder/advisory board 8. Apply awareness of cultural values and practices to the list, resource list design or implementation of public health policies or • Spring – Fall 2019 programs 13. Propose strategies to identify stakeholders and build coalitions and partnerships for influencing public health outcomes 18. Select communication strategies for different audiences and sectors

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

A weakness of this criterion relates to the change in our curriculum to meet new CEPH competencies and subsequent overlap with the original PUBH 720: Public Health Practice course. The first offering of the APE (PUBH 721-723) was spring 2019. Students admitted prior to fall 2018 follow the original curriculum in which they enroll in PUBH 720: Public Health Practice and complete a 120-hour practicum in one semester. PUBH 720 competencies were updated during the 2019 academic year to align with 2016 CEPH competencies. All students admitted during or after fall 2018 enroll in the APE 721-723 series.

In addition, our program selected five competencies that we require all MPH students to follow during the APE. This change was implemented during the fall 2019 semester with students enrolled in PUBH 720: Public Health Practice and took effect in spring 2020 with students enrolled in PUBH 721: Applied Practice Experience I.

119

Intentionally blank

120

Section D

D6. DrPH Applied Practice Experience Not applicable

121

Intentionally blank

Divider back

122

D6. DrPH Applied Practice Experience

Not applicable.

123

Intentionally blank

124

Section D

D7. MPH Integrative Learning Experience

125

Intentionally blank

Divider back

126

D7. MPH Integrative Learning Experience

MPH students complete an integrative learning experience (ILE) that demonstrates synthesis of foundational and concentration competencies. Students in consultation with faculty select foundational and concentration-specific competencies appropriate to the student’s educational and professional goals.

Professional certification exams (e.g., CPH, CHES/MCHES, REHS, RHIA) may serve as an element of the ILE but are not in and of themselves sufficient to satisfy this criterion.

The program identifies assessment methods that ensure that at least one faculty member reviews each student’s performance in the ILE and ensures that the experience addresses the selected foundational and concentration-specific competencies. Faculty assessment may be supplemented with assessments from other qualified individuals (e.g., preceptors).

1) List, in the format of Template D7-1, the integrative learning experience for each MPH concentration, generalist degree or combined degree option that includes the MPH. The template also requires the program to explain, for each experience, how it ensures that the experience demonstrates synthesis of competencies.

Template D7-1. MPH Integrative Learning Experience for Generalist Concentration Integrative learning experience How competencies are synthesized (list all options) Students completing a research project will develop a written report (manuscript) suitable for publication in a peer-reviewed journal. Students will demonstrate synthesis of competencies through application to a public health research project, specifically identification of the problem to be studied, review of Research Project the literature, collection and analysis of the data in order to answer the research question posed, discussion of the findings, and presentation of conclusions and recommendations based on the findings. Synthesis of the competencies is specifically evaluated on the rubric for the oral presentation. Students will demonstrate synthesis of the competencies through the student’s ability to critically review literature and synthesize published findings on a public health topic. Students conducting a systematic literature review will define a study question, Systematic Review identify appropriate literature using PRISMA Guidelines, read and review the literature, summarize the findings, present conclusions, and make recommendations based on the systematic review. Synthesis of the competencies is specifically evaluated on the rubric for the oral presentation. Students will demonstrate synthesis of the competencies through development of a plan to evaluate or monitor a new or existing public health program. Documentation will include a description of the program under review, the purpose of the evaluation, the Program Evaluation methods and procedures used to evaluate the program, and present the results of the evaluation. Synthesis of the competencies is specifically evaluated on the rubric for the oral presentation. Students writing a grant proposal for a public health initiative will Grant Proposal identify the problem for which the grant would be used, explain the significance, list the hypotheses/questions to be answered,

127

review the literature associated with the problem, create goals and objectives, methods, create a timeline and budget, and explain how the intervention would be evaluated. Students demonstrate synthesis of the competencies through application of public health knowledge in all steps of the grant proposal development process. Synthesis of the competencies is specifically evaluated on the rubric for the oral presentation. Students demonstrate synthesis of the competencies through development of a plan and description of the implementation process for a new public health program. Students will conduct a needs assessment, create a program mission, goals, and Written Plan for a Program objectives, develop an intervention, describe how the program will be implemented, and explain how the program would be evaluated. Synthesis of the competencies is specifically evaluated on the rubric for the oral presentation. Students demonstrate synthesis of the competencies through creation of a manual for a public health initiative. Students will demonstrate the need for the manual, create goals and objectives for the manual, create a table of contents, develop the Manual content for the manual, describe how the manual will be used, and explain how the manual would be evaluated. Synthesis of the competencies is specifically evaluated on the rubric for the oral presentation. Students demonstrate synthesis of the competencies through creation of a policy analysis utilizing a framework to examine a policy issue and link to health outcomes. Students will identify Policy Analysis key stakeholders and outline a potential plan to advocate for policy change. Synthesis of the competencies is specifically evaluated on the rubric for the oral presentation.

2) Briefly summarize the process, expectations and assessment for each integrative learning experience.

Process: In the semester prior to the Integrative Learning Experience (ILE), students are expected to identify their capstone project advisor (and faculty advisor, applicable if their capstone advisor is not an MPH faculty member) and establish the project plan. A project proposal form is used to guide the student’s development of the project (see ERF D7-1 Syllabus and Proposal Form). The project plan is carried out during the ILE (PUBH 730). Students are expected to meet biweekly with their project advisor to ensure the project is proceeding as planned. Several drafts of the written product capstone paper are submitted over the course of the semester for review.

Expectations: Students are expected to synthesize, integrate and apply the skills and competencies they have acquired through the MPH program to address a public health problem. Students will also demonstrate proficiency in communicating the details of their project to faculty, fellow students, and relevant stakeholders. The five competencies for the integrative learning experience are the same as those for the applied practice experience. Both written and oral products are produced at the end of the capstone project. Students are required to complete a high-quality capstone paper describing their project. Students are also required to give a 15-minute oral presentation summarizing their public health capstone project to a faculty committee comprised of their project mentor and MPH faculty. The synthesis of the competencies is specifically evaluated during the oral presentation.

Assessment: Rubrics are utilized for evaluation of the capstone paper and oral presentation. The projects are evaluated by the project advisor and at least one other MPH faculty member (see ERF D7-2 Rubrics & ILE Paper Format).

128

3) Provide documentation, including syllabi and/or handbooks that communicates integrative learning experience policies and procedures to students.

The syllabus for PUBH 730 can be found in the ERF (D7-1 730_ILE_syllabus_Fall2020). This folder in the ERF also contains the syllabus for PUBH 729 (PUBH 729 Syllabus) as students develop their ILE proposal during this course.

4) Provide documentation, including rubrics or guidelines that explains the methods through which faculty and/or other qualified individuals assess the integrative learning experience with regard to students’ demonstration of the selected competencies.

The rubrics used to evaluate the final project in the ILE are located in section D7-2 Rubrics & ILE Paper Format of the ERF.

5) Include completed, graded samples of deliverables associated with each integrative learning experience option from different concentrations, if applicable. The program must provide at least 10% of the number produced in the last three years or five examples, whichever is greater.

Graded examples of student projects from the ILE can be found in D7-3 Student Examples of the ERF.

6) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Given students’ feedback on wanting to know more information about the APE and ILE requirements earlier in their program matriculation, we created an orientation/resource center course shell in D2L to house information that is pertinent for the students’ entire program. Academic advisors are engaging in early conversations about these requirements and experiences as to put more onus on students to seek out placement sites. We have also revised the rubrics and continue to clarify ILE project expectations.

129

Intentionally blank

130

Section D

D8. DrPH Integrative Learning Experience D9. Public Health Bachelor’s Degree General Curriculum D10. Public Health Bachelor’s Degree Foundational Domains D11. Public Health Bachelor’s Degree Foundational Competencies D12. Public Health Bachelor’s Degree Cumulative and Experiential Activities D13. Public Health Bachelor’s Degree Cross-Cutting Concepts and Experiences

Not applicable

131

Intentionally blank

Divider back

132

D8. DrPH Integrative Learning Experience

Not applicable.

D9. Public Health Bachelor’s Degree General Curriculum

Not applicable.

D10. Public Health Bachelor’s Degree Foundational Domains

Not applicable.

D11. Public Health Bachelor’s Degree Foundational Competencies

Not applicable.

D12. Public Health Bachelor’s Degree Cumulative and Experiential Activities

Not applicable.

D13. Public Health Bachelor’s Degree Cross-Cutting Concepts and Experiences

Not applicable.

133

Intentionally blank

134

Section D

D14. MPH Program Length

135

Intentionally blank

Divider back

136

D14. MPH Program Length

An MPH degree requires at least 42 semester-credits, 56 quarter-credits or the equivalent for completion.

Programs use university definitions for credit hours.

1) Provide information about the minimum credit-hour requirements for all MPH degree options. If the university uses a unit of academic credit or an academic term different from the standard semester or quarter, explain the difference and present an equivalency in table or narrative form.

The MPH program requires 42 credit hours for completion. This includes 3 credits of applied learning experience, 3 credits of integrative learning experience, and 9 credits of electives.

2) Define a credit with regard to classroom/contact hours.

A credit hour for courses taught in a traditional classroom setting is defined as at least 15 contact hours as well as a minimum of 30 hours of student homework for each credit hour, giving a total of 45 hours per credit. Credit hours for courses delivered online are considered equivalent to those of students in a traditional classroom setting. Therefore, an equivalent of 45 hours of work by a typical student is required for each credit hour.

137

Intentionally blank

138

Section D

D15. DrPH Program Length D16. Bachelor’s Degree Program Length D17. Academic Public Health Master’s Degrees D18. Academic Public Health Doctoral Degrees D19. All Remaining Degrees

Not applicable

139

Intentionally blank

Divider back

140

D15. DrPH Program Length

Not applicable.

D16. Bachelor’s Degree Program Length

Not applicable.

D17. Academic Public Health Master’s Degrees

Not applicable.

D18. Academic Public Health Doctoral Degrees

Not applicable.

D19. All Remaining Degrees

Not applicable.

141

Intentionally blank

142

Section D

D20. Distance Education

143

Intentionally blank

Divider back

144

D20. Distance Education

The university provides needed support for the program, including administrative, communication, information technology and student services.

There is an ongoing effort to evaluate the academic effectiveness of the format, to assess learning methods and to systematically use this information to stimulate program improvements. Evaluation of student outcomes and of the learning model are especially important in institutions that offer distance learning but do not offer a comparable in-residence program.

1) Identify all public health distance education degree programs and/or concentrations that offer a curriculum or course of study that can be obtained via distance education. Template Intro-1 may be referenced for this purpose.

The program offers a generalist MPH degree via distance education (Template Intro-1). Template Intro-1. Instructional Matrix - Degrees and Concentrations Campus Distance Master's Degrees Academic Professional Based Executive Based Concentration Degree Degree Generalist MPH X

2) Describe the public health distance education programs, including

a) an explanation of the model or methods used,

The learning management system Desire2Learn (D2L) is utilized for delivery of the online MPH program, and all courses are offered asynchronously. Each course has student learning outcomes that are mapped to the competencies in D1, D2, and/or D4, and the student learning outcomes are provided to the students in the syllabus and content sections of the learning management system. A variety of instructional and assessment methods are utilized in the program and have been selected for consistency with the student learning outcomes. Examples of instructional methods used include textbook readings, journal articles, recorded lectures, online videos, and websites. Student learning is assessed using quizzes, examinations, discussion forum posts, projects, and papers. The instructional and assessment methods utilized foster self-directed learning, stimulate critical thinking ability, and are appropriate for the learning needs of online students.

Facilitation of interaction among students and between students and the instructor is a critical component of online courses. Courses in the MPH program achieve regular and substantive interaction through online discussion boards. Courses begin with an introductory discussion post in order to get to know their classmates and instructor(s). Regular discussions are utilized throughout the semester as well. For example, Public Health Theory and Practice (PUBH 702) includes weekly discussion topics to facilitate interaction among students and help students engage more deeply with the course content. Students are divided into small groups, and one group member serves as the discussion leader each week. Collaborative projects are another important method of facilitating student interaction. In Social and Behavioral Sciences in Public Health (PUBH 750), students work in teams to develop a recorded lecture and learning activity on a topic related to course content. This allows students to interact with their classmates and develop teamwork skills.

b) the program’s rationale for offering these programs,

145

The mission of the program states that, “The University of South Dakota and South Dakota State University Master of Public Health program is committed to improving health in rural and underserved communities by providing interdisciplinary learning, scholarship, and community engagement for public health professionals.” This commitment to promoting and protecting the health and well-being of rural and underserved communities provides the rationale for offering this online program. A 2009 survey from the South Dakota Department of Health and the South Dakota Public Health Association indicated a need to develop an in-state MPH program to address public health needs within the state of South Dakota. The online MPH program was developed to address those needs. Online delivery was selected to address the learning needs of working professionals and students in rural areas. Online delivery also allows the two universities to draw on their strengths in addressing the public health needs of rural and underserved communities in this collaborative program. c) the manner in which it provides necessary administrative, information technology and student support services,

Administrative support for the program is provided by the MPH program director and the MPH program coordinator. Each MPH student is assigned an academic advisor who supports student course selection and registration. Graduate student orientation is held at the beginning of the academic year to orient students to overall graduate school policies; online students are able to join orientation via videoconferencing technology. A D2L site for the MPH Orientation and Resource Center is promoted and available to students first in the MPH Orientation webinar and used throughout the program through advising to provide ongoing student support.

Technical support services are available to all students taking online courses via phone and e- mail. Contact information is provided in the MPH student handbook (see ERF A1-1 MPH Student Handbook 2020-21), course syllabi, and within D2L. Instructional Design Services at SDSU provides technology support and course design consultation for faculty. They also host summer academies to help faculty utilize educational technology in online and face-to-face courses. Similar technology support is provided to faculty and students at USD; as such, the mission of the USD Information Technology Services office is to manage computing resources and to provide support to the faculty, students, and staff. In addition, USD’s Center for Teaching and Learning provides instructional design workshops, technical assistance, and summer fellowships for course redesign.

Students portals (MyState Online - SDSU; MyUSD - USD) include links to student success platforms and support services. The MPH student handbook (see ERF A1-1 MPH Student Handbook 2020-21) contains academic and student services information for both universities. Descriptions and links for disability services, career development offices, financial aid, and registration offices are included. The syllabus template for the MPH program courses also includes a description of student support services at both USD and SDSU. Students have online access to the library at their university as well as the Wegner Health Science Information Center. Access information is provided in course syllabi. Each university has a writing center to assist students in developing written communication skills. d) the manner in which it monitors the academic rigor of the programs and their equivalence (or comparability) to other degree programs offered by the university, and

The South Dakota Board of Regents has established a rigorous quality review process for all online courses offered by regental universities. The goal of the review is to ensure that online courses are of high quality, provide a rigorous and engaging learning experience, and incorporate best practices in online education. This review is required for all new online courses, established online courses with new instructors, or if three years have passed since the last review. This review must take place prior to the start date of the course to ensure course quality. First time online instructors are also required to complete training on use of the

146

learning management system and best practices of online teaching. More advanced training, including online instructor certification, is available.

A system level rubric has been developed to guide the review process. Copies of the standards, policies, and rubrics across both campuses are available in the ERF (D20-1 SDBOR Policy on Quality Assurance Rubric; D20-2 SDBOR Policy on Quality Assurance Guidelines; D20-3 Quality Assurance Review Rubric; D20-4 SDSU Policy Quality-Assurance-Policy; D20-5 USD Course Readiness Self-Check Form). The general review standards include course overview and introduction; learning outcomes; assessment and measurement; resources and materials; learner interaction; course technology; learner support; and accessibility. Each general review standard has more specific standards under it. These specific standards are ranked in importance. Essential Standards are considered critical for all online courses and missing even one of these standards means that the rubric is not satisfied. Very important standards should be met in most courses but are not considered necessary in every online course. Finally, important standards may not be needed or possible in every online course and therefore are ranked lower in point assignment.

Integrity of exams is assured through use of Respondus Lockdown Browser or through the proctor system. Turnitin is available for students and faculty to use to verify integrity of written assignments. The Curriculum and Assessment Committee provides oversight of the quality and scope of the MPH curriculum. The committee does an annual review of syllabi and changes to the key assignments are approved by committee. Through the key curricular assessment report, students’ educational outcomes on competencies are monitored. e) the manner in which it evaluates the educational outcomes, as well as the format and methods.

Program faculty utilize a variety of course assessments to evaluate student learning including exams, quizzes, homework assignments, reflective writings, papers, and projects. The learning management system includes several tools used in the evaluation of student learning such as an exam/quiz tool, an assignment dropbox, and discussion boards. Achievement of the educational outcomes is monitored at the program level through collection of data regarding student performance on the key curricular assessments. The details of these assessments are outlined in Tables D2-2 and D4-1. Assessment data is collected at the end of each semester and assembled into the Key Assessment Report. This report is shared with the Curriculum and Assessment Committee, the program faculty, and the institutional assessment offices.

The course review process described above provides important evaluation of the format and methods used for course delivery. Professional instructional designers review the courses to ensure that best practices in online course delivery are followed. In addition, alumni and current student survey data is used to evaluate the online format for course delivery. In the survey of current MPH students, respondents indicated satisfaction with the program’s technology and online format. The response rate for the Student Survey in 2019 was 45%. The response rate for the Student Survey in 2020 was 52%.

147

Table D20-1. Student Satisfaction with Online Format 2019 2019 2019 2020 2020 2020 Statistically Overall Overall Mean Overall Overall Mean Significant Question # % Overall # % Overall (p<0.05) How satisfied were you with the ease of collaboration through different technology platforms? 1=Very Dissatisfied 0 0% 0 0% 2=Dissatisfied 1 3% 2 7% 3.40 3.40 No 3=Satisfied 16 53% 14 47% 4=Very Satisfied 13 43% 14 47%

Table D20-2. Level of Agreement with Online Format 2019 2019 2019 2020 2020 2020 Statistically Overall Overall Mean Overall Overall Mean Significant Question # % Overall # % Overall (p<0.05) The online format met my needs. 1=Strongly Disagree 0 0% 0 0% 2=Disagree 2 6% 0 0% 3.48 3.67 No 3=Agree 12 39% 10 33% 4=Strongly Agree 17 55% 20 67%

Similarly, alumni of the MPH program were satisfied with the online learning experience. Also, when asked about preference in delivery method, 58% of survey respondents indicated that their preference was completely online.

Table D20-3. Student Satisfaction with Online Learning Experience Information & Technology Resources Overall # Overall % Mean Overall Satisfaction with online learning experience 1=Very Dissatisfied 0 0% 2=Dissatisfied 1 3% 3.55 3=Satisfied 12 39% 4=Very Satisfied 18 58%

3) Describe the processes that the university uses to verify that the student who registers in a distance education course (as part of a distance-based degree) or a fully distance-based degree is the same student who participates in and completes the course or degree and receives the academic credit.

D2L is integrated with the regental registration system, and only students registered for the course are given access to course materials and assessments. Students access D2L using a username and password unique to the student. Proctors and exam monitoring technology are utilized to verify the identity of students taking course examinations.

148

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Overall, students appear to be satisfied with the technology and learning environment. We will continue to monitor for improvement opportunities.

149

Intentionally blank

150

Section E

E1. Faculty Alignment with Degrees Offered

151

Intentionally blank

Divider back

152

E1. Faculty Alignment with Degrees Offered

Faculty teach and supervise students in areas of knowledge with which they are thoroughly familiar and qualified by the totality of their education and experience.

Faculty education and experience is appropriate for the degree level (bachelor’s, master’s, doctoral) and the nature of the degree (research, professional practice, etc.) with which they are associated.

1) Provide a table showing the program’s primary instructional faculty in the format of Template E1-1. The template presents data effective at the beginning of the academic year in which the final self-study is submitted to CEPH and must be updated at the beginning of the site visit if any changes have occurred since final self-study submission. The identification of instructional areas must correspond to the data presented in Template C2- 1.

Template E1-1. Primary Instructional Faculty Alignment with Degrees Offered Name* Title/ Tenure Status Graduate Institution(s) Discipline in Current Academic or Degrees from which which instructional Rank Classification^ Earned degree(s) degrees area(s)+ were earned were earned

Hunt, Assistant Tenure-track PhD, University of Public Health Public Health Aaron Professor; MPH Nevada, Las Theory & Program Vegas Practice; Coordinator University of APE, ILE Nevada, (SDSU) Reno Puumala, Senior Non-tenure PhD, MS University of Epidemiology, Biostatistics; Susan Lecturer Minnesota, Statistics Epidemiology; Iowa State APE, ILE University (USD) Strobel, Assistant Tenure-track DNP, University of Nursing, APE, ILE Susan Professor MPH Colorado, Public Health (USD) University of Minnesota

TBH – USD Assistant Professor

2) Provide summary data on the qualifications of any other faculty with significant involvement in the program’s public health instruction in the format of Template E1-2. Programs define “significant” in their own contexts but, at a minimum, include any individuals who regularly provide instruction or supervision for required courses and other experiences listed in the criterion on Curriculum. Reporting on individuals who supervise individual students’ practice experience (preceptors, etc.) is not required. The identification of instructional areas must correspond to the data presented in Template C2-1.

153

Template E1-2. Non-Primary Instructional Faculty Regularly Involved in Instruction

Institution(s) Discipline in Title and FTE or Graduate Current Academic from which which Name* Current % Time Degrees instructional Rank^ degree(s) degrees Employment Allocated Earned area(s)+ were earned were earned Huang, Assistant Assistant 36% PhD, MS, University of Social & ILE (SDSU) Yen-Ming Professor, Professor, BPharm Wisconsin- Administrative SDSU Department Madison, Sciences in of Allied and National Pharmacy Population Taiwan Health University Jepperson, Adjunct, Project 1 course MPH University of Biology, Social and Shelby USD Coordinator, New England Public Health Behavioral SHS Sciences

Kenyon, Professor / Program within PhD University of Family ILE (USD) DenYelle Program Director, admin time Arizona Studies and Director, SHS, Human USD Associate Development Dean for Diversity & Inclusion, SSOM Killian, Adjunct Senior 1 course PhD, University of Epidemiology Public Health Scott Professor, Scientist, MPH California- Immunology USD Immunothera San (USD) peutics Co. Francisco, University of California-Los Angeles Lucas- Assistant Assistant 1 course PhD, Intro to US Molitor, Professor, Professor, OTD, Health Whitney USD SHS, MS, Systems and Department OTR/L, Policy (USD) of BCG Occupational Therapy Melstad, Adjunct Owner/ 1 course MPH Tulane Community APE, ILE Sandra Professor, Director, University Health (USD) USD SLM School of Education and Consulting, Public Health Communication LLC and Tropical Medicine Miller, Assistant Assistant 36% PharmD, North Dakota Pharmacy, ILE (SDSU) Erin Professor, Professor, MBA, BS State Business, SDSU Department University Pharmaceutic of Allied and al Sciences Population Health Pinto, Hoch Department within BSPharm, K.M. Pharmacy ILE (SDSU) Sharrel Family Head, admin/ DMM, MS, Kundanani Health Care Endowed Department research PhD College of Administration, Professor of Allied and time Pharmacy, Marketing for Population University of Management, Community Health, Bombay, Pharmacy Pharmacy SDSU India; The Practice/ University of Department Toledo (UT),

154

Head, Ohio; SDSU University of Florida (UF), Florida Stenvig, Associate Associate 26% PhD, University of Nursing Program Thomas Professor, Professor, MPH, MS Wisconsin- Planning and SDSU College of Madison, Evaluation Nursing, University of (SDSU) South Hawaii, South Dakota State Dakota State University University Wesner, Instructor Research 75% MPH, University of Health ILE, Public Chelsea Instructor, MSW Oklahoma Promotion Health and SHS Sciences / Native Administrative American & Community Communities Practice (USD)

3) Include CVs for all individuals listed in the templates above.

Faculty CVs can be found in the ERF (E1).

4) If applicable, provide a narrative explanation that supplements reviewers’ understanding of data in the templates.

The MPH program is fortunate to have several primary instructional faculty and non-primary instructional faculty involved in instruction. The four primary instructional faculty have primary appointments in the MPH program (Strobel is 50/50 with Nursing). The non-primary instructional faculty roles range from adjuncts, to faculty from other departments who teach core MPH courses (Lucas-Molitor, Stenvig), to faculty in the department in which the MPH is housed under at SDSU who engage in teaching, mentorship, and administration, and the program director, who’s role is mainly administrative.

5) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The program has made many recent improvements in this area with its new faculty hires. We also have greatly broadened the pool of non-primary instructional faculty available as research advisors for students’ Integrative Learning Experiences. We look forward to continuing to build this through filling the open assistant professor position.

155

Intentionally blank

156

Section E

E2. Integration of Faculty with Practice Experience

157

Intentionally blank

Divider back

158

E2. Integration of Faculty with Practice Experience

To assure a broad public health perspective, the program employs faculty who have professional experience in settings outside of academia and have demonstrated competence in public health practice. Programs encourage faculty to maintain ongoing practice links with public health agencies, especially at state and local levels.

To assure the relevance of curricula and individual learning experiences to current and future practice needs and opportunities, programs regularly involve public health practitioners and other individuals involved in public health work through arrangements that may include adjunct and part- time faculty appointments, guest lectures, involvement in committee work, mentoring students, etc.

1) Describe the manner in which the public health faculty complement integrates perspectives from the field of practice, including information on appointment tracks for practitioners, if applicable. Faculty with significant practice experience outside of that which is typically associated with an academic career should also be identified.

Several MPH primary and non-primary instructional faculty have experience in public health practice and remain engaged in practice through contracts with federal, state, and local nonprofit organizations. We also have guest lecturers from local non-profits (e.g., discussing experience with national-level policy, discussing population health).

Table E2.1 Faculty with Practice Experience Faculty PIF/non-PIF Role Practice Experience

Aaron Hunt, PIF • 2020-2023: Program Director, Rural Community Opioid PhD, MPH Response Program (HRSA-20-031), South Dakota State University. • 2018-2019: Program Coordinator, Primary Care Training and Enhancement Program (HRSA-16-042). A.T. Still University, School of Osteopathic Medicine in Arizona, Mesa AZ • 2013-2017: Biological Field Technician, Biowatch program (Department of Homeland Security), University of Nevada, Las Vegas • 2012-2016: Research Coordinator, NIH/PEPFAR Collaboration for Advancing Implementation Science in Prevention of Maternal-Child HIV Transmission, University of Nevada School of Medicine/University of Nevada, Las Vegas • 2012-2015: Program Coordinator, Ryan White Title IV Women, Infants, Children, Youth and Affected Family Members AIDS Healthcare Program, University of Nevada School of Medicine • 2012-2013: Program Coordinator, Nevada Care Program, Maternal-Child HIV Program. HRSA/AAP Healthy Tomorrow Partnership, University of Nevada School of Medicine • 2010-2011Research Analyst: The Nevada Institute for Children’s Research and Policy (NICRP), University of Nevada Las Vegas

159

DenYelle Non-PIF, Previous • 2013-2017: Founding Co-Director, Evaluation Core Kenyon, role as non-profit Services, Population Health Group, Sanford Research PhD researcher • 2012-2017: Lead on creation of pregnancy prevention program for middle school Native American youth (Kenyon, D. B., McMahon, T. R., Schwab, A., Huff, M., Green-Maximo, C., Griese, E.R., & Sieving, R.E. (2017). My Journey: Teen Pregnancy Prevention curriculum for Northern Plains American Indian youth. Sioux Falls, SD. Publisher). Scott Killian, Non-PIF, Senior • 2007 – 2012: Director of Research and Development PhD Scientist at in vaccine development, Therabiol, Inc., San Immunotherapeutics Francisco, CA. Co. • 2018 – present: Sr. Scientist in biomarker discovery in immuno-oncology, Immunotherapeutics Co., San Francisco Bay Area, CA. Sandra Non-PIF, Public • 2020-present, Public Health Consultant, University of Melstad, Health Consultant, South Dakota MPH small business • 2020-present, Public Health Consultant, Sioux Falls owner SLM Immunization Coalition Consulting, PhD • 2018-2019: Program Coordinator for Jurisdictional Student Vulnerability Assessment, South Dakota Department of Health, CDC cooperative agreement Emergency Response: Public Health Crisis Response • 2018-present: Public Health Consultant, South Dakota Department of Health Office of Child and Family Services • 2017-present: Subject Matter Expert and Public Health Consultant, Rocky Mountain Public Health Training Center • 2015-present: Public Health Consultant, Cooperative Extension Office, South Dakota State University • 2012-present: Public Health Consultant, South Dakota Department of Health Office of Chronic Disease Prevention and Health Promotion Susan PIF, Previous role • 2019-present: Program Evaluator, South Dakota Puumala, as non-profit Department of Health, CDC cooperative agreement PhD, MPH researcher and Overdose Data to Action (OD2A) public health • 2003-2006: Statistical Coordinator, University of architecture firm Nebraska Medical Center • 2001-2003: Biostatistical Coordinator, Transdisciplinary Tobacco Use Research Center, University of Minnesota Tom Non-PIF, previous • 1984-1993: Commissioned Officer, Great Plains Area Stenvig, role as US public Indian Health Service, U.S. Public Health Service PhD, MPH health service (Directed community-based programs serving officer American Indian communities) • 2011-2015: Appointment on National Vaccine Advisory Committee (Federal advisory body on national immunization and vaccine policy) Susan PIF, Previous role • 2019-present: Program Evaluator, South Dakota Strobel, as clinical Department of Health, CDC cooperative agreement DNP, MPH researcher Overdose Data to Action (OD2A) • 2004-2008: Epidemiologist/Clinical Analyst, RCRI, Inc, Minnesota

160

• 1994-1996: Epidemiologist/Clinical Research Analyst, Health Risk Management, Inc, Minnesota • 1992-1994: Epidemiologist/Clinical Research Analyst, University of Minnesota Center for Biomedical Ethics Chelsea Non-PIF, previous • 2018-2019: Team lead for Jurisdictional Vulnerability Wesner, role as medical Assessment, South Dakota Department of Health, CDC MPH, MSW social worker, cooperative agreement Emergency Response: Public current role DrPH Health Crisis Response student and • 2017-present: Program Evaluator, South Dakota researcher Department of Health, CDC cooperative agreements Overdose Data to Action (OD2A) and Prescription Drug Overdose: Data Driven Prevention Initiative (DDPI) • 2013-present: Intergovernmental Personnel Agreement (IPA), Native Diabetes Wellness Program, Division of Diabetes Translation, CDC • 2011-2012: Medical Social Worker, Intermountain Healthcare • 2009-2015: Program Planner, American Indian Institute, University of Oklahoma Note. Bold font designates work done outside full-time employment as University faculty member.

Table E2.2 Guest Lecturers with Practice Experience Guest Role Course Topic Lecturer Jordyn PhD Student, PUBH 760 “Native “Public Health and Research” Gunville, Senior Research American Communities • Reservation life challenges and MPH, CTTS Associate, Center and Public Health strengths for American • Her educational journey Indian Community • Health literacy Health, Tribal • Current research methodology Public Health and instrument development Consultant Susan Kroger, PhD Student, PUBH 740 “Population Health Evaluation Projects” MS Evaluation “Introduction to US • Population Health and social Director, Sanford Health Systems and determinants of health Research Policy” • American Cancer Society project on colorectal cancer screenings • Equine Therapy, American Indian youth, Suicide Prevention/Mental health treatment • Good Samaritan Society, Senior Home visiting program Jenn Folliard, Family and PUBH 740 “Public Health Policy Analysis and the MPH, RDN Community Health “Introduction to US US Legislative and Regulatory Process” Field Specialist II, Health Systems and • Kingdon Policy Streams Model South Dakota Policy • Legislative process State University • Example of Veteran’s food insecurity policy change

Public health practice in partnership with communities, tribes, university outreach, and governmental agencies demonstrate the capacity and commitment of our faculty to include relevant public health examples in teaching, engage students in practice through the APE and ILE, and contribute to emerging public health issues. Furthermore, the diversity of our faculty’s public health

161

practice and research experience strengthens our ability to respond to community needs in the region and sustain relationships with practice settings for student APE and ILE.

2) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Our faculty have a wealth and breadth of public health experience. These faculty engage students in their ongoing public health projects, exposing students to real-world applications of public health. We look forward to adding a faculty member with policy and community outreach experience.

162

Section E

E3. Faculty Instructional Effectiveness

163

Intentionally blank

Divider back

164

E3. Faculty Instructional Effectiveness

The program ensures that systems, policies and procedures are in place to document that all faculty (full-time and part-time) are current in their areas of instructional responsibility and in pedagogical methods.

The program establishes and consistently applies procedures for evaluating faculty competence and performance in instruction.

The program supports professional development and advancement in instructional effectiveness.

1) Describe the means through which the program ensures that faculty are informed and maintain currency in their areas of instructional responsibility. The description must address both primary instructional and non-primary instructional faculty and should provide examples as relevant.

The program director (USD) and department head (SDSU) are ultimately responsible for ensuring faculty are current and performing well in their assigned courses. The curriculum and assessment committee conducts an annual review of syllabi to ensure standardization and quality of content. The Faculty Database is sent annually to primary and non-primary faculty for faculty to self-report their activities on the previous academic year. This includes activities such as publications, grant proposals, trainings, service, professional development, recruitment, and admissions follow-up. Additionally, for the primary instructional faculty, during the annual review process the individual faculty’s goals are assessed in the areas of their appointment (teaching, research, and service), depending on their appointment. This faculty annual review includes a question on continuing education which is an assessment of faculty training and how they maintain currency in the public health field. This can include formal in person trainings, webinars, videos, listservs, articles, etc. Faculty are also encouraged to share resources amongst the group informally via email.

Specifically, in the area of teaching, the IDEA survey is used to obtain student feedback on courses. In addition, SDSU faculty must submit at least two additional forms of teaching evaluations. Particularly for non-primary faculty, course IDEA scores will be closely monitored; feedback will also be obtained through informal discussions with students about their experience in various courses and advising. These faculty metrics are included in the Program Goals Assessment Report and reviewed by the Curriculum and Assessment committee and All faculty committees.

2) Describe the program’s procedures for evaluating faculty instructional effectiveness. Include a description of the processes used for student course evaluations and peer evaluations, if applicable.

The program assesses faculty instructional effectiveness through formal and informal methods. Each course has an optional mid-point evaluation for students to share their perspectives on how the instruction and course materials are received. Faculty are also encouraged to create their own formal polls/surveys assessing students’ perspectives of coursework throughout the semester. Faculty are also encouraged to create a climate of ongoing feedback/receptivity to questions through office hours and question boards. At the end of each course, faculty are formally assessed through standard university tools (IDEA evaluation), which contain quantitative and qualitative questions. These metrics are taken into account as a part of the holistic review during annual performance evaluations. Faculty instructional effectiveness is assessed by metrics both from the student and supervisor perspective under the academic excellence goal in the Program Goals Assessment. The curriculum and assessment committee include these metrics in their annual review.

3) Describe available university and programmatic support for continuous improvement in faculty’s instructional roles. Provide three to five examples of program involvement in or

165

use of these resources. The description must address both primary instructional faculty and non-primary instructional faculty.

The program places high emphasis on faculty teaching performance, as evidenced by the MPH program’s first goal of: Academic Excellence, Promote and achieve excellence in public health teaching and learning. Responsibility for oversight is through both program administrative faculty and the Curriculum and Assessment Committee, and related topics are often discussed at faculty and smaller group meetings. There are also similar supportive resources at both universities. At USD the Center for Teaching and Learning (CTL) has many resources for continuous improvement in instruction. For example, in onboarding new faculty to a course (both primary instructional and non-primary instructional), there are several in-depth training modules and hands-on sessions with CTL staff (e.g., digital accessibility training to ensure content is user friendly for all). There are also in-person events showcasing excellence in online teaching (e.g., 9/5/19 online faculty symposium attended by Dr. Puumala). Another example that ensures consistency and high quality of course material is the Course Review Sign off overseen by CTL, which is mandated by the SDBOR for any faculty (primary instructional and non-primary instructional) with new courses or performed every 3 years. Similarly, SDSU has the Center for the Enhancement of Teaching and Learning (CETL) which provides faculty development sessions to enhance student learning. Instructional Design Services (IDS) at SDSU provides faculty training in use of educational technologies and best practices in online education. All faculty that are new to teaching online courses are required to attend the basic training. Advanced levels of online teaching certification are also available through IDS.

4) Describe the role of evaluations of instructional effectiveness in decisions about faculty advancement.

Both universities have a promotion and tenure process guided by the Board of Regents/Council on Higher Education policies and procedures. That process (described in A-1) is informed by the annual review process described in E3.1 Annual Professional Staff Evaluation USD and E3-2 Faculty Annual Review Form SDSU. Instructional effectiveness would be assessed under the “teaching” aspect of the faculty’s position. The supervisor assesses faculty on a scale of 0=failed to meet expectations, 1=met expectations, 2=performed above expectations, and 3=performed substantially above expectations. There is also room for open-ended comments about teaching and mentoring. Faculty track records in teaching are very important to faculty advancement at both USD and SDSU.

5) Select at least three indicators, with one from each of the listed categories that are meaningful to the program and relate to instructional quality. Describe the program’s approach and progress over the last three years for each of the chosen indicators. In addition to at least three from the lists that follow, the program may add indicators that are significant to its own mission and context.

Faculty Currency: The Curriculum and Assessment Committee annual review of syllabi. Ahead of each meeting, a primary and secondary reviewer (not the instructors of record) will be assigned to review and present the assigned syllabi at the meeting, providing feedback on current relevance of topics, readings, and instructional methods. This was conducted in summer 2019 by the ad-hoc CEPH accreditation committee through a curriculum mapping exercise. Several areas of improvement were identified. For example, several aspects of PUBH 702 were updated, including adoption of a new book and inclusion of more interactive discussion assignments.

Faculty Instructional Technique: Student satisfaction with instructional quality is included in the Program Goals Assessment Report, including IDEA course evaluation data, faculty performance evaluation data, as well as alumni and student survey data. The report is reviewed annually.

166

Table E3.1. Alumni and Student Perceptions of Instructional Quality Respondents % (n) Mean Alumni Satisfaction with quality of instruction Survey 1=Very Dissatisfied 0% (0) 2=Dissatisfied 6% (2) 3.3 3=Satisfied 58% (18) 4=Very Satisfied 35% (11) 2019 Satisfaction with the overall quality of graduate level teaching by Student faculty Survey 1=Very Dissatisfied 7% (2) 2=Somewhat Dissatisfied 10% (3) 3.3 3=Somewhat Satisfied 33% (10) 4=Very Satisfied 50% (15) 2020 Satisfaction with the overall quality of graduate level teaching by Student faculty Survey 1=Very Dissatisfied 3% (1) 2=Somewhat Dissatisfied 3% (1) 3.6 3=Somewhat Satisfied 27% (8) 4=Very Satisfied 67% (20)

In response to the pandemic and all courses being held online, a platform for transparency and communication was created through zoom sessions held twice a week for faculty and students to provide feedback with the online instruction, which is a large part of the program’s instructional quality. In March 2020, SDSU held departmental meetings for students, faculty and staff to provide an opportunity for socializing, questions, etc. MPH students and MPH faculty participated in these sessions coined “Monday Meet Up”. These sessions were recorded and available for future viewing. Additionally, during this same timeframe except on Fridays, program specific meetings for MPH students and faculty were held to discuss online teaching, what is working and what isn’t working. These sessions were called “Friday Reflection”. Frequent listserv communication (weekly – twice monthly) was provided to the MPH students by the Program Director to offer support, detail resources, covid-19 articles and webinars, and employment opportunities.

School- or program-level outcomes: The program has chosen to focus on implementation of rubrics as a program-level outcome we are working to standardize across all courses. This area of need was identified during the summer 2019 syllabi review. Much progress has been made on this goal, updating the rubrics for the six project options in the PUBH 730 Integrated Learning Experience. Additionally, rubrics were developed for class discussion participation, and future plans include expanding that rubric to all courses to improve student engagement and interactions. Recently, a need for a rubric for the PUBH 723 Applied Practice Experience final project arose, and the primary instructional faculty collaborated on drafting of the rubric. Another recent example arose during the review of the Annual Assessment report was the need for the Community Assessment and Program Planning Assignment to clarify instructions and separate out different aspects of the grading rubrics for the distinct parts of the various competencies.

Ways to continuously and seamlessly incorporate aspects of the program throughout the coursework are consistently discussed in faculty meetings. For example, one recent effort is the consistent messaging on the standards of APA formatting and incorporating it into each applicable course rubric. Several pieces of student feedback were incorporated into the new Fall 2020 student orientation.

167

6) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

As there has been considerable faculty turn-over which has been addressed with recent stability in program leadership, it has been difficult to track faculty teaching effectiveness using a holistic approach. Because the program is online, it is strong in utilizing technology in delivering content to students in an innovative way, with ongoing improvements driven by institutional priorities in Inclusive Excellence (e.g., standardization of recorded lecture transcripts). We have already begun standardizing key instructional components such as syllabi and grading rubrics and continue to incorporate ideas to improve communication of standards to students across the program (e.g., APA style). Through annual evaluations linked to program goals, we have been improving our tracking and starting to celebrate the ways in which faculty are incorporating technology and innovative methodology to share with other faculty in venues such as all-faculty meetings. The annual reviews in the curriculum and assessment committee and annual retreat discussions are key to incorporating innovative strategies and keeping content and methodologies fresh.

168

Section E

E4. Faculty Scholarship

169

Intentionally blank

Divider back

170

E4. Faculty Scholarship

The program has policies and practices in place to support faculty involvement in scholarly activities. As many faculty as possible are involved in research and scholarly activity in some form, whether funded or unfunded. Ongoing participation in research and scholarly activity ensures that faculty are relevant and current in their field of expertise, that their work is peer reviewed and that they are content experts.

The types and extent of faculty research align with university and program missions and relate to the types of degrees offered.

Faculty integrate research and scholarship with their instructional activities. Research allows faculty to bring real-world examples into the classroom to update and inspire teaching and provides opportunities for students to engage in research activities, if desired or appropriate for the degree program.

1) Describe the program’s definition of and expectations regarding faculty research and scholarly activity.

The MPH program has several goals for faculty research and scholarly activity. For example, that at least 50% of primary and non-primary faculty publish at least one scholarly article or commissioned report on topics in rural and underserved communities and that at least 50% of primary and non-primary faculty submit a research or practice grant proposal at PI or co-PI. To assess these and other aspects of faculty research and scholarship, procedures have been put into place for report of faculty output and development. Expectations will be reviewed and reiterated by the program director (USD) and department head (SDSU).

2) Describe available university and program support for research and scholarly activities.

The MPH program has high regard for research and scholarly activities. Faculty research is a unique and highly educational experience MPH students can get involved in. Tenure-track faculty have workload for research time. Each university has additional built in supports in addition to their faculty mentor. Both universities also have an Institutional Office of Research that supports the preparation and submission of grant applications, as well as post-award financial management support offices. USD has a School of Health Sciences Research Committee and SDSU has a Research and Scholarly Activity Committee to help support research and scholarly activities of its faculty. The College of Pharmacy and Allied Health Professions has a dedicated Grant Proposal Specialist who supports faculty development of research grant proposals. Some previous and current examples of sources of funding for extramural research support are the CDC (e.g., Overdose Data to Action, Cooperative Agreement for Emergency Response, Diabetes Prevention and Control Program), most often in partnership with the SD Department of Health. Intramural research and scholarly support have been obtained from sources such as the USD Sanford School of Medicine, USD School of Health Sciences, and the USD Center for Teaching and Learning.

3) Describe and provide three to five examples of faculty research activities and how faculty integrate research and scholarly activities and experience into their instruction of students.

Table E4.1. Integration of Faculty Research and Scholarly Activities in Teaching Faculty Member Research Activity How Incorporated into Instruction Chelsea Wesner Assessing County-level Upon completion of a collaborative project with the Vulnerability for Opioid South Dakota Department of Health, this faculty Overdose, HCV, and developed a case study analysis project for PUBH HIV in South Dakota 750 students. The case study helps students (CDC Cooperative interpret state and county level report cards on the syndemic of overdose, HCV, and HIV developed

171

Agreement – 2018- by MPH faculty and the South Dakota Department, 2019) as well as attain MPH Foundational Competency D2.6. Susan Puumala Overdose Data to Incorporation the idea of epidemiological data on Action (OD2A) (CDC opioid use on hospital policy into PUBH 710 Fall Cooperative Agreement 2020. – 2019-2021) Aaron Hunt CDC’s 1815 project, Examples from the project were used in PUBH 702 “Improving the Health of to explain the potential to utilize pharmacists to Americans through increase access to healthcare services, especially Prevention and among Native Americans in rural and underserved Management of areas, in lectures covering access to care and Diabetes and Heart health disparities impacts on health. This Disease and Stroke” information helped students see examples of interventions that can be used to improve health outcomes.

4) Describe and provide three to five examples of student opportunities for involvement in faculty research and scholarly activities.

Table E4.2. Student Opportunities for Involvement in Faculty Research and Scholarly Activities Faculty Member Research Activity How Students Involved Chelsea Wesner Overdose Data to Action Faculty mentored MPH student during their ILE to (OD2A) (CDC develop and administer a statewide assessment of Cooperative Agreement fatal and non-fatal overdose policies/protocols in – 2019-2021) emergency departments. Results will inform the development of a toolkit and recommendations for model policies and best practices that improve overdose prevention and response efforts in the state. Assessing County-level Faculty mentored MPH student during their Vulnerability for Opioid practicum to develop a statewide and county-level Overdose, HCV, and report cards that highlight the syndemic of opioid HIV in South Dakota overdose, HCV, and HIV in South Dakota. Report (CDC Cooperative cards and a data dashboard are not publicly Agreement – 2018- available and used to inform prevention efforts in 2019) South Dakota and tribal communities. Aaron Hunt CDC Chronic disease Faculty mentored MPH student during their APE in project which they assessed current policies in place that facilitate or hinder the use of pharmacist to provide healthcare services, such as medical therapy management, to patients with chronic diseases. This included assessing current legislation related to expanding prescribing privileges to pharmacists as well as expanding reimbursement and coverage of pharmacy services through private, public and employer-based insurance programs. Susan Strobel Overdose Data to Action Faculty mentored an MPH student during her ILE (OD2A) (CDC and after she graduated with her MPH. The student Cooperative Agreement developed a social media toolkit to assist – 2019-2021) vulnerable, rural, and tribal communities in raising awareness around opioid use and misuse, and to support education with appropriate social media messaging. Toolkit construction consists of five

172

pillars of guidance (What is social media, How to get started, Skill development, and Opioid education, and Resources).

5) Describe the role of research and scholarly activity in decisions about faculty advancement.

Both universities have a promotion and tenure process guided by the Board of Regents/Council on Higher Education policies and procedures. That process (described in A-1) is informed by the annual review process described in E3.2. Research and scholarly activity would be assessed under the “research and scholarly activity” aspect of faculty’s position. The supervisor assesses faculty on a scale of 0=failed to meet expectations, 1=met expectations, 2=performed above expectations, and 3=performed substantially above expectations. There is also room for open-ended comments about research and scholarly activity. Faculty track records in teaching are very important to faculty advancement at both USD and SDSU.

6) Select at least three of the measures that are meaningful to the program and demonstrate its success in research and scholarly activities. Provide a target for each measure and data from the last three years in the format of Template E4-1. In addition to at least three from the list that follows, the program may add measures that are significant to its own mission and context.

Template E4-1. Outcome Measures for Faculty Research and Scholarly Activities Outcome Measure Target Year 1 Year 2 Year 3 2017-2018 2018-2019 2019-2020 Percent of primary faculty participating in research activities 75% 100% 100% 100% each year Number of articles published in 5 7 6 6 peer-reviewed journals each year Presentations at professional 7 19 4 16 meetings each year

7) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The MPH program has a mix of faculty who are in tenure-track research positions and lecturer/ instructor positions. Even as a small program, we have a considerable contribution to research and scholarly work. Faculty have historically done an excellent job incorporating students into their research projects and offering opportunities for them as research advisors on their capstone projects. We look for this to continue growing with the addition of more faculty with research backgrounds and appointments. For example, we are creating evergreen activities, such as with the regional KIDS COUNT Data Center, for students to engage in data analysis and presentation in the fall and data dissemination in the spring. We continue to support junior faculty on their path to tenure, emphasizing publication and scholarly involvement that includes students and circles back around to inform and enhance their teaching.

173

Intentionally blank

174

Section E

E5. Faculty Extramural Service

175

Intentionally blank

Divider back

176

E5. Faculty Extramural Service

The program defines expectations regarding faculty extramural service activity. Participation in internal university committees is not within the definition of this section. Service as described here refers to contributions of professional expertise to the community, including professional practice. It is an explicit activity undertaken for the benefit of the greater society, over and beyond what is accomplished through instruction and research.

As many faculty as possible are actively engaged with the community through communication, collaboration, consultation, provision of technical assistance and other means of sharing the program’s professional knowledge and skills. While these activities may generate revenue, the value of faculty service is not measured in financial terms.

1) Describe the program’s definition and expectations regarding faculty extramural service activity. Explain how these relate/compare to university definitions and expectations.

The MPH program has a focus on rural and underserved communities. Therefore, faculty extramural service and community collaborations are key to meeting our overall mission, and we have several goals related to this aim. In fact, we have a goal that all faculty must engage in some extramural service to rural and underserved populations each year. We also recognize the importance of engaging with local, state, federal, and tribal health departments. This service component is one of the main missions of both USD and SDSU and is the third aspect assessed in promotion and tenure decisions.

2) Describe available university and program support for extramural service activities.

As previously stated, the main mission of the MPH program is focused on rural and underserved community health. As the APE and ILE are community-linked features of the program, maintaining and supporting connections to the community are of utmost importance. Both universities understand as South Dakota’s land grant institution (SDSU) and first and largest liberal-arts university (USD), they are indebted to the larger city and state communities. Many activities, such as speaking engagements, help bridge the universities with the broader community. There are several clubs/organizations (e.g., Center for Diversity and Community) that set the tone for the importance of service. Other supports are at the school/college level. One example is the recent establishment of a faculty member at the USD School of Health Sciences to have dedicated time to spearhead and guide relationships with tribal nations. Support of faculty members’ extramural work is evidenced through workload for service. Faculty members serve the professional community at the national level through board engagement and grant reviews, and the local community through coalitions and advisory boards. Support for these activities is made through flexibility of other commitments and celebrating this involvement as critical to advancing public health locally and nationally.

3) Describe and provide three to five examples of faculty extramural service activities and how faculty integrate service experiences into their instruction of students.

Table E5.1. Faculty Extramural Service Integration in Teaching Faculty Service Activity How Incorporated into Instruction Member Chelsea Weekend Backpack As a volunteer and board member, helped to establish Wesner Program/Tanager Take- a school-based food pantry for families with children in out the Vermillion School District in 2016. As part of PUBH 750, created assignment for student groups to create compendium and presentation on strategies to improve food security.

177

Sandi Public health consultant Included lecture content on topics on evidence-based Melstad policy partnership practice and health equity from work with the SD examples Department of Health. Policy change example Included case study from consultant work with State of SD on childcare and worksite UV policy. Tom Stenvig Immunization committee Plans to include examples from Immunization member committee into development of Dissemination and Implementation course.

4) Describe and provide three to five examples of student opportunities for involvement in faculty extramural service.

Table E5.2. Faculty Extramural Service and Student Involvement Faculty Service Activity How Students Involved Member Chelsea Community Connection As a volunteer and board member, helped establish a Wesner Center centralized Community Connection Center (CCC) comprised of 8+ nonprofits. Has engaged four MPH students through APE and ILE projects that support the CCC in the development of a health navigator program and student-run interprofessional health clinic. DenYelle Grant writing for Capstone student leading grant writing for Delta Dental Kenyon healthcare careers camp grant to fund camp activities. Mary Beth Community mental Led a community mental health needs assessment and Fishback health data analysis engaged more than three MPH students through APE and ILE projects to analyze and summarize quantitative and qualitative data, and disseminate results

5) Select at least three of the indicators that are meaningful to the program and relate to service. Describe the program’s approach and progress over the last three years for each of the chosen indicators. In addition to at least three from the list that follows, the program may add indicators that are significant to its own mission and context.

Table E5.3. Outcome Measures for Faculty Service Activities Year 1 Year 2 Year 3 Outcome Measure Target 2017-2018 2018-2019 2019-2020 Percent of primary faculty participating in extramural service 100% 100% 100% 75% activities each year Number of faculty-student service 5 5 7 7 collaborations Number of service projects 5 8 7 14

Table E5.3 outlines the metrics and progress of our primary faculty’s involvement in service activities over the past three years. These metrics represent service outside of the university experiences/committees, at the national, regional, state, and local levels. Indicators track the number of service projects and the number of projects students are involved in (including APE and IPE experiences that are outside research endeavors but were in partnership with public health organizations). If a goal continues to be consistently met for four consecutive years, new goals will be established by the Curriculum and Assessment Committee. Either a higher target will be set, or a new metric will be chosen to focus on.

178

6) Describe the role of service in decisions about faculty advancement.

Both universities have a promotion and tenure process guided by the Board of Regents/Council on Higher Education policies and procedures. That process (described in A-1) is informed by the annual review process described in E3.2. Service activity would be assessed under the “Service” aspect of faculty’s position. The supervisor assesses faculty on a scale of 0=failed to meet expectations, 1=met expectations, 2=performed above expectations, and 3=performed substantially above expectations. There is also room for open-ended comments about service activity. Faculty track records in service are very important to faculty advancement at both USD and SDSU.

7) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The MPH program has faculty strongly dedicated to both intramural and extramural service. Many examples of the extramural service are through service to community-based organizations, as well as academic organizations. Many examples of this service component carry out the overall mission of improving the health of rural and underserved populations. We have a strong history of involvement of students’ APE and ILE projects, and future goals include continuing to cultivate our external partners to mentor students to strengthen the program’s ties to the community.

179

Intentionally blank

180

Section F

F1. Community Involvement in School or Program Evaluation and Assessment

181

Intentionally blank

Divider back

182

F1. Community Involvement in Program Evaluation and Assessment

The program engages constituents, including community stakeholders, alumni, employers and other relevant community partners. Stakeholders may include professionals in sectors other than health (e.g., attorneys, architects, parks and recreation personnel).

Specifically, the program ensures that constituents provide regular feedback on its student outcomes, curriculum and overall planning processes, including the self-study process.

1) Describe any formal structures for constituent input (e.g., community advisory board, alumni association, etc.). List members and/or officers as applicable, with their credentials and professional affiliations.

The MPH program first convened a Community Advisory Board in August 2019. The group meets 1-2 times/year to advise the MPH program on aspects such as the mission/vision/goals, curricula, and emerging public health issues. We also ask the board to reflect on the public health workforce needs in the region. Potential members were brainstormed by all MPH faculty members, and invitations were emailed to a variety of public health professionals, with an eye towards having a variety of members from tribal health, government, service sectors, professional organizations, alumni and current students. Please find the list of members with their credentials below and the meeting minutes in the ERF (F1-1 MPH Advisory Board Meeting Notes 8-19-19; F1-2 MPH Advisory Board Meeting Notes 1-16-20).

Table F1. Community Advisory Board Membership Name Title/Role Tribal Health Jennifer Giroux, MD Former Medical Epidemiologist who established the Great Plains Tribal Epidemiology Center; currently with Great Plains Area Indian Health Service Government Joshua Clayton, PhD, MPH State Epidemiologist, South Dakota Department of Health Service Sector Mary Michaels, BS Public Health Prevention Coordinator, Live Well Sioux Falls Jyoti Angal, MPH, CIP Director of Clinical Research, Avera Research Institute Connie Halverson, BA Vice President of Public Benefit, Delta Dental Kris Graham, MS, LPC, Interim CEO, Southeastern Behavioral Health QMHP Professional Organizations Deana Around Him, DrPH Senior Research Scientist, Child Trends Alumni Tyler Hemmingson, MPH Laboratory Manager, Regional Health Michaela Seiber, MPH Research Coordinator, South Dakota State University, President of SDPHA Current Student Sylvia Lasley, MA, LPC Current MPH student (background in tribal behavioral health and university experience coordinating programs for first generation college students)

2) Describe how the program engages external constituents in regular assessment of the content and currency of public health curricula and their relevance to current practice and future directions.

183

Our program convenes the Community Advisory Board 1-2 times/year to advise the MPH program on aspects such as the mission/vision/goals, curricula, and emerging public health issues. We also ask the board to reflect on the public health workforce needs in the region. Their feedback at the first meeting was invaluable to shape the course of our concentration competencies and planned redesign of the ILE and new course to prepare for the ILE. Their feedback in the second meeting was immensely helpful for finalizing the benchmarks and measurement of the evaluation goals.

3) Describe how the program’s external partners contribute to the ongoing operations of the program. At a minimum, this discussion should include community engagement in the following:

a) Development of the vision, mission, values, goals and evaluation measures

The August 2019 Community Advisory Board meeting focused on an overview of the vision, mission, values, and goals of the program, and the January 2020 meeting helped finalize the measurement of the evaluation goals and setting appropriate benchmarks. In forthcoming years, the Advisory Board will review the vision, mission, values, goals and evaluation measures annually.

b) Development of the self-study document

At the August 2019 Community Advisory Board meeting, the program director provided an overview of the accreditation process and topics where their feedback would be most helpful. Specifically, the Advisory Board focused on the development of the five concentration competencies. The board members provided valuable feedback on emerging skills needed in the public health field, which the Curriculum and Assessment Committee incorporated in a revision of the program’s concentration competencies.

At the January 2020 Community Advisory Board meeting, much of the meeting focused on giving feedback on the program goals, measurement of the objectives, and setting appropriate benchmarks. Members gave feedback on the level of some of the target goals, helped brainstorm what knowledge would be important to gain in the APE and how to recruit and retain preceptors.

c) Assessment of changing practice and research needs

As described above, the Community Advisory Board provided valuable feedback on changing practice needs in public health in South Dakota, particularly about the diverse range of activities that public health professionals in the state are being asked to complete (such as budgetary aspects of grant management and navigating tribal research review boards). This type of feedback is gathered at every Advisory Board meeting, and if there is a focused topic we identity a need to get more in-depth feedback on, we will ask particular members for 1:1 review or key informant interviews.

We will also reach out to recent alumni who are employed to assess if the program content was relevant to their practice/employment needs. This assessment will be conducted via phone or email, through open-ended questions (e.g., What competencies is the MPH program getting right to prepare you for the workforce? What are the skills you wished you would have gotten more time to build?).

d) Assessment of program graduates’ ability to perform competencies in an employment setting

184

Much enriching conversation was held at the first Community Advisory Board meeting about student learning, practice, and research experiences during the program and how that prepared them for the real-world public health setting, particularly in South Dakota. For example, Advisory Board members shared the need for public health professionals in the state to have project management and advanced leadership skills. Based on this feedback, the program faculty and the Curriculum and Assessment Committee voted to support a curriculum change to incorporate an additional project management experience through the creation of a new core course in Leadership and Project Management. Discussion in the second Advisory Board meeting had focused discussion on building partnerships for APE activities and providing suggestions on organizations the MPH program could build partnerships. Several connections were made.

We have ongoing relationships with several public health organizations that employ our alumni (e.g., Sanford Research – Alyson Becker; South Dakota Department of Health – Amanda Nelson & Deb Rumrill; Siouxland Public Health – Eva Kluender; Monument Health – Tyler Hemmingson; Avera Research Institute Center for Pediatric & Community Research – Brian Juran),who we have included in the Employer Survey. 100% of respondents to the Workforce Survey (F1-3 Public Health Workforce Survey 2020) who have supervised an MPH graduate student (n = 12) in the past responded that the students had the skills and/or knowledge they expected. We are encouraged that 100% of those respondents also said that if they were in the position to hire in the future, they would hire that graduate they worked with. One example open-ended response discussed the specific public health competencies the student was assisting with “...participation in the MPH program has added a lot of value to our programming in terms of the importance of evaluation and a better understanding of public health issues like policy work, data, etc.” The workforce survey report can be found in the ERF (F1-4 Public Health Workforce Survey Results, August 2020).

4) Provide documentation (e.g., minutes, notes, committee reports, etc.) of external contribution in at least two of the areas noted in documentation request 3.

Please see the minutes from the advisory board meetings in the ERF (F1-1 MPH Advisory Board Meeting Notes 8-19-19; F1-2 MPH Advisory Board Meeting Notes 1-16-20). These minutes provide evidence of advisory board discussion and a key informant 1:1 discussion with advisory board member on mission, vision, and goals as well as skills needed for the public health workforce.

5) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

We are excited about the early input from our experienced and engaged Community Advisory Board and how it has already impacted major decisions about the concentration competencies and core curriculum. We hope these changes help provide a more ready and capable public health workforce in the future. We plan to continue to hold Advisory Board meetings to gain input on development of the self-study accreditation document as well as evaluation and assessment metrics. We will also continue to hold 1:1 key informant interviews with members to allow a deeper dive on emerging topics on which we would like additional feedback. The Workforce Survey underscored employers’ need for public health professionals to hold skills in communication, evidence-based approaches to public health, and planning and management to promote health. The MPH program will continue to strengthen curricula to build these skills so our alumni are skilled and viable public health professionals entering the workforce.

185

Intentionally blank

186

Section F

F2. Student Involvement in Community and Professional Service

187

Intentionally blank

Divider back

188

F2. Student Involvement in Community and Professional Service

Community and professional service opportunities, in addition to those used to satisfy Criterion D4, are available to all students. Experiences should help students to gain an understanding of the contexts in which public health work is performed outside of an academic setting and the importance of learning and contributing to professional advancement in the field.

1) Describe how students are introduced to service, community engagement and professional development activities and how they are encouraged to participate.

PUBH 750 (Social and Behavioral Sciences in Public Health) includes a service-learning component in which students complete at least 9 hours of service in a local health or social service organization. This experience synthesizes course content and informs a written assignment (Service Learning Reflection). Hours are intended to be spaced throughout the semester (e.g., 3 hours per month) and verification of service hours is required. This experience provides students the opportunity to align service with a Health Promotion Project. Some students have reported this service-learning experience led to a practicum opportunity. One student was inspired to host a large food drive after completing service learning with a regional food bank. Due to COVID-19 preventing some in-person service opportunities, Fall 2020 there is an optional Interprofessional Discussion Series titled Human Experience: Self and Others focused on diversity and social determinants of health topics.

South Dakota Public Health Student Association (SDPHSA): Faculty and students are currently creating the SDPHSA which will launch in Fall 2020. The SDPHA will recruit students to participate in service, community engagement and professional development activities throughout the year. The activities will be in person for students in South Dakota as well as virtual for distance participants. The SDPHSA will also encourage students to get involved in their local communities. The SDPHSA will be promoted at orientation as well as though social media, advising, and word of mouth. The SDPHSA will offer two scholarships to attend the Annual American Public Health Association meeting in October 2020 and continue for future years to encourage engagement with the largest public health organization in the United States as well as fostering professional development.

Student Portfolio: Students will be required to create a Public Health Portfolio that will be used to track student activities including professional development, services projects, community engagement, and all trainings. This will be introduced during student orientation and added to core courses to ensure completion by students. Trainings and events will be promoted to students through emails and social media. In addition, students will be encouraged to share any public health activities with faculty and students. Students will also be required to upload annual curriculum vitae (CV) and encouraged by advisors to attend trainings and community events to enhance their CV for future career opportunities.

2) Provide examples of professional and community service opportunities in which public health students have participated in the last three years.

The MPH program communicates service and professional development opportunities with students through the MPH email listserv and through the “Orientation/Resource Center” D2L site. We have had several students engaged in professional and community service opportunities across the past few years. For example:

In 2016, food insecurity was a major theme of service learning and projects in PUBH 750 (Social and Behavioral Sciences in Public Health). During that year, all students enrolled in the course completed service learning with nonprofit and governmental organizations that administer nutrition and food assistance programs. Most students served in regional food banks, local food pantries, and community kitchens, while a few served in WIC agencies.

189

Between 2017-2019, students enrolled in PUBH 750 have served in a variety of health and social service organizations including, but not limited to: • Clinical focus: community hospital emergency departments, community health centers (medical, dental, mental health) • Community-based: regional foods banks, food pantries, community kitchens, shelters, Dress for Success, domestic violence shelter, nonprofit for Somali women and families, Meals on Wheels, Health Connect • Governmental organizations: WIC, Veterans Affairs nursing home, city fire department, South Dakota Department of Social Services

Among the most impactful service experiences include: 1) a student who was inspired to become trained as a Court Appointed Special Advocate (CASA) and completed the four-day training because of their experience, 2) a student who completed training and requirements to become a foster parent because of their service learning and course content, and 3) students getting involved with the South Dakota Public Health Association through leadership roles, conference planning, and presentations. In addition, students are leading the formation of the South Dakota Public Health Student Association to being in Fall 2020, which will provide students with professional development, public health training, and outreach opportunities.

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Aside from PUBH 750, we have not established a method of tracking student participation in community and professional service. Starting in fall 2020, we will require that all students participate in at least one community or professional service activity each year. While the service-learning component in PUBH 750 will be an option for community service, our program plans to promote opportunities for other community and professional service activities throughout the year, as well as track participation on an annual basis through the student portfolio.

190

Section F

F3. Assessment of the Community’s Professional Development Needs

191

Intentionally blank

Divider back

192

F3. Assessment of the Community’s Professional Development Needs

The program periodically assesses the professional development needs of individuals currently serving public health functions in its self-defined priority community or communities.

1) Define the program’s professional community or communities of interest and the rationale for this choice.

The annual South Dakota Public Health Association (SDPHA) conference is held to provide an opportunity to connect health professionals and promote public health initiatives in rural, tribal, and urban communities. This is integral to our outlined mission and goals. Faculty, students, and alumni have taken a large responsibility for the conference planning and execution.

2) Describe how the program periodically assesses the professional development needs of its priority community or communities and provide summary results of these assessments. Describe how often assessment occurs.

The MPH program assesses the professional development needs of the public health community in several ways. Our Advisory Board is a key body of stakeholders in soliciting that feedback. One of the revolving agenda items in our Advisory Board meetings is on the professional development needs of the public health community. The notes are included the ERF (section F1). Recommendations include training in grant writing, Institutional Review Board processes, and research ethics training.

Another method of soliciting feedback is through the APE preceptor evaluation surveys, to gain feedback from preceptors on student performance, strengths, and recommendations (see ERF section D5). Questions include a focus on their perceptions of the local community’s workforce development needs, in addition to questions on their satisfaction with the mentoring experience.

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

We continue to assess the professional development needs of public health practitioners and researchers in South Dakota through annual SDPHA conference surveys and through our Community Advisory Board meetings. In addition, we plan to conduct key informant interviews with any stakeholders interested in discussing professional development needs further.

193

Intentionally blank

194

Section F

F4. Delivery of Professional Development Opportunities for the Workforce

195

Intentionally blank

Divider back

196

F4. Delivery of Professional Development Opportunities for the Workforce

The program advances public health by addressing the professional development needs of the current public health workforce, broadly defined, based on assessment activities described in Criterion F3. Professional development offerings can be for-credit or not-for-credit and can be one- time or sustained offerings.

1) Describe the program’s process for developing and implementing professional development activities for the workforce and ensuring that these activities align with needs identified in Criterion F3.

Considering the areas of interest from the SDPHA attendees and feedback from our Community Advisory Board, we have identified a goal of conducting/organizing two professional development activities on behalf of our program per year. These activities (e.g., webinars, in-person presentations, trainings) will be branded as sponsored/co-sponsored by the MPH program.

2) Provide two to three examples of education/training activities offered by the program in the last three years in response to community-identified needs. For each activity, include the number of external participants served (i.e., individuals who are not faculty or students at the institution that houses the program).

As detailed above, the SDPHA is a strong partner for connecting with the public health workforce in the area. One area of identified need in recent years is for local data in the opioid crisis, and a presentation was given at the annual conference with the South Dakota Department of Health in response to that need. Other identified needs from our Advisory Board were on capacity building, and covering topics such as grant writing, IRB training, program management, policy needs, and applying public health principles to real life hot topics. Therefore, a webinar was held by Dr. Susan Strobel including the latest data and policy changes in vaping. An additional hot topic of the team’s public health response to the COVID-19 was given June 2020 to showcase a public health case study response. Planned presentations in response to additional Advisory Board feedback include a webinar on grant writing, disease investigation, and an update on the MPH program for the annual SDPHA conference in September 2020.

Table F4. Education and Training Activities Estimated # Education/training External Participants Assessing County-Level Vulnerability for Opioid Overdose and Rapid Spread 175 of HIV and Hepatitis C, June 2019 SDPHA Conference, Wesner (faculty) and Melstad (adjunct) co-presenter Vaping webinar, January 2020, Strobel presenter 55 Maternal Mortality in the Dakotas. April 2020 National Public Health Week 57 webinar (Ashley Briggs & Aaron Hunt). Responding to COVID-19 in rural America: A public health case study, June 40 2020 ECHO Presentation (Wesner, Puumala, Strobel, & Kenyon (co- presenters) USD-SDSU MPH Program Update, SDPHA Conference presentation, planned -- for September 2020, Kenyon & Hunt presenters

3) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

197

We will continue to assess the professional development needs of the public health workforce through the SDPHA conference surveys and through our Advisory Board meetings. We plan to hold follow-up 1:1 key informant interviews with any stakeholders interested in discussing professional development needs further. We plan to continue development of various marketing methods (email, listservs, Facebook) to advertise the professional development opportunities.

198

Section G

G1. Diversity and Cultural Competence

199

Intentionally blank

Divider back

200

G1. Diversity and Cultural Competence

Aspects of diversity may include age, country of birth, disability, ethnicity, gender, gender identity, language, national origin, race, historical under-representation, refugee status, religion, culture, sexual orientation, health status, community affiliation and socioeconomic status. This list is not intended to be exhaustive.

Cultural competence, in this criterion’s context, refers to competencies for working with diverse individuals and communities in ways that are appropriate and responsive to relevant cultural factors. Requisite competencies include self-awareness, open-minded inquiry and assessment and the ability to recognize and adapt to cultural differences, especially as these differences may vary from the program’s dominant culture. Reflecting on the public health context, recognizing that cultural differences affect all aspects of health and health systems, cultural competence refers to the competencies for recognizing and adapting to cultural differences and being conscious of these differences in the program’s scholarship and/or community engagement.

1) List the program’s self-defined, priority under-represented populations; explain why these groups are of particular interest and importance to the program; and describe the process used to define the priority population(s). These populations must include both faculty and students and may include staff, if appropriate. Populations may differ among these groups.

For students, our priority URM population is defined as meeting one or more of the following criteria: • American Indian/Alaska Native • First-generation • Lives in a rural community (i.e., the population is less than 2,500)

The first-generation indicator is collected on the common application as well as the current and alumni survey. Students are asked to respond to the following question, “What is the highest level of education attained by any parent?” A student who responds with any response other than “bachelor’s degree or higher” will be considered first-generation. This logic was developed by the South Dakota Board of Regents: https://www.sdbor.edu/administrative- offices/academics/aac/Documents/2017-04-AAC/4_H_AAC0417.pdf

More broadly, outside the priority student URM population we will track URM faculty and students with the following definition of underrepresented minority groups—blacks or African Americans, Hispanics or Latinos, and American Indians or Alaska Natives.1

2) List the program’s specific goals for increasing the representation and supporting the persistence (if applicable) and ongoing success of the specific populations defined in documentation request 1.

The program plans to increase the representation and persistence of URM students through continued word of mouth recruitment, emphasizing the program’s focus on rural and underserved populations, particularly Native American communities. This helps highlight our focus on rural, Native American, and first-generation students as priority populations. When admission is extended to students of these populations, starting in spring 2020, we reach out with follow-up inquiries to answer questions about the program and encourage matriculation.

1National Science Foundation, National Center for Science and Engineering Statistics. 2017. Women, Minorities, and Persons with Disabilities in Science and Engineering: 2017. Special Report NSF 17-310. Arlington, VA. Available at www.nsf.gov/statistics/wmpd/.

201

The program will continue its efforts towards supporting diverse, URM faculty. This is accomplished through personalized outreach and posting of job ads on a variety of job boards, many of which focus on rural and American Indian public health efforts. Much effort is given to recruitment of diverse faculty to mentor and teach in our program, as well as efforts to retain diverse faculty. Oftentimes URM faculty have higher workplace demands, therefore, these role strains will be taken into account when assigning workload, particularly for service obligations to ensure URM faculty are not at higher risk for burnout. This commitment to diversity is accomplished in slightly different ways by the institutions, such that SDSU includes a diversity representative on all search committees, and USD has an annual diversity training all committee members are required to complete.

3) List the actions and strategies identified to advance the goals defined in documentation request 2, and describe the process used to define the actions and strategies. The process may include collection and/or analysis of program-specific data; convening stakeholder discussions and documenting their results; and other appropriate tools and strategies.

In order to make progress on increasing the representation and persistence of URM students, the program has several processes in place to define and evaluate our strategies to achieve these goals. One is an annual report presented to the USD Health Affairs Administrative council, which includes race/ethnicity of current students and alumni, and percent of target populations. These numbers are compared with the previous year and are also shared at the Fall MPH retreat. Data is also collected in the Faculty Database, which is sent annually to primary and non-primary faculty to self-report their activities on the previous academic year. This includes activities such as recruitment and admissions follow-up. Prior to the next years’ admissions cycle, these data are shared back during All Faculty meetings, and discussions are held on participation at upcoming recruitment events.

Table F1-3. Strategies to Advance Diversity Representation Students Assessment Word of mouth recruitment focusing on Document recruitment efforts MPH vision/mission & priority populations Admission follow-up Document admissions contacts Faculty Position outreach Document search process URM Faculty retention Document workload assignments

The program faculty are involved in many recruitment and retention efforts for Native American, rural and first-generation students. There are continued interactions with faculty and staff from TRiO, Upward Bound, SDSU’s Wokini Initiative, and USD's President’s Council on Diversity and Inclusion, and USD’s HHMI Inclusive Excellence program. As the other part of the program director’s position is Associate Dean for Diversity and Inclusion for the Sanford School of Medicine, the program is primed to contribute to diversity and inclusion efforts at the broader Universities and throughout the region.

4) List the actions and strategies identified that create and maintain a culturally competent environment and describe the process used to develop them. The description addresses curricular requirements; assurance that students are exposed to faculty, staff, preceptors, guest lecturers and community agencies reflective of the diversity in their communities; and faculty and student scholarship and/or community engagement activities.

As detailed above, diversity is one of the MPH program’s core values. Our mission/vision also focuses on optimal health for rural and underserved populations, particularly American Indians. Many of our goals and objectives reflect the importance of diversity and engagement with rural and underserved communities, therefore, there is content woven throughout the MPH curriculum that

202

focuses on our priority areas. There are expectations for ongoing diversity training for faculty and service components for students.

The process to create and maintain our culturally competent environment is through regular faculty meetings and annual retreat to review student, alumni, and other stakeholder (e.g., Advisory Board, APE/ILE preceptors) perceptions of our program’s service to diversity. For example, in the summer 2019 retreat where a curricular mapping occurred, this area of improvement was identified and a curriculum change to move the course PUBH 760: Public Health and Native American Communities to our core courses was put forth to faculty and the Curriculum and Assessment Committee.

5) Provide quantitative and qualitative data that document the program’s approaches, successes and/or challenges in increasing representation and supporting persistence and ongoing success of the priority population(s) defined in documentation request 1.

In summer 2019, our Student Survey was administered to active students to gather feedback on the program. The response rate on the 2019 survey was 45%. One of the questions obtained student input on factors that have hindered students’ educational progress. Thirty percent of the respondents indicated that unfair treatment because of gender, race/ethnicity, sexual orientation, or religion had hindered their educational progress somewhat or to a great extent. Further investigation was needed to gain insight on the challenges the students are facing and how the program can better support students from our priority populations, as it was unclear if students are answering this question more broadly about their environment or their time in the MPH program. We want to know more about their direct experiences with MPH coursework, faculty, and advising. New questions, launched in Spring 2020, were developed on the MPH climate for diversity and inclusion to gather more program-specific information. These questions were included on the current, alumni, and exit surveys. The results are shown in table G1.2. below.

As mentioned above, the Student Survey was administered again in summer 2020 to active students to gather feedback on the program. The response rate on the survey was 52%. Three percent of the respondents indicated that unfair treatment because of gender, race/ethnicity, sexual orientation, or religion had hindered their educational progress somewhat or to a great extent. The difference between the mean in 2019 (2.00 on a 4.00 scale) and 2020 (1.10 on a 4.00 scale) is statistically significant. We were happy to see this improvement, with much fewer students reporting unfair treatment in their direct experiences in the MPH program.

Table G1. Outcome Measures for Recruitment and Admissions Outcome Measure Target Year 1: Year 2: Year 3: 2017-2018 2018-2019 2019-2020 Percentage of priority URM students 40% 37.78% 40.74% 38.46% accepting offers of admission The composition of URM faculty in the 10% 33% 20% 10% MPH program (all faculty) (see ERF G1- 1 Documentation for Faculty URM Data)

203

Table G1.2. Student Perceptions of Diversity and Inclusion During Program (N=30) Diversity & Cultural 2019 2019 2019 2020 2020 2020 Statistically Competency Overall Overall Mean Overall Overall Mean Significant Question # % Overall # % Overall (p<0.05) How much has unfair treatment because of your gender, race/ethnicity, sexual orientation, or religion HINDERED your educational progress in the MPH program? 1=Not at All 13 43% 29 97% 2=Slightly 8 27% 0 0% Yes 3=Somewhat 5 17% 2.00 0 0% 1.10 (p-value 0.0001) 4=To a Great 4 13% 1 3% Extent

Student, alumni, and recent graduate perceptions of the culture on diversity and inclusiveness were collected from the most recent surveys. The results are included below.

Table G1.3. Student, Recent Graduates and Alumni Perceptions of Diversity and Inclusion During the MPH Program 2020 Exit Survey Graduate Alumni Current Student (N=10)* Survey Climate and Diversity Survey (N=7)** (N=32)* Overall # Overall % Overall # Overall % Overall # Overall % In terms of diversity, the general climate of the MPH program is: Exclusionary 1 3% 0 0% 0 0% Neither inclusive nor 4 13% 1 17% 3 43% exclusionary Inclusive 23 77% 3 50% 3 43% Not Important to Me 2 7% 2 33% 1 14% During the program, I feel that I have personally experienced (check all that apply): Apathy/being ignored 3 9% 1 14% 2 22% Bias 2 6% 1 14% 0 0% Discrimination 0 0% 0 0% 0 0% Exclusion 1 3% 0 0% 1 11% Isolation 0 0% 0 0% 1 11% None of the above 26 81% 5 71% 5 56% In MPH courses, issues of diversity and inclusion are: Not adequately 0 0% 0 0% 0 0% addressed Adequately addressed 16 55% 4 67% 4 57% in some courses Adequately addressed 13 45% 2 33% 3 43% in all courses *These questions were optional; therefore, not all respondents answered every question. **31 respondents completed the graduate alumni survey; however, the diversity and inclusion questions were added in fall 2019; therefore, only the fall 2019 alumni had the opportunity to respond to the question. The response rates are as follows: 2020 Current Student Survey: 52%; Exit Survey: 48%; Graduate Alumni Survey: 54%.

204

6) Provide student and faculty (and staff, if applicable) perceptions of the program’s climate regarding diversity and cultural competence.

As mentioned above, new items were developed on student and alumni perceptions of the culture on diversity and inclusiveness which will be included in all student surveys (current students, exit, and alumni) during spring 2020. A faculty and staff survey including open-ended questions on their perceptions of the MPH program, including diversity and inclusion will be collected Fall 2020.

7) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Currently, our percentages for underrepresented students and faculty are strong. Reviewing the last three years of data, we are nearing or exceeding our benchmarks, and foresee continued growth in the future. However, we need more in-depth data to better understand perceptions around diversity and inclusion for our program. Our piloted question in the previous survey on unfair treatment because of students’ gender, race/ethnicity, sexual orientation, or religion hindering their educational progress yielded higher responses than we’d like, and it is unclear if that unfair treatment occurred within the MPH program. Wanting to know more about their direct experiences with MPH coursework, faculty, and advising, new questions were launched in Spring 2020 to gather more program-specific information. Reports of unfair treatment were much lower. We plan to add additional questions, including open-ended questions to better understand why/how the climate of diversity/inclusiveness could be improved. We hope to gain knowledge about what we can implement in the future, if students’ unfair treatment is a part of the MPH program or due to external factors. We also plan to implement a new faculty/staff survey to gain a fuller picture of the diversity and inclusion climate for the MPH program. This data will be reviewed in all-faculty and annual faculty retreats to celebrate positive areas and address areas of concern.

205

Intentionally blank

206

Section H

H1. Academic Advising

207

Intentionally blank

Divider back

208

H1. Academic Advising

The program provides an accessible and supportive academic advising system for students. Each student has access, from the time of enrollment, to advisors who are actively engaged and knowledgeable about the program’s curricula and about specific courses and programs of study. Qualified faculty and/or staff serve as advisors in monitoring student progress and identifying and supporting those who may experience difficulty in progressing through courses or completing other degree requirements. Orientation, including written guidance, is provided to all entering students.

1) Describe the program’s academic advising services. If services differ by degree and/or concentration, a description should be provided for each public health degree offering.

Upon admission to the MPH program, students receive an admission letter from the Graduate School and welcome letter from our program with information on getting started and orientation (see ERF H1-1 New Student Welcome Letter). Shortly after admission, academic advisors connect with students to schedule an initial advising session (30-60 minutes), during which they complete the program of study, review the MPH Orientation and Resource Center, discuss expectations for the APE and ILE, and review steps for registration. Thereafter, students are encouraged to visit with their advisor each semester and schedule meetings as needed. The role of academic advisors includes: 1) helping students navigate university systems and program policies, 2) connecting students with faculty, resources, organizations, and community partners as needed, 3) providing academic advising, and 4) serving as a mentor for career advising.

2) Explain how advisors are selected and oriented to their roles and responsibilities.

Students are assigned to an academic advisor by the program director (USD students) and program coordinator (SDSU students) upon matriculation. If public health interests of students are known at the time of application, the program director and program coordinator aim to find a match with current faculty expertise and interests. Academic advisors are orientated to their roles and responsibilities through individual meetings with the program director for a faculty advising orientation, which includes an overview of faculty advising resources (e.g., program of study, catalogs, Graduate School forms), common procedures for academic advising (e.g., welcome letter, introduction to advisor, registration), and scenarios for supporting students at risk for withdrawing from the program. In addition, academic advisors receive training/resources in navigating university databases for advising, as well as forms and procedures for drop/add, withdrawal, leave of absence, change of catalog, program of study, and registration cancellation.

3) Provide a sample of advising materials and resources, such as student handbooks and plans of study, that provide additional guidance to students.

See ERF H1 for a sample welcome letter (H1-1), advising materials (course rotation schedules – H1-2, program of study – H1-3; advising resources – H1-4), and student handbook (H1-5).

4) Provide data reflecting the level of student satisfaction with academic advising during each of the last three years. Include survey response rates, if applicable.

Our program uses three surveys to collect data on student satisfaction with academic and career advising: Alumni Survey, Exit Survey, and Student Survey. During summer 2019 & summer 2020, an email invitation and two follow-ups were sent to all alumni and current students inviting them to participate in one of the surveys. Email addresses for the alumni were collected from the Alumni Office, the student information system, and the master student list. The student’s home institution email address was used for the student survey. If the email address was invalid (i.e., bounce back/filtered), an alternate email address was used to collect feedback. To improve the response rate, alumni who completed the survey were entered in a drawing for a $25 gift card to Walmart.

209

The surveys were open for two weeks. Only non-respondents were contacted in the reminder emails.

An additional method was devised to increase survey responses for the Alumni Survey, a personalized follow-up of alumni who did not respond to the initial survey was conducted encouraging them to complete the survey. This prompted an additional six alumni to complete it.

The Exit Survey was launched at the end of the fall 2019 semester. This survey was distributed to students enrolled in the capstone course PUBH 730.

A full description of the student survey including results and faculty meeting minutes where results were discussed can be found in the ERF (H1-6 Current Student Survey).

Table H1. Alumni and Student Survey Characteristics Alumni Survey Number of Alumni Number of Alumni 57 Number of Bounce Back 0 Number of Alumni Contacted 57 Number of Alumni Completing the Survey 31 Overall Response Rate (%) 54% Student Survey 2019 Number of Students Number of Current Students 69 Number of Bounce Backs/Filtered 0 Number of Students Contacted 69 Number of Students Completing the Survey 31 Overall Response Rate (%) 45% Student Survey 2020 Number of Students Number of Current Students 61 Number of Bounce Backs/Filtered 0 Number of Students Contacted 61 Number of Students Completing the Survey 32 Response Rate 52% Exit Survey 2020 Number of Graduates Number of Graduates 21 Number of Bounce Backs/Filtered 0 Number of Graduates Contacted 21 Number of Graduates Completing the Survey 10 Overall Response Rate (%) 48%

Alumni were asked to rate their level of satisfaction with the academic advising received when enrolled in the MPH program. Seventy-seven percent of respondents were satisfied or very satisfied with academic advising in the MPH program.

Table H1.2. Alumni Satisfaction with Academic Advising (n=31)

Overall # Overall % Mean Overall Alumni Satisfaction with Academic Advising. 1=Very Dissatisfied 4 13% 2=Dissatisfied 3 10% 2.97 3=Satisfied 14 45% 4=Very Satisfied 10 32% Note. The alumni who completed the survey, may not have answered every question. Response rate was 54%.

210

Recent graduates were asked to rate their level of satisfaction with the academic advising received when enrolled in the MPH program. 100% of the survey respondents were satisfied or very satisfied with academic advising in the MPH program.

Table H1.3. Recent Graduates’ (Exit Survey) Satisfaction with Academic Advising (n=10*)

Overall # Overall % Mean Overall Recent Graduates’ Satisfaction with Academic Advising. 1=Very Dissatisfied 0 0% 2=Dissatisfied 0 0% 3.25 3=Satisfied 6 75% 4=Very Satisfied 2 25% Note. *The graduates who completed the survey, may not have answered every question, including the question above. Eight of the ten respondents responded to the question above. Response rate was 48%.

A survey was administered in 2019 and 2020 to current MPH students asking them to rate their academic advising experience thus far. In 2019, eighty-five percent of respondents agreed or strongly disagreed with the statement, “Overall, I am satisfied with my advisor.” compared to ninety- four percent on the 2020 survey.

Table H1.4. Student Satisfaction with Academic Advisor 2019 Current Student Survey 2020 Current Student Survey Students Students Reporting Reporting Total Mean Total Mean “Agree” / “Agree” / Question Respondents Agreement Respondents Agreement “Strongly “Strongly Rating Rating (N) Agree” (N) Agree” % (n) % (n) Overall, I am 27 85% (23) 3.30 31 94% (29) 3.65 satisfied with my advisor. My advisor answers 30 83% (25) 3.20 31 97% (30) 3.71*** my questions promptly and accurately. My advisor can 29 83% (24) 3.34 30 97% (29) 3.63 answer questions about university policies and procedures related to my degree program. My advisor assists 29 79% (23) 3.28 29 93% (27) 3.45 me in selecting and enrolling in classes each semester (if help is needed).

211

Table H1.4. Student Satisfaction with Academic Advisor 2019 Current Student Survey 2020 Current Student Survey Students Students Reporting Reporting Total Mean Total Mean “Agree” / “Agree” / Question Respondents Agreement Respondents Agreement “Strongly “Strongly (N) Rating (N) Rating Agree” Agree” % (n) % (n) My advisor provides 29 83% (24) 3.24 31 94% (29) 3.58 me with information about graduation requirements, plan of study and deadlines. My advisor is able to 26 85% (22) 3.23 31 97% (30) 3.61 see me or communicate with me within a reasonable time when I have asked a question or requested information. ***Indicates the difference in the mean from 2019 to 2020 is statistically significant (p<0.05).

In 2019, seventy-three percent of the respondents were somewhat satisfied or very satisfied with the quality of academic advising and guidance. In 2020, ninety-three percent of the respondents were somewhat satisfied or very satisfied with the quality of academic advising and guidance.

Table H1.5. Student Satisfaction with Quality of Academic Advising and Guidance 2019 Current Student Survey 2020 Current Student Survey Students Students “Somewhat “Somewhat Total Mean Total Mean Satisfied” / Satisfied” / Respondents Respondents Question “Very Agreement “Very Agreement Rating Rating (N) Satisfied” (N) Satisfied” % (n) % (n) Please rate your level 30 73% (22) 3.00 29 93% (27) 3.48 of satisfaction with the quality of academic advising and guidance. How satisfied are you 29 59% (17) 2.62 30 93% (28) 3.37*** with the advising you've received in regard to emotional support and encouragement? How satisfied are you 29 52% (15) 2.55 30 83% (25) 2.97 with the advising you've received in networking/connectin g with others in your field/discipline?

212

How satisfied are you 29 59% (17) 2.79 30 90% (27) 3.30*** with the advising you've received in identifying strategies for achieving success in your field/discipline? ***Indicates the difference in the mean from 2019 to 2020 is statistically significant (p<0.05).

Advisors set a goal to discuss the student’s academic progress with their advisees at least once a semester. In 2019, 65% of the students formally discussed their academic progress with their advisor at least once a semester, with 10% of the students never contacting their advisor. Dramatic progress was made on this goal in 2020. 90% of the students formally discussed their academic progress with their advisor at least once a semester, with all the survey respondents contacting their academic advisor at least once. Additionally, 72% of the survey respondents on the 2020 current student survey indicated their advisor helped them develop connections to others in public health (i.e., “to some extent” and “to a great extent”) compared to 55% on the 2019 current student survey.

5) Describe the orientation processes. If these differ by degree and/or concentration, provide a brief overview of each.

Orientation for the MPH program is available online through D2L and through a live orientation meeting before classes start. Live orientation sessions were held between 2015 and 2017 with limited online participation, and through establishing the live option again, we hope to encourage more interaction and participation. The agenda for the fall 2020 orientation session can be found in the ERF (H1-7 MPH Orientation Agenda). To engage students upon admission and provide ongoing support, we developed an MPH Orientation and Resource Center via D2L that includes the following: • welcome letter and video from the MPH Director • student handbook • programs of study, course catalogs, and graduation forms • examples of full- and part-time programs of study • information about graduation • information about the APE and ILE • a discussion board to share introductions • a discussion board to post and share questions • student spotlights • professional development opportunities • academic resources

As we prepare for the 2020-21 academic year and new student orientation, we are adding additional content such as faculty spotlights and welcome videos, video orientations to the USD and SDSU libraries, video orientation to the USD and SDSU writing centers, and a frequently asked questions (FAQ) page.

Upon admission, all USD and SDSU students are invited to on-campus orientations hosted by respective Graduate Schools. On the USD campus, orientation sessions provide information about doctoral/master’s programs, graduate and professional student associations, and Graduate School policies. Similarly, all SDSU students are invited to a two-hour session that covers the basics of graduate school, current policies and procedures that impact students, and professional development opportunities. Time is set aside for students to ask questions of the staff and information is made available electronically following the orientation session. Supplemental material, not covered during the orientation session, is offered as part of the Graduate School Professional Development Program.

213

6) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

See response to H1.5.

214

Section H

H2. Career Advising

215

Intentionally blank

Divider back

216

H2. Career Advising

The program provides accessible and supportive career advising services for students. Each student, including those who may be currently employed, has access to qualified faculty and/or staff who are actively engaged, knowledgeable about the workforce and sensitive to his or her professional development needs and can provide appropriate career placement advice. Career advising services may take a variety of forms, including but not limited to individualized consultations, resume workshops, mock interviews, career fairs, professional panels, networking events, employer presentations and online job databases.

The program provides such resources for both currently enrolled students and alumni. The program may accomplish this through a variety of formal or informal mechanisms including connecting graduates with professional associations, making faculty and other alumni available for networking and advice, etc.

1) Describe the program’s career advising and services. If services differ by degree and/or concentration, a brief description should be provided for each. Include an explanation of efforts to tailor services to meet students’ specific needs.

Faculty in roles of academic advisors provide both academic and career advising to MPH students. When possible, students are matched with academic advisors who have shared public health practice and/or research interests. During advising meetings, faculty check-in with students about future plans and provide mentorship and connect students with other professionals and/or encourage participation in professional association and organizations such as the American Public Health Association and South Dakota Public Health Association. In addition, faculty advisors provide information on additional credentials relevant to public health careers such as a registered environmental health specialist (RHES), certified health education specialist (CHES), and others as they relate to the student’s career choice.

2) Explain how individuals providing career advising are selected and oriented to their roles and responsibilities.

Faculty advisors receive a brief orientation on career advising during the academic and career advising orientation (described in H1). Many of our MPH faculty have prior or ongoing public health practice and research experience outside academia and are able to help students navigate career planning and relevant employment or internship opportunities.

3) Provide three examples from the last three years of career advising services provided to students and one example of career advising provided to an alumnus/a. For each category, indicate the number of individuals participating.

First, we have shared opportunities for employment, internships, and career fairs through the program listserv since the inception of our program in spring 2015. To strengthen this process, our program plans to develop a monthly newsletter with upcoming events, reminders, student spotlights, and career/internship/scholarship/professional development opportunities.

Second, we added content to our program website for ‘Careers in Public Health’ and ‘Professional Organizations’ during the spring of 2016, and often refer students to these resources. Our program websites are limited to in scope and content due to university requirements that they be used for marketing and recruitment rather than current students and alumni.

Third, we created a USD-SDSU MPH Orientation and Resource Center via D2L in fall 2018 that serves as a central hub for academic, research, service, and professional development resources. This site serves as a portal for current students, which is something we needed to resolve the university requirements for our program websites.

217

Outreach for alumni on career advising is less structured and has been more 1:1 advisor and alumni mentoring. For example, one faculty reports supporting more than five alumni in visiting about career and employment opportunities post-graduation as well as writing letters of recommendation or serving as a reference for employment. Our faculty also have shared employment opportunities through the South Dakota Public Health Association member network, of which our local students are a part. In addition, we created an alumni listserv during summer 2019 to stay connected with graduates, share opportunities, and send invitations for the alumni survey.

4) Provide data reflecting the level of student satisfaction with career advising during each of the last three years. Include survey response rates, if applicable.

The Exit Survey was launched at the end of the fall 2019 semester. This survey was distributed to students enrolled in the capstone course PUBH 730.

Table H2. Alumni and Student Survey Characteristics Alumni Survey Number of Alumni Number of Alumni 57 Number of Bounce Back 0 Number of Alumni Contacted 57 Number of Alumni Completing the Survey 31 Overall Response Rate (%) 54% Student Survey 2019 Number of Students Number of Current Students 69 Number of Bounce Backs/Filtered 0 Number of Students Contacted 69 Number of Students Completing the Survey 31 Overall Response Rate (%) 45% Student Survey 2020 Number of Students Number of Current Students 61 Number of Bounce Backs/Filtered 0 Number of Students Contacted 61 Number of Students Completing the Survey 32 Response Rate 52% Exit Survey 2020 Number of Graduates Number of Graduates 21 Number of Bounce Backs/Filtered 0 Number of Graduates Contacted 21 Number of Graduates Completing the Survey 10 Overall Response Rate (%) 48%

218

Using a 4-point Likert scale (1=very dissatisfied, 2=dissatisfied, 3=satisfied, 4=very satisfied), alumni were asked to rate their career advising experience while enrolled in the MPH program.

Alumni Level of Satisfaction with the Career Advising Support and Services: • 61% of the survey respondents were satisfied or very satisfied with career advising (average satisfaction 2.6 out of 4). • 77% of the survey respondents were satisfied or very satisfied with the support for career goals (average satisfaction 2.9 out of 4). • 63% of the survey respondents were satisfied or very satisfied with the information received on career options (average satisfaction 2.6 out of 4).

Utilizing a 4-point Likert scale (1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree), alumni were asked to rate the level of support for career and/or job placement received from the university and the MPH program.

Alumni Level of Agreement with the Following Statements Related to Career Advising: • 30% of the survey respondents agreed or strongly agreed that the university provided career and/or job placement support (average agreement 2.1 out of 4). • 33% of the survey respondents agreed or strongly agreed that the MPH program provided career and/or job placement support (average agreement 2.1 out of 4).

Using a 4-point Likert scale (1=very dissatisfied, 2=dissatisfied, 3=satisfied, 4=very satisfied), recent graduates were asked on the exit survey to rate their career advising experience while enrolled in the MPH program.

Recent Graduate’s Level of Satisfaction with the Career Advising Support and Services: • 75% of the survey respondents were satisfied or very satisfied with career advising. (average satisfaction 3.0 out of 4). • 75% of the survey respondents were satisfied or very satisfied with the support for career goals. (average satisfaction 3.0 out of 4). • 75% of the survey respondents were satisfied or very satisfied with the information received on career options. (average satisfaction 2.9 out of 4).

Utilizing a 4-point Likert scale (1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree), recent graduates were asked on the exit survey to rate the level of support for career and/or job placement received from the university and the MPH program.

Recent Graduate’s Level of Agreement with the Following Statements Related to Career Advising: • 78% of the survey respondents agreed or strongly agreed that the university provided career and/or job placement support. (average agreement 2.9 out of 4). • 78% of the survey respondents agreed or strongly agreed that the MPH program provided career and/or job placement support. (average agreement 2.9 out of 4).

Utilizing a 4-point Likert scale (1=very dissatisfied, 2=somewhat dissatisfied, 3=somewhat satisfied, 4=very satisfied), the current MPH students were asked to rate their satisfaction level with the career advising.

2019 Current Student Level of Satisfaction with the Career Advising Support and Services: • 69% of the survey respondents were somewhat satisfied or very satisfied with career advising received in developing important career-related skills (average satisfaction 2.8 out of 4).

219

• 59% of the survey respondents were somewhat satisfied or very satisfied with the career advising received in developing their knowledge about career options/possibilities (average satisfaction 2.6 out of 4). • 47% of the survey respondents were somewhat satisfied or very satisfied with the quality of career advising and guidance (average satisfaction 2.5 out of 4).

2020 Current Student Level of Satisfaction with the Career Advising Support and Services: • 90% of the survey respondents were somewhat satisfied or very satisfied with career advising received in developing important career-related skills (average satisfaction 3.3 out of 4). The difference in the mean from 2019 to 2020 is statistically significant (p<0.05). • 97% of the survey respondents were somewhat satisfied or very satisfied with the career advising received in developing their knowledge about career options/possibilities (average satisfaction 3.3 out of 4). The difference in the mean from 2019 to 2020 is statistically significant (p<0.05). • 87% of the survey respondents were somewhat satisfied or very satisfied with the quality of career advising and guidance (average satisfaction 3.1 out of 4). The difference in the mean from 2019 to 2020 is statistically significant (p<0.05).

5) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

Current MPH students were asked to identify ways to improve the career advising experience. The students indicated desire for more or better career/professional guidance, more structured feedback, and more interest and encouragement would have improved the experience. Students also suggested providing a list of employment opportunities for students to pursue after graduation as well as more discussions on career goals is warranted. We have launched an Orientation and Resource Center via D2L that houses relevant career and internship options. Career and employment opportunities are further promoted through a monthly programmatic email with upcoming events, reminders, and employment, professional development, and service opportunities. With greater numbers of faculty, we have actively worked to decrease the advising ratios for each faculty. Filling the open assistant professor position will continue to address that. We are also in active discussions about ways to improve and standardize advising topics and are planning a series of professional development activities in which students may participate. This is particularly important because only around one-third of alumni said the university or MPH program provided career or job placement support. To assist with this, we have recently built a database to help streamline student tracking and standardizing academic advising, including prompts on career advising. There will also be individualized and group career advising built into the PUBH 729 Leadership and Program Management course.

220

Section H

H3. Student Complaint Procedures

221

Intentionally blank

Divider back

222

H3. Student Complaint Procedures

The program enforces a set of policies and procedures that govern formal student complaints/grievances. Such procedures are clearly articulated and communicated to students. Depending on the nature and level of each complaint, students are encouraged to voice their concerns to program officials or other appropriate personnel. Designated administrators are charged with reviewing and resolving formal complaints. All complaints are processed through appropriate channels.

1) Describe the procedures by which students may communicate any formal complaints and/or grievances to program officials, and about how these procedures are publicized.

The following outlines the procedures by which students may communicate any formal complaints and/or grievances to school or program officials.

Academic Concern and/or Complaint Where minor concerns arise, students are asked to raise these concerns with the instructor or appropriate staff member with the goal of resolving the issue at this level. If the concern is not resolved at this level, oftentimes a conversation with the Program Director ensues. It is recommended the student visit with the appropriate Department Head and Dean as needed. If a complaint cannot be handled through these channels, the students may address the concern/complaint formally through the Academic Affairs Office pursuant to SDBOR Policy 2.9.

SDBOR 2:9 Student Appeals for Academic Affairs guides disputes for academic decisions with a significant degree of finality such as a course grade or removal from a program. SDBOR Policy 2.9: https://www.sdbor.edu/policy/documents/2-9.pdf

SDBOR 2:33 Student Academic Misconduct (i.e. plagiarism or cheating). Faculty suspecting academic misconduct shall begin the resolution process under this policy. Allegations of academic misconduct, both when disposition is achieved and not achieved under SDBOR 2:33, will then move to SDBOR Policy 3:4. Graduate students can expect communication from the Director of Student Rights & Responsibilities regarding the final disposition of allegations of academic misconduct; referring faculty and the Graduate Dean will be notified of disposition as well. SDBOR Policy 2.33: https://www.sdbor.edu/policy/Documents/2016_0428MapPolicy2_33_StudentAcademicDishonest y.pdf

SDBOR 3:4 Student Code of Conduct. Policy obliging students and student organizations to minimum standards of behavior within which, prohibited conduct, process regarding allegations of misconduct, possible outcomes, and appeal rights are treated. Students may be called upon to respond to allegations of misconduct in the form of notification from the Director of Student Rights & Responsibilities (or another designee). Students, too, may initiate a conduct complaint against another student, students, or student group under this policy. SDBOR Policy 3.4: https://www.sdbor.edu/policy/Documents/3-4.pdf

Non-Academic Concern and/or Complaint For concerns or complaints unrelated to academic, the following options exist: • Contact 911 • Contact the University Policy Department • Make an online report • Call a toll-free hotline • Contact the Vice President for Student Affairs Office or the Office of Title IX/EO

SDBOR 1:18 Human Rights Complaint Procedures (https://www.sdbor.edu/policy/documents/1- 18.pdf). This policy governs the process through which human rights complaints such as

223

discrimination occurring in the delivery of or access to education services are to be resolved. Students may initiate a complaint by contacting the Director of Equal Opportunity & Title IX Coordinator. Students may also be called upon to respond to a complaint under this policy. Should the basis of a student’s academic grievance under SDBOR 2:9 suggest sexual harassment or another form of discrimination, the matter will be investigated under this policy prior to any action under SDBOR 2:9.

State Regulatory Information Any person may file a complaint with the Executive Director of the South Dakota Board of Regents to obtain a review and appropriate action on allegations that an institution governed by the Board: • Violated South Dakota consumer protection laws; • Engaged in fraud or false advertising; • Violated South Dakota laws relating to the licensure of postsecondary institutions or programs; • Failed to provide an educational program meeting contemporary standards for content and rigor; • Failed to assign qualified instructors; or • Violated one or more accreditation requirements.

Where the institution has not already considered and acted upon the complaint, the Executive Director will refer the matter to the institutional president for review and action. If the complainant challenges an institutional disposition of the complaint, the Executive Director will provide for an independent review and disposition of the allegations.

Consumer Protection Laws Allegations involving violation of consumer protection laws may also be filed with the Office of Attorney General.

Out-of-State Distance Education Students Pursuant to the United States Department of Education’s Program Integrity Rule, South Dakota State University is required to provide all prospective and current students with the contact information of the state agency or agencies that handle complaints against postsecondary education institutions offering distance learning or correspondence education within that state. For students residing in a SARA state, the complaint must be brought to the institution’s home state SARA portal entity.

Institutional Accreditor Complaint Procedure Institutional accreditation by the Higher Learning Commission (HLC) of the North Central Association (NCA) of Colleges and Schools provides assurance to prospective students, parents, and others that an institution of higher education meets the agency's clearly stated requirements and criteria. Institutional accreditation is the standard against which colleges and universities are evaluated as a whole for the quality of their education activities. The HLC is an independent organization founded in 1895 as one of six regional institutional accreditors in the United States. The University of South Dakota and South Dakota State University are both accredited by HLC.

Information is Publicized The information is publicized in the MPH student handbook and on each of the institutions’ websites.

2) Briefly summarize the steps for how a complaint or grievance filed through official university processes progresses. Include information on all levels of review/appeal.

The MPH student handbook references the SDBOR policy. The language was copied from the SDBOR policy for consistency purposes. Students who have general complaints about any issue may voice their concerns informally to their advisor, the program coordinator, or the program director.

224

South Dakota Board Policy 3.4 (https://www.sdbor.edu/policy/Documents/3-4.pdf) refers to situations involving the Student Conduct Code. South Dakota Board Policy 2.9 (https://www.sdbor.edu/policy/documents/2-9.pdf) academic disputes involving students. Such disputes most commonly arise as a result of student dissatisfaction with assigned grades, but students may also invoke the standards and procedures provided under this policy to challenge academic responses to instances involving alleged student academic misconduct or to challenge other decisions, justified on academic grounds, that affect their participation in or completion of university academic programs. After the investigation, the Office of Student Rights & Responsibilities should provide confirmation to the Graduate School if misconduct occurred or if the charges were not valid.

The evaluation of students involves the exercise of professional judgment informed by prolonged and specialized training in an academic subject matter and by experience in presenting those techniques and knowledge to persons who may be unfamiliar with them. Deference should be given to judgments that reflect the academic standards accepted by the university as appropriate to the discipline involved in the dispute and for instruction in that discipline. No deference should be given to actions that do not embody accepted academic standards, particularly if the motive for such actions is unrelated to academic concerns.

When a complaint presents facts that would suggest that the challenged action stemmed from conduct violating Board Policy No. 1:18 or 1:19, which prohibit sexual harassment and other forms of discrimination, the matter will be referred under Board Policy No. 1:18 to the institutional Title IX/EEO coordinator for investigation and resolution under those policies. No further action will be taken under Board Policy No. 2:9 pending the completion of proceedings under Board Policy No. 1:18.

If the Board Policy No. 1:18 proceedings result in findings that the academic action stemmed from prohibited discrimination, review under Board Policy No. 2:9 will resume to determine what remedial action is proper. If the Board Policy No. 1:18 proceedings do not result in findings that the academic action stemmed from prohibited discrimination, the proceedings under Board Policy No. 2:9 shall be dismissed, unless there are other factors that may have independently been subject to challenge under this policy.

The policy specifies timing and substantive conditions regarding appeals. Academic appeals may be brought only by students who were registered during the term in which the disputed action was taken. Academic appeals may be brought only from final course grades or other actions that have similar finality, such as, without limitation, denial of admission to an undergraduate major or refusal to permit the continuation of an academic program. Academic appeals must be brought within thirty calendar days from the date that the student received notification of the action. If this action occurs within fifteen calendar days before the end of the term, the student must bring an appeal within fifteen calendar days after the beginning of the academic term (fall, spring, or summer) following the term in which the challenged action was taken. A student may petition the president or president’s designee for an extension of this timeline if circumstances prevented a timely appeal.

Academic appeals may be brought to challenge a grade or academic decision typically on one or more of three grounds: (1) if an academic decision resulted from administrative error or from misapprehension of some material fact or circumstance, e.g., evaluation reflected an error in the examination or question itself or misread the student’s written response; (2) if an academic decision departs substantially from accepted academic standards for the discipline and the university; or (3) if circumstances suggest that an academic decision reflected the prejudiced or capricious consideration of student opinions or conduct unrelated to academic standards, of student status protected under Board policy, state or federal civil rights law or of other considerations that are inconsistent with the bona fide exercise of academic judgment.

225

A student who wishes to challenge a grade or academic decision shall discuss the matter first with the course instructor or the person(s) responsible for the decision. The instructor or academic decision-maker(s) shall listen to the student’s concerns, shall provide explanation, and shall change the grade or reconsider the decision if the student provides convincing argument for doing so. If, after the discussion with the instructor or academic decision-maker, the student's concerns remain unresolved, the student may appeal the matter to the appropriate immediate administrative superior. That person, if he or she believes that the complaint may have merit, shall discuss the matter with the instructor or decision-maker. If the matter still remains unresolved, the dispute shall be referred to the president or the president’s designee. The president or the president’s designee shall make a final decision, which may include an administrative change in grade or academic status. If resolution of the dispute requires the resolution of a question involving academic standards, the president or the president’s designee may obtain expert recommendations concerning those standards, whether by convening a panel of faculty or by obtaining recommendations from experts from outside the university.

3) List any formal complaints and/or student grievances submitted in the last three years. Briefly describe the general nature or content of each complaint and the current status or progress toward resolution.

The program received two formal complaints in July 2019. One came as an email to the USD Graduate School and the other was a phone call to the Program Director. Please see the ERF H3- 1 Student Complaint Reports for details of the nature of the complaint, resolution, and status.

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

The MPH program provided a student-centered response to two considerable student concerns the summer of 2019. The program administration team (program director, department head, new program coordinator, and academic advisors) constantly make themselves available for students’ questions and concerns. With recent hires, we hope to gain stability and consistency for student support. Several measures were put into place to prevent this level of student concern again. As stated above, the restructuring of the SDSU College of Pharmacy and Allied Health Professions provided greater oversight and support for the program coordinator. Attention to continuous quality improvement through addressing student issues at all-faculty meetings will help prevent considerable student issues in the future.

226

Section H

H4. Student Recruitment and Admissions

227

Intentionally blank

Divider back

228

H4. Student Recruitment and Admissions

The program implements student recruitment and admissions policies and procedures designed to locate and select qualified individuals capable of taking advantage of the program’s various learning activities, which will enable each of them to develop competence for a career in public health.

1) Describe the program’s recruitment activities. If these differ by degree (e.g., bachelor’s vs. graduate degrees), a description should be provided for each.

To this point, the main mode of recruitment for our MPH program has been word of mouth. There have been some in-person efforts in guest lectures across campuses and at local (e.g., USD graduate school recruitment days) and regional (e.g., SDPHA conference) events. Many of our students find out about the program through electronic methods, through website search engines, and listservs. Hard copies of program flyers are available in MPH office spaces (see ERF H4-1 Master of Public Health Online Brochure; H4-2 MPH Public Health fact sheet; H4-3 SDSU MPH Brochure 2020).

2) Provide a statement of admissions policies and procedures. If these differ by degree (e.g., bachelor’s vs. graduate degrees), a description should be provided for each.

Applicants are asked to fill out an application form and provide official transcripts from previous degrees, two letters of recommendation, and statement of purpose. Additional requirements are in place for international students, and upon admittance, students must have a successful completion of criminal background check. A joint Admissions Committee with representation from USD and SDSU review and score applications. Recommendations are given for admittance for each student and in cases of disagreement among members, further discussion is held to come to a final decision. Action forms for each student are filled out by the committee chair and sent to the respective graduate schools that subsequently send admission/denial letters to each student.

3) Select at least one of the measures that is meaningful to the program and demonstrates its success in enrolling a qualified student body. Provide a target and data from the last three years in the format of Template H4-1. In addition to at least one from the list, the program may add measures that are significant to its own mission and context.

Template H4-1. Outcome Measures for Recruitment and Admissions Outcome Measure Target Year 1 Year 2 Year 3 2017-2018 2018-2019 2019-2020 Percentage of priority under- 40% 37.78% 40.74% 38.46% represented students (as defined in Criterion G1) accepting offers of admission

An under-represented student is defined as a student meeting one or more of the following criteria: • American Indian • First-generation • Lives in a rural community (i.e., the population is less than 2,500)

The first-generation indicator is collected on the common application as well as the current student, exit, and alumni surveys. Students are asked to respond to the following question, “What is the highest level of education attained by any parent?” A student who responds with any response other than “bachelor’s degree or higher” will be considered first-generation. This logic was developed by the South Dakota Board of Regents: https://www.sdbor.edu/administrative- offices/academics/aac/Documents/2017-04-AAC/4_H_AAC0417.pdf.

229

4) If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area.

There are future plans to highlight the MPH program in targeted electronic recruitment efforts. In the past, prospective students who were offered admission were not asked to reply if they accepted admission, which in some cases led to confusion about which students actually matriculated. The 2020 admissions cycle has been moved earlier to provide more time before the fall semester starts, and a reply request of admissions acceptance was included in the welcome letter.

230

Section H

H5. Publication of Educational Offerings

231

Intentionally blank

Divider back

232

H5. Publication of Educational Offerings

Catalogs and bulletins used by the program to describe its educational offerings must be publicly available and must accurately describe its academic calendar, admissions policies, grading policies, academic integrity standards and degree completion requirements. Advertising, promotional materials, recruitment literature and other supporting material, in whatever medium it is presented, must contain accurate information. ` 1) Provide direct links to information and descriptions of all degree programs and concentrations in the unit of accreditation. The information must describe all of the following: academic calendar, admissions policies, grading policies, academic integrity standards and degree completion requirements.

Both SDSU and USD have their educational offerings publicly displayed on their website. All materials (online and print) are reviewed on an annual basis by the MPH program director and program coordinator to align with the changes for the new academic year. The collaborative relationship of the MPH faculty and staff ensures the program information is consistent across all medias. The information below are direct links to the information and descriptions of the degree programs within the unit of accreditation.

SDSU Graduate Catalog 2020-2021: https://catalog.sdstate.edu • Degree Programs: https://catalog.sdstate.edu/preview_program.php?catoid=39&poid=9106&returnto=6226 • Academic Calendar: https://www.sdstate.edu/academics/academic-calendar • Admissions Policies: https://catalog.sdstate.edu/preview_program.php?catoid=39&poid=9106&returnto=6226 • Grading Policies: https://catalog.sdstate.edu/content.php?catoid=39&navoid=6343#grading • Academic Integrity Standards: https://catalog.sdstate.edu/content.php?catoid=39&navoid=6343#integrity_appeals • Degree Completion Requirements: https://catalog.sdstate.edu/preview_program.php?catoid=39&poid=9106&returnto=6226

USD Graduate Catalog 2020-2021: http://catalog.usd.edu/index.php • Degree Programs: http://catalog.usd.edu/preview_program.php?catoid=30&poid=5818&returnto=1727 • Academic Calendar: https://www.usd.edu/registrar/calendars • Admissions Policies: http://catalog.usd.edu/preview_entity.php?catoid=30&ent_oid=1696&returnto=1727 & http://catalog.usd.edu/content.php?catoid=30&navoid=1724 • Grading Policies: http://catalog.usd.edu/content.php?catoid=30&navoid=1724#grading_sys • Academic Integrity Standards: http://catalog.usd.edu/content.php?catoid=30&navoid=1724#Academic_Integrity • Degree Completion Requirements: http://catalog.usd.edu/preview_program.php?catoid=30&poid=5818&returnto=1727

233