Denver Public Schools s6

Denver Public Schools External Research Request Handbook Procedures and Application

Procedures for Requesting Data or Conducting Research in the Denver Public Schools Denver Public Schools Research Review Board

Last Updated: 19 May 2018

of 16 Table of Contents

II. General Policies...... 3

General Guidelines...... 3

Key Departments and Contacts for Specific Data...... 3

HR Data:...... 4

How to Create a Successful Application...... 4

III. Procedures...... 5

Who needs to submit an External Research Application?...... 5

Research Review Board review process...... 5

Submission of application...... 5

IV. Research Request Submission Components...... 5

Application Checklist...... 5

Cover Sheet...... 6

Executive Summary or Abstract...... 6

Full Proposal...... 6

V. Conducting the research...... 7

Research Modification Coversheet...... 8

VI. Dissemination of results...... 8

Appendix A: Research Request Application Checklist...... 9

Appendix B: Request to Conduct Research Coversheet...... 10

Appendix C: DPS Sponsor Statement of Support...... 11

Appendix D: Principal Consent Form...... 12

Appendix E: Time Requirement Form...... 13

Appendix F: Checklist for the Informed Consent...... 14

Appendix G: Research Modification Coversheet...... 15

of 16 I. Introduction

Welcome to the Denver Public Schools (DPS) Research Review Board! Our district is a leading urban district with a wide diversity of schools, staff, and students that warrants a robust research capacity. By collaborating with universities, foundations, the State of Colorado, and the U.S. Department of Education, DPS is poised to discover practices in leadership, teaching, and learning that advance student achievement within DPS, across the state of Colorado, and throughout the Nation. Research, however, is secondary to our goal of educating each and every student we serve. Approval is reserved for proposals that support the ambitious 2020 Denver Plan, avoid duplication of effort, and do not interfere with instruction or the efficient operation of DPS, or its schools.

The aim of the research request protocol is to maintain an open attitude toward cooperative and collaborative research that can improve district practices while ensuring that it will not interfere with classroom instruction and the efficient operation of the school district. DPS also welcomes opportunities to partner with research agencies, universities, educational programs, and other organizations to co-author research publications.

The administration of research programs in the Denver Public Schools is the responsibility of the Research Review Board (RRB). The RRB is facilitated by the Department of Accountability, Research and Evaluation. The responsibilities of the RRB include: (1) review of research proposals, (2) monitoring of projects, (3) coordination of research projects, and (4) dissemination of research findings to staff members.

We welcome your interest in the Denver Public Schools!

II. General Policies

General Guidelines

The guidelines for conducting research projects and pilot programs are as follows:

1. All external data or research requests must be submitted to the RRB for approval within three windows before beginning the research project. Window One occurs March 1st through March 31st. Window Two occurs September 1st through September 30th. Window Three occurs December 15th through January 15th. Requests submitted outside of these windows will not be considered.

2. The study must have the potential for making a definite contribution to the district’s six priority areas listed below. The connection to specific district strategies, including a detailed plan of how the research will be communicated to the relevant district staff, will be an essential component of a successful application. All these areas have student achievement as their overarching goals. Proposed research must reference, evaluate, and directly inform staff in the specific action areas outlined within each priority area. Please click here for more information regarding the DPS Priorities.

1. Great Schools in Every Neighborhood (Quality of schools in every neighborhood; access to great schools) 2. A Foundation for Success in School (Promotion of early literacy and childhood academic successes) 3. Ready for College and Career (Preparing students for college or a career after graduation) 4. Support for the Whole Child (Effectiveness of supports provided for the Whole Child) 5. Close the Opportunity Gap (Promotion of success among underrepresented student groups) 6. Ensure Great Teachers and Leaders (Promotion of teaching and leadership success)

Key Departments and Contacts for Specific Data

Specific types of data are housed within different departments in DPS. You may find that your research will use data from multiple departments. You will need to provide approval from the specific department(s) from which your data will be pulled. Please consult with the listed contact(s) before submitting your proposal. Page 3 of 16 RRB Data Pull Type Department Contact Email Assessment Data ARE Kathryn Johnson [email protected] Parent Satisfaction Survey ARE Kathryn Johnson [email protected] Student Satisfaction Survey ARE Kathryn Johnson [email protected] HR Non-Teacher/Non-Principal Data HR Betsy Conti [email protected] Teacher and School Leader Data HR Tina Scheppe [email protected] Attendance DoTS Doug Ferguson [email protected] Behavior DoTS Doug Ferguson [email protected] Course Grades (with teacher link) DoTS Doug Ferguson [email protected] Student-Related Data (excluding assessment data) DoTS Doug Ferguson [email protected] Enrollments Planning Sara Baris [email protected] Choice Planning Sarah Baris [email protected] CollaboRATE (District employee survey) Planning Elizabeth Stock [email protected] Direct-Certified Planning Elizabeth Stock [email protected] ICAP PSR Danielle Sulick [email protected] Concurrent Enrollment PSR Danielle Sulick [email protected] Remediation PSR Danielle Sulick [email protected]

HR Data: Items HR is unable to share:  Information that is in any way identifiable. This means that demographic data can only be shared if the sample sizes for each sub-group are at least five.  School leader demographic or performance information. Requests related to teacher performance data and teacher performance decisions require an internal sponsor from someone on the Senior Leadership Team.

How to Create a Successful Application

Successful applications will reference DPS documents and highlight how the research will inform these areas. Example documents include the Denver Plan 2020, Board Docs, and the Early Literacy Plan. Some other requirements:

1. The research must be of value to the district, as evidenced through their contributions towards the district priorities. 2. Researchers must secure DPS sponsorship from a director, department chair, or higher-level district administrator and provide evidence of ongoing partnership with DPS as part of the application process. 3. The research must be pedagogically and methodologically sound and have the potential for successful completion. 4. Research must be feasible to carry out in terms of the amount of time involved, the numbers of teachers and/or students participating, and the possible expense to the district. It must not impose undue burdens upon students, staff, or other resources such as DPS funds or volunteers, or DPS staff time. Any required time for DPS to pull data (exceeding 2 hours) and follow up with researchers on requests will be billed at $35 per hour. 5. There must be a reasonable certainty that no child in the Denver Public Schools will suffer educationally or otherwise as a result of the research. 6. The study must protect the privacy of students and staff and ensure compliance with state and federal law. The Federal Educational Rights and Privacy Act (FERPA) is based on the idea that information about an individual student is private and confidential and generally may only be accessed with parent consent.

The policies and guidelines are not intended to discourage the development of creative projects, but are set forth to ensure the greatest benefit from the time and resources expended.

of 16 III. Procedures

Who needs to submit an External Research Application?

 Individuals, organizations, or agencies outside of DPS who wish to conduct independent research.  Companies or entities that wish to conduct product research to study a product’s effectiveness for potential sale to the DPS.  Individuals or groups that have established programs within DPS (e.g., grantees, foundations, service providers) who wish to evaluate their programs.  DPS employees who wish to conduct research for personal purposes outside of their work duties and/or work hours (e.g., research for the completion of a master’s thesis or doctoral dissertation). Employee theses or dissertations will only be considered if the topic aligns with district priorities.  Agencies that have established a formally approved research collaborative relationship with DPS will need to follow the Handbook procedures to submit a request.

Research Review Board review process

1. Applicants must submit a complete application between three windows: March 1st through March 31st, September 1st through September 30th, and December 15th through January 15th. Applications received outside of these dates will not be reviewed. Any incomplete applications will not be considered, and can be resubmitted during the next window. Requests requiring a data pull may take significantly longer. 2. If the application represents a recurring request, we will need to see evidence of how the research is benefiting DPS before sharing additional data. 3. Applicants will receive a confirmation email once their application is received, and an RRB review member contact you with follow-up questions about your proposal. 4. Applicants will be notified of the RRB decisions within one month. Potential decisions include: a. Approval of the request. b. Rejection of the request. DPS reserves the right to reject applications that are incomplete, inappropriate, or do not align with the district’s interests. 5. If a researcher chooses to re-submit an application that had previously been rejected, the new application will be treated as such, with no ties to the previous application.

(Note that health-related studies are reviewed quarterly on the following days: January 15, April 15, July 15, and October 15. These requests are not reviewed by ARE, and are submitted to Scott Romero at [email protected].)

Submission of application

 Submit electronic copies of all necessary documents (as described in Section IV Research Request Submission Components) to [email protected].  Submit application for health-related studies to Scott Romero at [email protected]. They may be subject to additional requirements as determined by the reviewer.

IV. Research Request Submission Components

Application Checklist

Complete the Application Checklist (see Appendix A).

Cover Sheet

Complete the Request to Conduct Research Coversheet (see Appendix B).

of 16 Executive Summary or Abstract

In no more than 500 words (one single-spaced page), provide a synopsis of the proposed research including purpose, participants (e.g., students, teachers, etc.), and length of time data collection is to take place. This statement should be written such that persons unfamiliar with the research will be able to understand, in a short time, what the researcher is proposing.

IRB Approval Documentation

If you are affiliated with a university or other research institution, you are required to provide documentation of IRB approval from your home institution before we can approve the start of any research within DPS. The RRB process can be initiated and completed while you are awaiting IRB approval, but we cannot allow research to start without IRB approval.

Full Proposal

The proposal must contain the following elements, when applicable, in the order outlined below:

1. Project title 2. Researcher status and title: Statement naming Primary Investigator, title and status within institution or organization (e.g., student, professor, director, etc.). DPS employees must disclose DPS employment status and position in this section. 3. Statement of purpose, research problem, and alignment to DPS area(s) of research interest. This should include a detailed proposal of how findings will inform DPS priorities and a timeline of how the researcher will collaborate with DPS staff to ensure that the findings inform DPS decisions. This timeline should be corroborated by the sponsor’s statement. Specific meetings to update DPS and review findings, along with a list of staff that will participate, should be included with the proposal. 4. Research question(s) and hypotheses. 5. Benefit/cost to Denver Public Schools and/or the profession of education. Benefits include any positive changes or discovery of knowledge that could improve the function of the district, classroom instruction, and/or administrative procedures Costs include any financial or time commitments that are incurred as a result of research participation. Researcher must also obtain a DPS Sponsor’s Statement of Support (see Appendix C) as evidence of the proposed research’s value to the district. a. Research within one school only: A sponsor may be a school principal if the research is bound within the context of one school. The Principal Consent (see Appendix D) is sufficient and no additional sponsor is required. b. Research within more than one school: If the research involves data from more than one school, the sponsor must be a director, department chair, or a higher-level district administrator. In addition, if the research involves access to school buildings for data collection, each school’s principal must give consent (Principal Consent; see Appendix D). For any request that takes place in more than one school, the sponsor will need to write out a detailed summary of how they plan to use the results of the research, and a timeline for ongoing partnership with the researcher, as well as a date(s) to review ongoing and final results. Sponsors should expect to be available for a call from the review committee in the weeks after the deadline to explain how the research will benefit their work. c. Student thesis/dissertation amendment: While a director, department chair, or higher-level district administrator is the preferred DPS sponsor, students who are requesting to conduct research for a thesis or dissertation may use a manager within DPS as their sponsor. However, this option is only available for students who a) do not plan to publish their thesis or dissertation, and b) who do not plan to present their results at a professional setting, such as a conference. Note that you will still need sponsorship from a director, department chair, or higher-level district administrator if you plan to publish your thesis/dissertation (including in online thesis/dissertation databases) and/or present your research in a professional setting. 6. Methods

of 16 a. Sample: sample population, number of students, grade levels, procedure for selecting the sample. b. Data collection methods: interviews, surveys, test scores, video/audio taping, etc. Describe what the participants will be asked to do, what secondary data is needed, etc. i. Secondary data: Please state specific data variables (e.g., specific assessments, attendance, enrollment) and specific years pertaining to the data variables. 1. For example, if you would like CMAS/PARCC data. We would need to know: Year = 2015-2016; Content Areas: Literacy, Math; student ID, demographics (which ones); school (which ones); student vs. school aggregate c. Time required of participants. If primary data is collected, complete the Time Requirement form (see Appendix E). d. Instruments: interview protocol, questionnaire, etc. Copies of all instruments used must be included with your application. e. The methods to be used for analyzing or evaluating the results (e.g., analysis software, statistical strategies, etc.). f. Research timeline: must include tentative start and end date of all research activities, data collection, and reporting of results. Please note: input from the DPS Sponsor is needed regarding dissemination of findings. 7. Human subject protection a. Procedures to ensure participant confidentiality and privacy i. You should outline a detailed plan about how you intend to keep data confidential and secured during the course of your research. ii. You should also discuss how you will destroy your data following the conclusion of your research project. b. Benefit, compensation, and risk to participants c. If the study requires direct contact with participants or the collection of information that may lead to the identification of participants, attach active consent letters or parental/guardian consent letters in the cases where the participants are minors. Refer to Checklist for the Informed Consent for letter content requirements (see Appendix F). If there is the possibility that parents and/or students speak a language other than English, forms in appropriate alternative languages must be provided. d. Parental Consent is required for all research that collects primary data from students. Parental consent may not be necessary for research where the teacher is the object of study and no identifiable information on students is being collected (e.g., a study on teacher effectiveness where lessons are videotaped and students are present but not explicitly identified). However, it is preferred that parents are given a letter detailing the research. Please contact our office if you have any questions. e. Informed Consent: Include an informed consent if data is being collected from participants or if personally, identifiable data is being requested about participants. 8. If applicable, briefly describe how the project is funded. 9. Describe plans for the dissemination of research findings. Be sure to include plans to disseminate results to DPS. 10. Principal Consent Form: If the study involves collecting primary data in school sites, the Principal Consent Form must be obtained for each of the school sites used (see Appendix D).

V. Conducting the research

After the proposal has been approved, the researcher must comply with the following:

1. The procedure must be carried out according to the original proposal, as approved. Researcher must sign the approval letter and fax/email to the RRB following the instructions on the letter. 2. For secondary data requests, a DPS analyst will contact the researcher upon receiving the signed approval letter with an estimated total cost for pulling data, if required. Payment must be received before data is released to the researcher. Secondary data will be released to the researcher by the agreed due date.

of 16 Research Modification Coversheet

1. Any major deviation from the approved proposal must have prior clearance. The applicant must complete the Research Modification Coversheet (see Appendix G) and attach a brief description and rationale for the proposed changes. a. Deviations include, but not limited to: inclusion of additional years in the study, pulling additional variables, or adding more schools. b. Large changes to research procedures, instruments, or otherwise key components may require modification through your institution’s IRB. Please check with your IRB to make sure you are following proper ethical research protocols. 2. Studies are typically approved for a period of one year, though some exceptions for longer-term studies may be considered. Extensions to the study must be requested using the Research Modification Coversheet (see Appendix G). 3. Along with the Modification and Continuing Review Process Coversheet, researchers must obtain an updated DPS Sponsor’s Statement of Support outlining and agreeing to the modified changes.

VI. Dissemination of results

Once the study has concluded, the researcher must submit a report of the findings to the DPS RRB as specified in the research timeline prior to publication or distribution of findings to other parties. If co-authorship with DPS is established, DPS should be actively involved in the writing of the report or paper. Please send results to the [email protected] and to the DPS Sponsor.

If the researcher does not follow through with disseminating the results to DPS as outlined in the application, this will impact the ability of future researchers from the same institution to receive approval. If this is part of a recurring request, subsequent data will not be pulled and the researcher will need to submit a new request.

DPS reserves the right to provide comments or suggestions that the author will consider prior to dissemination and/or publication of findings. Further, DPS reserves the right to restrict dissemination and/or publication if participant, school and/or district anonymity is compromised.

of 16 Appendix A: Research Request Application Checklist

Prior to submitting your research request, please make sure all of the items listed below are completed (if applicable). Please submit completed checklist along with all other materials.

Completed Research Request Documentation Comments Documentation Y N Cover Sheet ☐ ☐ Executive Summary (500 Words or Less) ☐ ☐ Project Title ☐ ☐ Researcher Status and Title ☐ ☐ Statement of Purpose ☐ ☐ Research Problem ☐ ☐ Alignment to DPS areas of research interest ☐ ☐ Research Question ☐ ☐ Hypothesis ☐ ☐ Benefit/Cost to DPS and/or profession of education ☐ ☐ DPS Sponsor statement of support ☐ ☐ IRB Approval Documentation (if applicable) ☐ ☐ Method Sample (e.g., # of Students, Grade, Procedure for selecting ☐ ☐ the sample) Data Collection Methods ☐ ☐ Time required of Participants (if primary data are collected) ☐ ☐ Instruments ☐ ☐ Method for analyzing results ☐ ☐ Research Timeline ☐ ☐ Human Subject Protection Procedures to ensure confidentiality ☐ ☐ Benefit, compensation & risk to participants ☐ ☐ Consent Letters (if study requires direct contact) ☐ ☐ Informed Consent (if applicable) ☐ ☐ Principal Consent Form (if study requires direct contact) ☐ ☐ Dissemination and Funding Project Funding Description (if applicable) ☐ ☐ Dissemination Plan ☐ ☐

of 16 Appendix B: Request to Conduct Research Coversheet Request to Conduct Research Coversheet Denver Public Schools Research Review Board

I. Background Date______

Title of the Study: ______

Name of Researcher: ______Phone: ______

Institution of Researcher: ______

DPS Employee: YES NO If YES, School or Department______

Street address: ______City: ______State: ___ Zip: ______

E-mail: ______

DPS Sponsor of the Study:______Department: ______

Indicate what type of data the researcher will collect:

___ Primary Data¹ __ Secondary Data² __ Both

Indicate which of the following individuals will participate in the study (Mark all that apply):

___ Students __ Teachers __ Other: ______

DPS Priority Area that aligns with the study (Mark all that apply): ___ Great Schools in Every Neighborhood ___ A Foundation for Success in Schools ___ Ready for College and Career ___ Support for the Whole Child ___ Close the Opportunity Gap ___ Ensure Great Teachers and Leaders

Is the researcher willing to include DPS as co-author in publications that might results from this study? YES NO

Approximate date on which results will be submitted to DPS ______

II. Application Checklist

Please complete the Research Request Application Checklist. All documentation must be completed prior to submitting for external review.

III. Agreement

If granted permission to conduct this research in the Denver Public Schools, I agree to follow the guidelines for conducting research as described in the External Research Request Handbook.

______Signature of Researcher Signature of Research Supervisor (if applicable)

Submit all materials to: [email protected]; submit health-related studies to : [email protected] ¹ Primary data- data that has not already been collected; the researcher will actively participate in the data collection process ² Secondary data- data that DPS maintains; the researcher has no involvement in the data collection process Page 10 of 16 Appendix C: DPS Sponsor Statement of Support DPS Sponsor Statement of Support Denver Public Schools Research Review Board

I. Research Background (to be completed by researcher)

Title of the Study ______

Name of Researcher______Phone ______

E-mail: ______

II. Description of Research Proposal

Researcher is to provide the sponsor with a copy of the executive summary and the time requirement form.

III. Description of Alignment to DPS Priorities

Please summarize: 1) the importance of the proposed research project to you and DPS, 2) how you will use the data to inform your work and others, and 3) how the research directly aligns to one of the DPS priority (approximately 250 words).

IV. Timeline for Collaboration

Frequency of check-ins with researcher: Please indicate how often you and the researcher will meet to ensure continued alignment to the proposed research. If this alignment is no longer present or the researcher does not collaborate, please notify ARE at [email protected]

Dissemination of findings: Please indicate when the researcher will be presenting initial and final results to your team to inform their work.

V. Agreement (to be completed by the sponsor)

I, ______(name), ______(title) of ______

______(department), support the proposed study, with the understanding that

 the privacy and confidentiality of any staff or student will be protected,  I have the right to terminate the research study at any time,  I have the right to review all consent forms and research documents at any time during the study,  findings from this study will be distributed to me.  I have reviewed the executive summary and the time requirement form, if applicable, of the above named research.  I find the above named research valuable; its findings will be used to inform the work of my department/team.  I understand that data should be released only by the departments that own them. My staff and I shall not release data to the researcher without approval from the RRB.

______Signature of DPS Sponsor

of 16 Appendix D: Principal Consent Form Principal Consent Form Denver Public Schools Research Review Board

I. Research Background (to be completed by researcher)

Title of the Study ______

Name of Researcher______Phone ______

E-mail: ______

II. Description of Research Proposal Researcher is to provide the principal with a copy of the executive summary and the time requirement form.

III. Timeline for Collaboration Frequency of check-ins with researcher: Please indicate how often you and the researcher will meet to ensure continued alignment to the proposed research. If this alignment is no longer present or the researcher does not collaborate, please notify ARE at [email protected]

Dissemination of findings: Please indicate when the researcher will be presenting initial and final results to your team to inform their work.

III. Agreement (to be completed by principal)

I, ______, principal of ______school, understand

 the study and what it requires of the staff, students, and/or parents in my school,  the privacy and confidentiality of any staff or student will be protected,  I have the right to allow or reject this research study to take place at my school,  I have the right to terminate the research study at any time,  I have the right to review all consent forms and research documents at any time during the study and up to three years after the completion of the study.

 I have reviewed the executive summary and the time requirement form, if applicable, of the above named research.

 I find the above named research valuable; its findings will be used to inform the work of my department/team.

 I understand that data should be released only by the departments that own them. My staff and I shall not release data to the researcher without approval from the RRB.

 I grant permission to the researcher to conduct the above named research in my school as described in the proposal.

 I DO NOT grant permission to the researcher to conduct the above named research in my school as described in the proposal.

______Signature of Principal

of 16 Appendix E: Time Requirement Form For Primary Data Collection Only Time Requirement Form Denver Public Schools Research Review Board

Complete the following chart for all participant categories. The first row provides an example.

Participant Category Number of participants Activity Total time required for each participant

Students 20 Fill out online survey 15 minutes

of 16 Appendix F: Checklist for the Informed Consent Checklist for the Informed Consent Denver Public Schools Research Review Board

All external research surveys, assessments, analyses or evaluations require the prior written and active informed consent of a parent or legal guardian on behalf of a minor student before data collection commences.

Regardless of the sponsoring university or agency’s requirements, parent consent form must be written in easily understood language. If needed, the informed consent form should be translated into students’ home language and submitted with the research request application.

The consent form must provide the following information:  Title of the study and identification of the researcher(s) conducting the study (e.g., doctoral candidate at University of Colorado)  The purpose(s) for collecting data (e.g., to learn about foreign language acquisition in primary school students)  Description of what participants will be asked to do (e.g., complete a written survey, respond to a group interview with eight other students)  Amount of time required of participants  Notice that participants will be audio- or videotaped (if applicable) with an explanation of how the recordings will be used and what happens to the tapes after the research is completed  Description of any individually identifiable student data to which the researcher seeks access (e.g., test scores, attendance records, address and phone number) and how and when data will be destroyed. There must also be a separate yes/no check box referring to the release of student level data  Notice that all information will remain confidential  Notice that participation is entirely voluntary and participants may withdraw from the study at any time, without negative consequences  Notice of any reasonably foreseeable risks or benefits to the participant  A local or toll free telephone number of researcher(s), so that the parent or participant may call if there are questions or concerns  Space and lines for the student’s name and parent/guardian signature. There must be a check-off for either consent or refusal to participate and the parent signature must refer to “Yes, I agree to have my child participate” or “No, I do not give consent for my child to participate”. Active consent, i.e., signature approving participation, is required under the federal Protection of Human Subjects regulations. Passive consent, failure to object after notice, is not sufficient  The consent form must not be longer than one page  In addition, the Protection of Pupil Rights Amendment (PPRA) passed under the No Child Left Behind Legislation requires that parents/guardians have access to the materials that will be used in school with their children. Therefore, all consent forms should include the following text:

Parents please be aware that under the Protection of Pupil Rights Act, you have the right to review a copy of the questions asked of or materials that will be used with your students. If you would like to do so, you should contact [INSERT RESEARCHER NAME] at (XXX) XXX-XXXX to obtain a copy of the questions or materials.

of 16 Appendix G: Research Modification Coversheet

Research Modification Coversheet Denver Public Schools Research Review Board

Date______I. Background

Request ID: ______

Title of the Study: ______

Name of Researcher: ______Phone: ______

Institution of Researcher: ______

Are you a DPS Employee? YES NO If YES, School or Department ______

Street address: ______City: ______State: ______Zip: ______

E-mail: ______

DPS Sponsor of the Study______

II. Indicate purpose of the submission (select all that apply):

____ Modification

Select all that apply and submit revised proposal.

Statement of purpose, research problem, or justification of research Research questions or hypotheses Benefit/ cost to DPS schools and education Methodology Human subject protection

Provide a brief explanation of modification:

____ Extension to research timeline

Anticipated end date: ______

Submit all materials to: [email protected]; submit health-related studies to: [email protected]

of 16 Page 16 of 16