Keystone Precodes PowerPoint Transcript

Slide 2 This presentation will be recorded and posted on the Pennsylvania Accountability System (PAS) webpage. There will be one live Q&A session webinar. PDE will use PennLinks and email broadcasts to inform LEAs of the live webinar. The Presentation: . will be recorded and posted; . can be downloaded from www.education.pa.gov/pas under the 2016-17 ‘Trainings’ subheading; and . There will be no Q & A session as this is similar to the presentation given on September 29, 2016, Winter Keystone Precodes.

Slide 3 Before we begin with the agenda, I wanted to take a moment to tell you a little about us. The division of Performance Analysis and Reporting (PAR) is part of the Bureau of Curriculum, Assessment and Instruction. PAR is responsible for collecting and reporting assessment data for the PSSA and Keystone Exams. All the PIMS trainings listed in the 2016-17 PIMS calendar will be conducted by the division of Performance Analysis and Reporting (PAR). Today’s agenda will cover the timelines, due dates and templates needed to complete this internal snapshot. Then we’ll talk about the Business Rules, the Data Element details and finally the reports.

Slide 4: Who submits data to PIMS? Any entity that teaches a Keystone-related course, must assess the student in Algebra, Biology and/or Literature Keystone Exams as end-of-course exam(s). Keystone courses are designated in PIMS by the LEA as ‘trigger’ courses. Designating a course as an end-of-course Keystone course is a local decision and not mandated by PDE. Any entity providing educational services is an “Educating Entity”. Educating Entities include SDs, CS, IUs, CTCs, PRRI, APS and SJCI. Educational Entities are responsible for ensuring that all students enrolled in a Keystone course for whom the entity is providing educational services are assessed. Student demographic information is required for students testing online OR testing paper/pencil. To prevent Educating Entities from having to provide student demographic information by hand bubbling information on answer booklets or submitting it one student at a time for online testers, Educating Entities can submit this data via PIMS to allow for the production of precode labels or test set up sessions. By uploading into the PIMS files, Educating Entities will be spared of extensive work during the administration of the assessment.

Slide 5: Timeline: Lessons learned from last year that may help EEs this year: 1. Please work with your SIS vendor to make sure they can deliver the data required in this collection. 2. Although it is not required, we recommend using the sandbox as a tool to test data against the DQE rules. 3. PIMS Administrators who waited until the last day to upload data and encountered problems were not able to meet the deadline. Since there are no extensions, data was unable to be submitted for the upload, which means the Educating Entity did not get precode labels and their booklets had to be hand-bubbled and test set up sessions had to be entered one student at a time. 4. We strongly encourage PIMS Administrators to update data in the sandbox ahead of time, but want to be clear that you must LOAD the data into PIMS Production, in order for it to be available in the snapshot. The deadline for submitting data for this collection is 12:00 p.m. (noon) on Jan. 25, 2017. After 12:00 noon on Jan. 25, 2017, PIMS will shut down and no data can be uploaded. There are Pre-snapshot Verification Reports that will be available in the sandbox and Cognos prior to noon on Jan. 25, 2017.

Slide 6: In order to submit the correct data, PIMS Administrators must first identify the students who are taking the Winter Keystones and upload them for this internal snapshot. These students should be coded with a valid value other than ‘Z’ in field 214 in order for the student record to be sent in the file to DRC for generating precode labels and test set up sessions. There are three templates (Student, School Enrollment and Programs Fact) that need to be updated and uploaded in Collection Window 6 to make sure that the data passes the Data Quality Engine Rules. Also, it’s important that PIMS Administrators verify that data has been successfully uploaded by running verification reports. PIMS Administrators are also encouraged to run the Preliminary Accuracy Certification Statement (Pre-ACS) because the data that is in that report is the data that PIMS shows as uploaded for the EE. The deadline is reiterated here. Final data must be uploaded by noon on Jan. 25, 2017. Please note that data cannot be corrected after the internal snapshot deadline. Slide 7: This is the Assessment and Accountability Data Flow. This is a complex process and we will discuss it in smaller sections. It is beneficial to understand this complex process because it shows how the assessment data that begins with the EEs submitting data to PIMS ends with accountability reporting. Numbers 1-6 describe the Precodes and Testing Process, while numbers 7-10 deal with accountability.

Slide 8: We begin with the Educating Entity submitting data to PIMS using the Student, School Enrollment and Programs Fact templates as you see in #1. Then in #2, PIMS takes an internal snapshot of the statewide data set and sends a file to DRC. This file is uploaded into DRC’s eDIRECT system.

Slide 9: Once the data is loaded into DRC’s eDIRECT system, this next section describes the actions that occur between the Educating Entity and DRC. Although you as the PIMS Administrator are not directly involved with this process, it can have disastrous results if you (PIMS Administrator) have not loaded student data into PIMS correctly. In #3, someone from your Educational Entity who is involved with assessments, generally your District Assessment Coordinator or DAC, will access DRC’s Enrollment System to indicate the number of test booklets needed for paper/pencil testers and the number of students who are testing online. Students who are taking the paper/pencil assessment will need precode labels for their tests. Student information contained on the precode label comes from the data you (the PIMS Administrator) provide to PIMS. #4 deals with online testers. Students taking the test online must be set up for a test session. Student demographic is required in order for your Assessment Coordinator to set up test sessions. Again, this information is taken from the PIMS information that you as the PIMS Administrator provide to PIMS. If that information has not been submitted to PIMS, your Assessment Coordinator is inputting this one student at a time. Special Note: Your Assessment Coordinator is responsible for setting up the test sessions. This does not happen automatically. Shifting gears and moving to paper/pencil testers, we move to #5 where DRC sends booklets and precode labels to the Educating Entity. Next the test is administered and the Educating Entity returns the test materials to DRC as shown in #6. DRC then scores the assessments, which then moves us into the next section of this complex process of the Accountability part of this Data Flow Chart.

Slide 10: Accountability begins after the booklets are scored. The students who tested are loaded into DRC’s eDIRECT system and matched to the Educating Entity’s PIMS ACCOUNTABILITY data. Because the focus of this webinar is PIMS precode data, we will present the last steps of this flow chart in the webinar dedicated to Accountability. That webinar will take place in March 2017. This concludes the explanation of the Assessment and Accountability Data Flow chart for this webinar. For this presentation, the PSSA Precodes, slides 8 and 9 are relevant.

Slide 11: We will now move on to the Business Rules related to the Keystone Precodes.

Slide 12: We just talked about the Accountability file and this presentation is about the precode file, but the two files are eventually matched, so it’s important that the precode upload passes the DQE Rules. Records may be rejected because they do not meet the current business rules. Please also refer to slide 16 for more criteria that are used to determine if a student record is included in the DRC file. 1. A student reported as enrolled in a Keystone course. (For example, a student enrolled in Algebra 1, Biology and/or Literature must take that subject-specific Keystone Exam.) 2. Number 2: A Student record must include the student’s first name, last name, birthdate and PAsecureID. These four criteria are especially essential to the matching of the Accountability file. 3. The third Business Rule: If there are any special characters other than dashes and apostrophes, it will cause a mismatch in the DRC system and the Educating Entity will have to manually match the student. A word of caution when looking forward to the Accountability file, if the student does not appear in your PIMS accountability file, but does appear in the DRC system as having taken the test, the student cannot be matched. This is because all test records must match the PIMS accountability file and no students can be added to the Attribution system. 4. Rule 4: The student is school aged. That means the student should be within the age range of 6 and 21. 5. Business Rule #5 is new. The student’s school/district/state entry dates are correct and valid. I want to emphasize the reason for Business Rule #5. Last year, we had EEs that entered school/district/state entry dates that were in the future, which caused all those students to not be included in the accountability file later in the SY. This caused the student to have a test record, but could not be matched to a PIMS record. To prevent that from happening again, PDE has now made this a DQE rule.

Slide 13: Business Rule #6 is covered in several slides and discusses how records are unduplicated and the EE that will receive the precode label. Business Rule #6, parts ‘a’ and ‘b’, primarily address students in part time and full time CTCs. We begin with 6a, which states that if a student is reported by an Occupational CTC (part time) and one or more EE of any type, then the record submitted by the ‘Other’ EE will receive the precode label because it is the Other educational entity that is providing the educational services for the subjects assessed. 6b says that if a student is reported by a Comprehensive CTC (full time) and a school district/charter school, then the Comprehensive CTC will receive the precode label because it’s the Comprehensive CTC that is providing the educational services for the subjects assessed.

Slide 14: Continuing with Business Rule 6 for unduplicating student records… 6c states that if a student is reported by an IU and one or more EE, then the record submitted by the IU will be used for precode labels and setting up online test sessions. 6d states if the student is reported by a CS and one or more school district, then the charter school receives the precode label and setting up online test sessions. 6e says that is a student is reported by an APS and one or more school district/CS/CTC, then the APS will receive the labels and setting up online test sessions.

Slide 15: Continuing with Business Rule 6 for unduplicating student records… 6f explains that if a student is reported by a PRRI and one or more EE, then the PRRI will receive the record for precode labels and online test set up sessions. 6g is if multiple SDs submit a record for the same student, then the district with the most recent DISTRICT ENTRY DATE will be used for precode labels and online test set up sessions. 6h is if multiple charter schools submit a record for the same student, then the CS with the most recent ENTRY DATE will be used for precodes labels and online test set up sessions. This completes the 6 Business Rules for the Precode internal snapshots.

Slide 16: In addition to the Current Business Rules in slides 12-15, let’s discuss how data is either included or excluded in the DRC precode file based on the data that the PIMS Administrator submits to PIMS. This flow chart asks a series of questions that show you what data is essential in the PIMS upload if a student record is to be included in the DRC file. We begin with the Educating Entity at the top left of the slide and ask the question: 1. Did the EE upload data to PIMS using the Student, School Enrollment and Programs Fact templates? If No, then the student is not included in the precode file that will be sent to DRC, which means the student will not receive a precode label, the booklet will have to hand bubbled and online test set up sessions will have to be loaded one student at a time. If the answer is ‘Yes’ that the EE uploaded the data to PIMS then the next Q is: 2. Did the data pass the DQE Rules? If No, then the EE will have to upload the data again into PIMS. If Yes, then the next Q is: 3. Did the School Enrollment Template include the student? If No, then the student is not included in the precode file that will be sent to DRC, which means the student will not receive a precode label, the booklet will have to hand bubbled and online test set up sessions will have to be loaded one student at a time. If Yes, then we move to the next Q: 4. Are the School/District/State Entry dates within the valid date ranges? If No, then the student is not included in the precode file that will be sent to DRC, which means the student will not receive a precode label, the booklet will have to hand bubbled and online test set up sessions will have to be loaded one student at a time. If Yes, then we move to the next Q: 5. Is field 214 in the Student template coded with a valid value other than ‘Z’? If No, then the student is not included in the precode file that will be sent to DRC, which means the student will not receive a precode label, the booklet will have to hand bubbled and online test set up sessions will have to be loaded one student at a time. Please note that if the PIMS Administrators enters code ‘Z’ meaning ‘Not participating in any assessment’, then the answer to this question is NO and precodes will not be generated. If Yes, then we move onto the next Q: 6. Is the student coded with Location Code 9999? If Yes, then the student is not included in the precode file that will be sent to DRC, which means the student will not receive a precode label, the booklet will have to hand bubbled and online test set up sessions will have to be loaded one student at a time. If No, then the next Q that needs to be answered is: 7. Is the student enrolled in an Occupational CTC? If Yes, then the Student will be reported by the ‘Other’ EE. ‘Other’ is defined as an EE of any other type. If No, then we move to the next Q: 8. Is the student at a Comprehensive CTC, IU, PRRI, APS, Location Code 0000, charter school or school district? If Yes, then the Student will be reported at the EE that is uploading the data, which means that the student will be included in the precode file that will be sent to DRC, a precode will be generated and/or an online test session will not have to be uploaded one student at a time. If No, then we move to the next Q. 9. Does the student’s District Code of Enrollment match the code for the Educating Entity that is uploading the data? If No, then the student is not included in the precode file that will be sent to DRC, which means the student will not receive a precode label, the booklet will have to hand bubbled and online test set up sessions will have to be loaded one student at a time. If Yes, then the Student will be reported at the EE that is uploading the data, which means that the student will be included in the precode file that will be sent to DRC, a precode will be generated and/or an online test session will not have to be set up one student at a time. Note: The reporting Educating Entity can list an entity other than itself as the educating Entity if there is a special education referral, for example, but it should amount to a small subgroup of your students. If all your students are uploaded to be placed outside the reporting Educational Entity, then the student is not included in the precode file that will be sent to DRC. If you are the Educating Entity and you want to receive precode labels, you must upload the student with your District Code of Enrollment that designates you as the Educating Entity. This is extremely important . Last year, we had an instance where an entire EE’s data was not included in the DRC file because that entity used the District Code of Enrollment code as being another Educating Entity, which means the student records could not be matched to the reporting EE’s PIMS Accountability file.

Slide 17: This is a quick overview of an internal snapshot: 1. Educating Entities upload templates to ensure that their student level data passes the DQE Rules. 2. Next, Educating Entities use presnapshot reports and run the preliminary ACS before PIMS locks down after 12:00 p.m. noon on Jan. 25, 2017. 3. After that time, PIMS will lock down and an internal snapshot will be taken. It’s important to note that after the deadline when PIMS locks down, no data can be submitted or changed.

Slide 18: Once the snapshot is taken, a file is prepared for DRC to generate precode labels. Since this is an internal snapshot, there is no corrections window that would normally follow a Data Collection Window. An internal snapshot freezes data at a point in time and that’s why PDE cannot give any extensions. If data is uploaded AFTER the internal snapshot, it will NOT be included in the file to DRC and precodes will not be generated, which means booklets will have to hand bubbled and online test sessions will have to uploaded manually, one student at a time.

Slide 19: Now we can look at the details of the data elements that are important in this internal snapshot.

Slide 20: It begins with the student represented here by the blue circle, which includes the student’s four matching criteria and surrounding the student are the components that make up the student: the Student Demographics, Program participation and School Enrollment as reported in PIMS by the PIMS Administrator.

Slide 21: It is very important that the four matching criteria: First Name, Last Name, Birthdate and PAsecureID, are correct because these basic data fields identify the student and this is how the student is initially linked to test results later in the year. This is why there is so much emphasis on getting these fields right. Most of these fields are now part of the DQE Rules when you submit data to PIMS. This year the DQE will check last name, birthdate and PAsecureID as well as the other DQE rules. Please refer to Current Business Rules on slide 12 to make sure that the student’s four matching criteria meet the requirements. Sometimes, there are students with similar names like Jan, Jane and Janet as the first name with the same last name. Sometimes, names are abbreviated where Robert could be listed as Bob or Rob or a student’s name could be hyphenated in the PIMS file, the EE’s Student Information System and/or the PAsecureID system. DRC matches the first two characters in the first name and all characters in the last name in the initial matching process. A student’s birthdate must indicate that the student is school aged, which is ages 6-21. Sometimes there are instances where there is one PAsecureID for multiple students or multiple PAsecureIDs for one student. Please go into the PAsecureID system and make sure the data for the student uniquely identifies that student. For the precodes, if data is incorrectly printed on the precode label, the booklet will have to be hand bubbled or the online test session will have to be set up manually, one student at a time. If the erroneous precode label is used on the answer booklet and even if the EE corrected the student data reported in the accountability upload later in the SY, the student test record will not match. This means that the student record will be placed in the EE’s accountability file to make a manual match. The EE will use the data reported in PIMS as well as their SIS to match the student, which is why the Student Demographics are important.

Slide 22: The demographics for each student are embedded in precode labels or test tickets. If the student cannot be matched using the four matching criteria, these demographics can help you or someone in the EE to identify students in the Attribution Window. Some important demographics found in the Student Template are students identified as having an IEP, English Learners, Economically Disadvantaged students as well as the students’ race and ethnicity. The first three are used in the SPP to report the Historically Underperforming Subgroup (HUS). HUS are an unduplicated count of students who have IEPs, are ELs or who are ED. We use race/ethnicity as subgroups for federal reporting. There may be changes in the student’s demographics from one upload to another such as economically disadvantaged and IEP status. Also, please note that race/ethnicity is self-reported and although it is not likely, it CAN change.

Slide 23: Another part of matching the student to the test record when the student record does not match on the four matching criteria of First Name, Last Name, Birthdate and PAsecureID is by collection data from the Programs Fact template, which shows Program Participation. This comes from the second template needed for the PSSA Precodes Upload. This is where Title I information is collected using valid values of 015-018 to be included as part of a student’s demographics. For this upload, please include both individual and school-wide program participation. Title I information is collected and used for reporting Federal Designations in accountability reporting. Slide 24: The third template needed for the PSSA Precodes Upload is the School Enrollment template because it is used to track student entry and withdrawal from the EE. An entry date must be entered when a student is being educated by you. If the student leaves, a withdrawal date is entered showing that the student is no longer being educated at your entity. The school enrollment template also tracks a student’s grade or school building such as moving from an elementary school to the middle school, which is designated by using withdrawal code WD05 (student changes school or grade within the School Year). For accountability, PDE uses the student test result for the calculation of performance and participation rates if the student has Full Academic Year (FAY) status. Entry and withdrawal dates help PDE calculate FAY status based on the dates entered by the EE. The definition of Full Academic Year, which is used for accountability, is a student who is enrolled on or before Oct 1 of the current school year and remains continuously enrolled as of the last day of the testing window for the PSSA testing window in English Language Arts, Math and Science.

Slide 25: Enrollment data is used to determine if a student was educated for a Full Academic Year. We discussed how the School Enrollment template is used to calculate FAY status. FAY status is crucial to accountability reporting. The test results and test participation of a student who is FAY are used in reporting on the School Performance Profile (SPP) and the Required Federal Reporting Measures (RFRM). Last year, PDE began using school enrollment as the denominator for the calculation of Participation Rate. This is consistent with how participation rate is calculated for the Keystone Exams. It also meets State Audit requirements for the state to show that every eligible student tested in the required assessments. The slide shows that when accurate enrollment data is submitted to PIMS, it will be used in the calculation of Participation Rate. The green arrow shows that Participation Rate is calculated by taking the number of booklets scored and dividing it by the enrollment, then multiplying by 100 to get a percentage. The target for participation rate is at least 95%. A participation rate below 95% automatically designates the school as a focus school, which is reported in the federal designations for accountability reporting.

Slide 26: In order to include the student in the Precodes for the Winter Keystones internal snapshot, field 214 must be coded with a code other than ‘Z’ in the Student Template. As mentioned earlier, the LEA has a few options this year on how to submit student records. Once the student is in the internal snapshot, the student will appear in the DRC system and can be placed into test sessions. Even if you submit all students, only those coded in 214 in Student and indicated as currently enrolled in the LEA in the School Enrollment template will be included in the file that goes to DRC.

Slide 27: The school/district/state entry dates in the Student Template are also used to determine if a student record is included in the DRC file from this internal snapshot. Refer to Slide 16 for criteria used to determine if a student record is included or excluded in the DRC file. The guidelines for submitting the school/district/state entry dates are as follows: 1. The dates must be in ISO format, which is the four digit year, followed by the 2 digit month and then the 2 digit day as shown in this slide (YYYY/MM/DD). 2. These fields cannot be left blank. They are required fields. 3. The dates must fall within the current SY or before. As referenced earlier in this presentation, PDE became aware of cases where school/district/state entry dates were entered with future dates. This impacts FAY status and there is a DQE rule that ensures this does not happen moving forward. 4. The state entry date, which is field 109 in the Student template, is the date the student entered a PA public school. 5. The district entry date, which is field 99 in the Student template, must be on or after the State Entry date. This means that the student must have a district entry date on or after the state entry date. It also means that the school entry date can be on or after the district and state entry dates. Essentially, the student must be in the state and district on or before s/he can be in the school. 6. The school entry date, which is field 98 in the Student template, must be on or after the district and state entry dates. This means that the student must have a school entry date on or after the district and state entry dates. The next slides will describe the school/district/state entry dates in different ways to explain these data fields.

Slide 28: This is another way to look at the school/district/state entry dates. The ‘greater than or equal to’ symbol represents ‘on or after’. In simple terms, the school entry date can be ‘on or after’ the district entry date, which can be ‘on or after’ the state entry date. There is one more way to emphasize these concepts using dates.

Slide 29: Using dates to explain school/district/state entry dates may be another way of looking at the relationship between these data fields. There are three columns labeled school entry date, district entry date and state entry date with mathematical symbols in between. Let’s take a look at the first row, which shows that the school, district and state entry dates can be the same date, September 1. This could be a student who moves to your district from out of state. The second row shows that a student enrolled in the school on October 10, which was after the district entry date of September 1. The district and state entry dates can be on or after the same date. This could be a student who comes to your district from out of state, but cannot start at the school until a later date. The third row represents a student who started at your school on or after the same date as s/he entered the district. This could be a student from another district who is transferring to your school. The last row shows a student who started at your school after the date they entered the district, which is after the student entered the state. This could be a student who started at the district in kindergarten and moves from the elementary school to the middle school. Here, the school entry date would be after the district entry date, which could be after the state entry date.

Slide 30: We will now talk about some of the reports that are available to PIMS Administrators.

Slide 31: These reports are available in the sandbox and can be used to test your data before you load it to Production and the internal snapshot is taken. There are verification reports and Extract Warnings.

Slide 32: These reports reflect data loaded into Production.

Slide 33: As mentioned in earlier, there are two sets of reports in Production. We discussed the Presnapshot for Precode Production reports and now we will look at the Precode reports in Production. The first report is the Precode ACS, which can be run after the internal snapshot is taken. The report does not specify that it is for the PSSA because it is used by ELL as well. As stated previously, ACS reports are not required for the precode snapshots. They will be required for the Accountability snapshots for the PSSA and Keystones later in the year. The second report is the ACS for the Keystones, which is not relevant to our discussion today. The next three reports are Extract Warning reports that alert you to potential errors where a student may be reported in multiple LEAs, where the reporting district is different that the DOR, or those students listed with location code 9999.

Slide 34: All assessment and accountability internal snapshots will be submitted using collection window 6. This is open all year and data can be uploaded, updated and deleted (depending on your permissions). You can update data as often as you like to make data submissions easier. Slide 35: I have listed some resources for you from the PAR webpage as well as the RFRM and PIMS. All trainings will be posted on the PAR webpage, which is listed at the top of this slide.

Slide 36: Here is contact information for me, my supervisor, John Weiss and the PIMS Help desk.

Slide 37: This concludes the presentation on the Updates for the Winter Keystone Precodes for the internal snapshot in PIMS. We will give you a few minutes to type your questions and give ourselves time to get organized. There will be a few minutes of silence while we prepare to respond to your questions. Thank you for your patience.