“QRIS Delivery System Infrastructure” The process for completing the rating of a program for a QRIS will be examined. A description of the rating process in 4 states will be presented with descriptions of the role of licensing, rating assessment and monitoring for maintenance of rating. The use of technology and data collection will be included. North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars Standards 5 categories of 2 categories of performance 7 categories of performance 4 categories of performance performance standards: program, staff standards: licensing status, standards: staff qualifications standards: learning education plus quality point. administration, staff and professional development, environment, family 5 levels of performance qualifications and learning program, leadership partnerships, training standards: 1 star licensing to responsibilities, Director, and management, partnerships and education, staff- 5 stars. Accreditation not learning environment, parent with family and community. 4 child ratio and group included. involvement, program levels of performance size, and accreditation evaluation. 4 levels of standards: One Star to Four through an approved performance standards: One Stars. Accreditation can be organization. 5 levels star licensing, One star Plus, substituted for the 4 star of performance Two Star, Three Star. requirements standards: provisional Accreditation through an (NAEYC/NSACA/NAFCC/NECPA) to Star 4. approved national accrediting in conjunction with an average body is the performance ERS score of 5.25 for all standard for the Three Star. sampled classrooms and documented minimum hours of professional development. Orientation to QRIS Qualistar Early Learning Process is integrated with the Process is integrated with the Programs contact Regional process: Information provides three trainings licensing process: pre- licensing process: resource Keys for information. sent to provider, to their Child Care licensing workshops and at guide provided at the time of STARS Orientations are workshop, coach Resource and Referral the time of licensing licensing application. Initially required to enroll in the STARS assigned, etc. (CCR&R) network application. Information statewide workshops. Stars program. This orientation partners. These three provided on website and by Outreach Specialists available provides introductory trainings allow the R&R agencies and Smart Start for TA. information on PA’s continuous service delivery areas partnerships. quality improvement system (SDA) to build capacity and reviews the supports that in their local are currently available. communities. The first 1 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars training is Orientation At site visits, small group work to the Rating, a train session, conferences and over the trainer model the phone, Regional Key staff which provides an provide information and overview of the education on the continuous process, a copy of all quality improvement system, forms used, and assist the practitioner in instructions on understanding the Standards completing the forms. and access resources The information is given for child care centers and family homes. The second training is Getting Ready for Ratings, a train the trainer model which provides a Getting Ready for Ratings notebook containing a program self assessment checklist and resources for each of the five components of the rating. The third training is Coaching and the Rating, which is a training for coaches to build their skills and improve their understanding of program coaching and

2 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars the continuous quality improvement cycle. Providers Eligible To be eligible for a All licensed providers: All licensed providers: The Keystone STARS voluntary rating, a program must centers, homes, preK, Head centers, homes, Head Start. program is open to all child be licensed by the state Start. 100% participation. Voluntary participation. care regulated providers of Colorado. Programs FFN not included. (Centers and Homes - not FFN) include centers, family and Head Start (which is not homes, Pre- regulated in PA) Kindergarten programs, and Head Start. Family Friend and Neighbor (FFN) is not included. Application Process The Qualistar QRIS is a All licenses are rated. Interested providers submit Providers wishing to Voluntary, by mail, voluntary system. A Providers choose to apply for application with specified participate submit an onsite, by web, fees, program application is a higher rated license. supporting documentation. application. STAR status is sent via mail after the Licensor provides application Star status is non-expiring but renewed annually. No fees. program has forms at the time of the can be increased or committed to receiving annual license review. decreased at any time. There a rating and the Licenses good for 3 years. is no fee. funding is secure. The Licensing fee. fee for a biennial rating is $1200 per classroom. Assessment: A Qualistar Quality Licensor checks all program Stars Outreach Specialist STARS Managers assist the conducted by whom, Rating Specialist is standards (basic licensing and reviews the application, program in managing the frequency, steps in assigned to a program enhanced standards such as supporting documentation process, resources and process, appeal. and is responsible for reduced ratios, and licensing compliance to understanding the Standards. collecting data for each administration standards, grant star status. At two star of the five rating parent involvement, etc...) level, provider is required to STARS Designators (different components (see data Providers may choose to have obtain ERS assessment within person with the Regional Key) collection). The an ERS assessment to earn one year. One classroom actually designate the assessment takes one additional points. assessed every four years program. month with the Quality Staff education forms unless accredited – window

3 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars Performance Profile submitted to state licensing of time given to program. If ERS Assessors - employed by (QPP) as the end agency for review by score is challenged, a second the Regional Key, reliability product. The QPP Workforce Unit. Total score = assessment can be between assessors. FDCH has outlines the results of Points earned for program completed. ERS at STAR 2, Centers at STAR the assessment, standards, including ERS 3 +4 every other year for both highlighting the assessment plus points for (annually if director turns over, programs strengths and staff education. more than 50% staff turnover). recommendations for Baseline score needed is stated quality improvement. in the STARS Standards. If not achieved can redo but maintain lower STAR level. NAEYC Accredited programs have ERS in years 2 and 4 of accreditation cycle (if continuing).

Process for taking STARS away or reducing STAR level is in place. Monitoring for A Rating is conducted Licensor monitors on annual Program must notify licensor Start with STARS and STAR 1 maintenance biennially during which licensing visit. of changes within 5 days. are paperwork process. STAR 2 time a program must Selected criteria can call for a Licensor documents and above STARS Designator maintain their license new assessment before 3 yrs, compliance with all criteria documents compliance with in good standing. e.g., new director, 50% annually. If concerns arise Standards annually. Programs participate in turnover in staff. during routine licensing Expectation is for STARS a mid rating self study Licensing violations leading to monitoring visits (three times Manager to have contact which allows the administrative actions results a year), i.e., staff turnover or (phone, site, email) at least program to report on in probationary or provisional serious non-compliance, a quarterly for relationship their improvement and license with no stars. plan of correction is required. development. goals for their program. Qualistar has a process to follow up on

4 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars program complaints / concerns received from the community. Appeal Process A program may initiate May appeal staff If initially denied, can request If licensing violation occurs a Technical Review of qualifications to Workforce clarification and reapply. If STARS can be suspended or its Qualistar Rating Unit and ERS assessment to reduced, can appeal and/or taken away. Can appeal within 30 calendar days Advisory Committee for ERS. propose an alternative decisions. of receiving their settlement. Qualistar Rating Cannot reapply for six months Consultation. The if denial or reduction due to purpose of the non-compliance. technical review is to correct errors in documentation collected prior to the consultation or a mistake in the rating calculation that may have occurred.

A program may initiate a Dispute Resolution Process within 30 calendar days of receiving their Qualistar Rating Consultation. The Dispute resolution process is used to address instances where established Qualistar protocols

5 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars were not used with fidelity, to address instances where any component of the Qualistar Rating was applied inappropriately or to address instances of inappropriate or unprofessional actions on the part of a Qualistar employee that resulted in an incorrect rating. Data Collection: Each classroom in a Licensing status (ratings) Licensing database available Certification (licensing) data purpose, type, use program receives a 3 including violations, to parents includes star status base available to parents hour observation using administrative actions. and licensing violations. Data includes violations, general the ITERS-R, ECERS-R or Available to public including collected includes type of program referral information FCCERS-R Classroom parents and technical facility and number of and STARS Status. Programs Environment Rating assistance network. subsidized children by star sorted for referral preference Scale to assess the status. Notice of changes in given to STAR 2 and higher. quality of the Qualifications of each staff star level sent to R&R. Working to connect all Data classroom learning person ERS assessment data sources together in PELICAN environment. The ERS assessments compiled to identify areas (certification, subsidy, Early observation is Number of children served, where increased training Learning Network - children’s conducted by a ages, group size staff-child could address common outcomes and STARS). Qualistar Rating ratios, program problems. Specialist who is activities/curriculum. Monthly STARS reporting by reliable on the ERS tool Subsidy children served. region for the ‘inside’ public used. Classroom ratios including movement up/in and and groups sizes are Used for licensing and out. collected by a Qualistar subsidy administration, Rating Specialist at 8 informing public, evaluation, Program, County, Regional and

6 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars time points for full-day performance monitoring of Statewide ERS reports used to programs and at 4 time county Smart Start drive professional points for part-day partnerships. development, TA and inform programs. Programs system submit training Information also used to certificates and formal inform decisions on use of transcripts on each quality funds, professional staff member to be development, training, reviewed by a rating technical assistance and specialist to assess the special projects by R&Rs and training and education Smart Start partnerships. level of staff. Family surveys are distributed and returned to Qualistar and rating specialists conduct an interview and review program documentation regarding the types of family partnership activities offered in the program. The program also submits an accreditation certificate if they have achieved national accreditation. Evaluation studies The Qualistar Early Previously – validation of Research from 1999 through Research Study in December Learning is in the rating standards and 2003 validated link between 2006 major findings: process of conducting a outcomes for children. quality and star level and  There is sufficient evidence to support the Keystone STARS validation study of the Currently, collection of improvement in quality of QRS as a reliable indicator of Qualistar Rating changes in ratings and care. Currently, workforce quality.  Child care quality is 7 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars System. The workforce information. data from registry, wage improving. Keystone STARS continues to improve quality overarching research supplement program and in child care centers questions concern: market rate survey.  Programs with a defined 1. How well do the curriculum have higher quality  Teachers with college degrees individual provide higher quality early component education and care. measures work (e.g. are they capturing what should be captured, are there more effective ways to collect information)? 2. How well do the components work together (e.g. are they moderately well correlated; are the components measuring different aspects of program quality)? 3. Do programs that have had experienced being rated improve over time? 4. Is the QRS and are individual component measures related

8 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars to other measures of process quality? 5. Is the QRS and are individual component measures related to selected social and cognitive child outcomes? If they are related to the above, how should the individual component measures be weighted in an overall QRS score? Information on QRIS Parents are provided License displays total possible Initial broad public awareness Major media campaign 5/08. provided to parents with an Early Learning stars, points earned in campaign. Provider receives Governor filmed TV spots, and the public Report which they can categories of program and in certificate, decal and staff posters, brochures and radio download from the staff education, total points pins. R&R’s and Child Care add. Qualistar website. It earned by program and stars Locator website show star R+R shows STAR levels describes the strengths earned by center. levels. Community Engagement and area for continues to link child care to improvement of the State agency website has all the other ECE programs rated program by ratings including other (schools, Head Start, EI) Public quality component. information on programs. Engagement campaign called Parents can also access PA Promise for Children "A Guide to the State agency website includes Qualistar Rating" a search function for parents through our website to search for care by that describes each geographic area, type of component of the program and rating of rating and what it program.

9 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars means for children. Information about the Qualistar Rating also appears in our Consumer Education brochure for Parents titled "Working Your Way through the Child Care Maze". Rewards for Grants may be Tiered subsidy Tiered subsidy All funding and resources Participation in QRIS available through local reimbursement, Smart Start reimbursement (greatest leveraged to Keystone STARS Child Care Resource improvement grants, increase between one and programs- Support Grants, and Referral (CCR&R), scholarships, wage three star is 200%), double Merit Awards and Education Early Childhood supplements, health conference vouchers, and Retention awards to Councils or insurance reimbursement, scholarships, wage programs serving at least 5% foundations. Grants loan program. supplement program, subsidized population. can include quality Subsidy rate increases limited previously bonus points for improvements, to 3-5 stars. quality grants. Must be STAR one or higher professional (no subsidy minimum) to development, Access to technical assistance receive: coaching, and capital and grants. improvements, as well  Free ERS visit and reports as for a Qualistar  Health and Safety Rating. resources and materials  Free STARS Technical Assistance In the city of Denver,  Invitations to participate in parents can receive special initiatives like Mind differential tuition in the Making, Ages and credits to send their Stages program, overnight child to a preschool conferences, Color Me based on a Qualistar Healthy materials and

10 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars Rating. training  T.E.A.C.H Scholarships and vouchers for credential based PD.  CDA, SACC Credential, Director Credential enrollment  EC Mental Health Consultation

Relation of Colorado offers a Providers approved to serve Centers must be above one Tiered Reimbursement Participation in QRIS county option of tiered families with cc subsidy must star level to serve families available at STAR 2 and higher. and in Subsidy reimbursement where be regulated providers and all with child care subsidy unless Program the rates are set by regulated providers get a care is unavailable each individual county. rating. Rates are determined by a number of factors: local economics, child care availability, market prices/rates and quality. Currently, two counties tie reimbursement to the Qualistar Rating. Others counties are looking at it for next fiscal year. Relation of School-Readiness All participants must be Public schools seek providers Child Care has to be a STAR 3 Participation QRIS and Grant: Colorado regulated providers. Some with higher star levels to or above to participate in state in Other Support established a school programs limited to specific collaborate on funded PA Pre-K Counts. And Programs for Child readiness child care levels of star license. prekindergarten programs. many Head Start agencies seek Care Providers subsidization program STAR 3 or higher to partner

11 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars (HB 1238) The initiative PreK classrooms must have a with for HS services. makes three-year 4 or 5 star rating. grants available to child care centers that fed into low performing public schools. To receive three-year funding the centers were required to obtain a quality rating and show improvement every year.

In the City of Denver, programs participating in a parent tuition credit program receive a Qualistar Rating (tied to parent tuition credit) and receive quality improvement funding based on their rating. Impact of Loss or This has not been Ultimate impact on subsidy Significant reduction in If program assigned a new Reduction of Rating analyzed. reimbursement. For sale of subsidy reimbursement level, awards and center – may wait 6 months occurs four months after reimbursements are given at for a new rating assessment reduction of rating. Cannot the new level. but maintains previous rating reapply for six months if Could be significant financial subsidy reimbursement for reduction due to non- loss. that 6 months. For licensing compliance. violation – no reduction in subsidy rate until completion of corrective action plan. No

12 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars change in other benefits as long as facility is still licensed. Local grants may be affected. Changes in Process or Because Qualistar has From 3 components – Creation of transitional level Went from 5 categories to 4 in Standards since conducted a compliance history, program, between one and two star; the standards. Added more on beginning comprehensive and staff education to 2 change in how often criteria curriculum, observation and thorough examination components -program and are monitored and how feedback of teachers, of the rating, many and staff education licensing non-compliance is professional development substantial changes defined; less frequent ERS /qualifications (gave have been made to it assessments; strengthened forerunner notice of since the beginning if criteria in staff professional substantial changes and its implementation in development plans, lesson grandfathering with plan to 1999. Documenting plans, outdoor interest achieve), and strengthened the each change is beyond centers, daily reading, staff content and expectations of the scope of this evaluations the program CQI plan. document. Changes to the ratings have occurred, not just because of evaluation results, but as well based on the cost of measuring aspects of the child care environment, based on feedback from the provider community and based on observations from quality improvement staff.

13 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars Substantively the major changes that have occurred involved: 1. Adding ratios and group sizes as a component because it soon became clear that state licensing ratios did not support high quality programming and many programs were in violation of state regulations. 2. Modifying the conceptual underpinnings and data collection methodology (at least 5 times) of the parent involvement/family partnership criteria to assure that the measures were adequately discriminating between low and high quality programs.

14 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars 3. Removing accreditation as a mandate to reach Star 4 as many of the criteria in NAEYC accreditation overlapped with other rating criteria and may have inadvertently caused a financial hardship for providers to administer both systems. 4. Modifying (at least 5 times) the training and education criteria for a variety of different reasons (e.g. to make them attainable, to account for the time each staff effected children, to try and understand better which staff should be included in the calculation).

15 North Carolina 5 Star Rated Oklahoma Reaching for the Colorado Qualistar Pennsylvania Keystone Stars License Stars 5. Reducing the time spent collecting the ERS (from 10 hours to 3). Weighting the overall and individual component measures differently as empirical data became available through the evaluation. For more information Gladys Wilson Peggy Ball Nancy vonBargen Debi Mathias contact [email protected] [email protected] [email protected] [email protected] http://qualistar.org www.ncchildcare.net www.okdhs.org/programsand www.pakeys.org, services/cc/stars/

16