Instructions for Coding of Caregivers and Children in Chinese Orphanages

v.4/12/2010

Definitions

A ward is a group of children who are cared for as a group; the ward may be a regular group of the orphanage or a Half the Sky supplementary group located within or outside the orphanage. For the purpose of this project, there are up to 6 ward types – infants, toddlers, special needs infants, special needs toddlers, and Infant Nurture Program (INP) and Little Sister Program (LSP). INP and LSP are only applicable for post-intervention assessments. If structural changes are in place to integrate children across age and disabilities, ward definition for post-intervention needs to be determined.

A five-minute observation will be called an episode, and the activities going on at the time will be called the context.

A caregiver is defined as a regular institutional caregiver, or a Half the Sky Infant Nurture Program nanny or Little Sister Program teacher.

When Should Observations Take Place?

During the observation period, which could span 1 to 2 days if needed, observations will take place over a continuous block of time – preferably an entire morning, beginning when the children wake up in the morning.

o For pre-intervention assessment, assuming a traditionally organized CWI, there will be a minimum of 4 wards being observed – infants, toddlers, special needs infants, and special-needs toddlers.

o For post-intervention assessment, assuming a traditional CWI plus HTS programs, there will be a minimum of 6 wards being observed – the four above, and HTS Infant Nurture Program (INP) and HTS Little Sister Program (LSP).

o For post-intervention assessment, assuming some structural change to integrate children by age and disability has taken place, the sampling strategy is to be determined.

We will determine, based on actual conditions and logistics, whether observers would stay in one ward continuously throughout the observation period, or they would switch between wards during observation. We can weigh trade-offs with both options (e.g., reducing stranger anxiety or reducing rater fatigue; balancing rater across wards, or age/special needs across raters). On the day before observations are to be made, determine the schedule of activities for each ward that will be observed the next day.

How to Introduce Yourself Before you observe caregivers in a ward, introduce yourself to them and indicate that you will be observing the entire ward, including all the children and all the caregivers. Tell them that they should carry out their duties in the same way that they do every day, and that there is no right way, wrong way, or special way they should behave while you are watching them. They should be natural and do whatever they need to do in their caregiving role. Also, make sure they understand that only you and

1 | P a g e the research team are interested in these observations, no information about individuals, individual behavior, or individual ratings would be shared with their supervisors or administrators or anyone else.

2 | P a g e Who Should Be Observed?

A rater is required to rate a minimum of 2 caregivers per ward (unless there is only one). If possible, an attempt should be made to rate all regular caregivers and others (e.g., supervisors if they perform routine caretaking duties on a regular basis) who are working in that ward the morning the assessor makes the observations. It is possible that there will be more than 2 or 3 caregivers (e.g., a single very large ward with 4 “subgroups” “living” in different areas of the large room); in this case, try to observe as many as you can. It is better to have one set of ratings on as many caregivers as possible than to have two sets of ratings on fewer caregivers. However, it is desirable to have at least one rating on each of the three contexts for each caregiver who is rated. but a minimum of 2 caregivers for at least two 5-min. episodes in all three contexts. If you get only one episode in one or more contexts, that is still useful. Determine at the beginning of the morning, who are regular caregivers on duty that morning and target them for assessments. You will need to have one or more coding sheets for each caregiver; it may be helpful to write on top of each form something that readily identifies each caregiver, such as their physical appearance (tall, short) or article of clothing (red blouse, running shoes) that will help you identify and distinguish between caregivers. Note: there is some advantage to having the caregiver name if caregiver turnover is small so one can do longitudinal analyses; but the name would have to be replaced with an ID number before sharing data with OCD).

Those workers who should not be observed include aides, whose primary assignments are changing the beds, preparing and bringing in food, and cleaning; and specialists, such as psychologists, physical therapists, physicians, or other professionals who provide occasional specialized services to the entire group (e.g., music teacher) or to only a few children and may even remove those children from the group to provide those services. Also, do not observe volunteers or parents who visit and play with the children; only regular caregivers and those who provide routine caretaking duties should be assessed.

Making the Observations and Ratings Review definitions. Review the definitions of each of the Caregiver-Child Behavior and Children with Disability dimensions to be rated before you begin observations for each of the three contexts to make sure that all aspects of each dimension are fresh in your mind. Position for observation. When making observations, try to keep both the caregiver and children in your view, especially their faces, and where you can hear them at all times, because it is the caregiver-child interaction and responses to each other that are major components of the assessment. You need to be able to see and hear the caregiver and the children to whom she is attending, what she and they are saying, what she is doing with the children, her facial expressions, as well as the responses of the children. Be as close as you can but not intrusive. You must not be too close or too obvious that you make the caregiver feel uneasy or self-conscious or that you attract the children’s attention away from the caregiver. If the caregiver or children frequently look at you and watch you or appear awkward or concerned about being observed, move further away. If the caregiver continues to appear awkward and concerned about you, assure her that whatever she does is fine. Try to be respectful of the caregivers and do your best given the circumstances to stay out of their way so they can do their job— do not be so close that you have to move when the caregiver interacts with the children. Children’s actions with you. Ignore children’s friendliness. Children may eagerly come up to you and hug you, want you to play with them, and engage you in conversation. Smile at them if appropriate, but do not respond to them more than this. Do not pat or hug them. Gently and with a smile do not let them hold onto you, and do not talk or play with the children even when you are waiting. Unfortunately,

3 | P a g e being too responsive to the children encourages them to want to play with you and interferes with your observation.

4 | P a g e OBSERVING CAREGIVER-CHILD INTERACTIONS

Selecting a caregiver You will make a minimum of 21 five-minute observations on each regular caregiver, one at a time. More if time permits. All the observations on a single caregiver will not be done at the same time one right after the other. Instead, a five-minute observation will be made on one caregiver, then on another caregiver, on another caregiver, etc., and then back to the first caregiver. Selecting which caregiver and which five-minutes should be done as randomly as possible, subject to who is present on the ward and active in the target context (e.g., feeding, bathing, free play). Which caregiver and which five minutes should not be selected on the basis of the caregiver’s behaviors at the time (i.e., don’t pick a caregiver because she is especially engaged with a child at the moment). Start by arbitrarily selecting one caregiver who is present at that time and engaged in the target activity of the context and start five minutes of observation on that caregiver. After the five minutes of observation, stop and make the ratings; then arbitrarily pick another caregiver and start another five- minute observation, etc. It is important that as many of the caregivers be rated as possible but not less than 2 caregivers and preferably at least two times (episodes) in each context.

Rating caregiver and children

Remember that you are rating an individual caregiver, so focus your attention on that caregiver and what she does with the children, ignoring the behavior of other caregivers. If the caregiver behaves very differently with two different children during one five-minute episode, you should “mentally average” those different behaviors when determining that caregiver’s ratings for that one episode.

Also, the Caregiver-Child Rating Scale (CCSERRS) includes ratings of “the children,” and those children should be the children that the caregiver under observation attends to and interacts with during the five-minute episode. If the caregiver only interacts with a single child, then the child ratings are based only on the behavior of that child; but if the caregiver attends to most or all children on the ward by watching or talking to them or sitting in a group activity, then make the child ratings on the basis of all of the children.

Contexts to be observed

Observations and ratings should be made in three different contexts and it is desirable to have at least one rating on each of the three contexts for each caregiver who is rated:

 Changing/bathing/dressing. Infants are changed and bathed individually, and a caregiver may change more than one infant in five minutes. Older children may dress or shower in groups and the caregiver may help individual children with part or all of dressing. Children are often changed, bathed, and dressed in one continuous activity. Try to have all three of these specific component activities represented in the observations across all caregivers, if not for an individual caregiver.

Try to obtain at least two five-minute observation episodes, more if possible, of each caregiver engaged in the context of changing/dressing/bathing (two minimum). It is more important to have one set of ratings on as many different caregivers as possible than it is to have two or more episodes on only some of them. Begin an episode on a caregiver only when she begins or is already engaged in this activity. If a caregiver is out of the room, wait until she returns and begins this activity with at

5 | P a g e least one child. But if she leaves this activity during the five minutes, keep the clock running. If she engages in “transition activity” (i.e., replaces an infant in a crib after changing and gets another to change), include her transition behavior in your ratings; if she leaves and conducts activities irrelevant to changing/dressing/bathing (e.g., takes a break, fills out records), keep the clock running and score her behavior as zero for this part of the observation, which should be mentally averaged with her on-task behavior for this episode. However, if the changing/bathing/dressing is completed for the entire ward (i.e., the caregiver stops this behavior and does something else that is irrelevant to changing/bathing/dressing because all the children have had that done with them), stop the episode and score the caregiver only for the time she spent in changing/bathing/dressing. It is possible that a specific caregiver does not engage in this activity at all, leaving this to other caregivers, in which case she is not rated at all on this context.

Try to obtain at least onetwo five-minute episodes for each caregiver (minimum of two); more if time permits. It is more important to have ratings on as many different caregivers who are available than it is to have two or more than two episodes in a single context on only some of them.

Observe a single caregiver for five minutes, unless the activity completely stops and is finished (see above). Mark the coding form after each five-minute observation and before you start another five- minute episode. Once you have completed a single five-minute episode on one caregiver, conduct the next observation on a different caregiver until all caregivers engaged in this context activity have been rated before you return to the first caregiver to do a second episode.  Feeding. Use essentially the same procedure as described above for changing/dressing/bathing to observe caregivers feeding children. Infants must be fed one at a time, so start an episode when a caregiver starts or is already feeding an infant. Do not begin the observation when the caregiver is getting the child from the crib or bringing food to the table. Older children may eat in a group, sitting at tables. Select a caregiver who is at least present in the room and watching the children at the beginning of the five-minute episode. Continue the observation for five minutes, although the caregiver may not feed children during all of those five minutes (follow the same procedure as described above in this case). However, base your ratings on the behavior of the caregiver in the feeding context and in transition activity; then mentally average in zero for the time she spent in activities irrelevant to feeding (e.g., talking to other caregivers, leaving the ward). Again, if the caregiver stops feeding and does something irrelevant to feeding because there are no more children to feed or the meal is over, stop the episode early and rate it only on the basis of the time in feeding and transition. Follow the procedure described above in moving from caregiver to caregiver.

 Free play. Free play consists of time when children are awake but are not being fed, changed, bathed, or dressed. It may consist of unstructured time in which children do whatever they want, including playing with toys, watching TV, or engaged in structured or unstructured group activities with caregivers, nannies, or teachers (but not with specialists, such as physicians or physical therapists or rehabilitation specialists). Follow the procedure described above for selecting a caregiver. An episode starts when the caregiver is present in the room at least watching the children at the time of free play. A caregiver may simply stand around, talk to other caregivers, or interact with the children--all of these behaviors are part of the five-minute episode; but if she is changing, bathing, or feeding during free play time, do not consider that part of her five-minute free play episode. Once the episode starts, if the caregiver leaves the context for reasons irrelevant to free play or other caregiving (takes a break), keep the clock running but her free play behavior is

6 | P a g e zero for the time she is absent. Stop the episode early if free play time is ended for the group and rate only on the basis of behavior during free play. If having the TV on is common during free play, so be it. If TV time is scheduled as a separate time period and there is another scheduled period of free play without TV, try to have episodes in each in proportion to the time allotted for TV vs. no TV. Remember, it is better to have one set of ratings on as many caregivers as possible than to have two sets of ratings on fewer caregivers. It is desirable to have at least one rating on each of the three contexts for each caregiver who is rated at all.

CHILD BEHAVIOR SCALE (CBS)

After you have observed a minimum of one five-minute episode each for two caregivers in free play only, make one five-minute episode in which you observe the behavior of all the children present and rate Child Behavior Scale at the end of that episode (only in groups in which most children are 18 mos. of age or older). Then return to conduct second five-minute episodes on two additional caregivers, followed by another five-minute episode rating children’s behavior, and continue for as long as free play lasts. Of course, if a caregiver leaves the room, make the children’s behavior observation instead of the absent caregiver, and then code the caregiver when she returns.

Indiscriminate Friendliness One of the Child Behavior Scale is on indiscriminate friendliness which is assessed in a highly specific situation:  First, as with all the child behaviors, it is only rated in wards in which most of the children are 18 mos. of age or older.

 Second, it must be rated in the context of free play, that is, when essentially all the children on the ward could stop whatever they are doing and attend to, greet, or touch a stranger who enters the room.

 Third, the stranger must be indeed unfamiliar to the children, so if you have been conducting assessments on the ward for 60-90 minutes, you will not be a stranger to the children on that ward. Therefore, when you first arrive in the morning, if children are in “free play” as defined above, then you can be the stranger. If children are not engaged in free play when you arrive, one of the other assessors must come to the ward during free play and be the stranger.

7 | P a g e Procedure for Rating Indiscriminate Friendliness The stranger enters the room and stands near the middle of the room and remains there for three minutes. The stranger looks at the children but does not greet them or talk to them; the stranger should smile and be pleasant, but not encourage social interaction with the children. If children come to the stranger and greet the stranger, the stranger should smile or nod to acknowledge the child but not encourage conversation or interaction. If children touch the stranger, the stranger should not encourage the behavior but should accept that it is happening. Children who grab hold or climb on the stranger should be gently put back on the floor with a smile but without talking. After five minutes, the assessor makes a rating of the extent and pervasiveness of children’s indiscriminately friendly approach, interaction, and especially physical contact with the stranger.

The rater should attempt to rate indiscriminate friendliness even in Special Needs wards. We will discuss later what the results mean in cases of wards where most or all of the special needs children are immobile.

8 | P a g e CHILDREN WITH DISABILITIES RATING SCALE (CDRS) A separate form for rating Children with Disabilities has space for several ratings that pertain to children with disabilities. These ratings are not made after each five-minute episode, but rather are made once at the end of all Caregiver-Child Ratings within a single context (i.e., once at the end of changing/bathing/dressing, once at the end of feeding, and once at the end of free play). Each of these three ratings should reflect the activities that are described which transpired during the entire observation of that context. Some of the ratings pertain only to specific contexts, so not all ratings will be made for each context (see Rating form).

Without structural change in place, this rating form primarily applies to the designated “Special Needs” wards for infants and for toddlers, in both pre-intervention and post-intervention assessments. In the non-special-needs wards, if there are no children with severe or multiple disabilities in a particular ward, do not create a separate rating for children with disabilities for this ward (even if there are a handful of children with mild disabilities mixed in with a larger number of children who are typically developing.)

9 | P a g e OVERALL SEQUENCE OF OBSERVATIONS AND RATINGS The list below summarizes the sequence in which observations/ratings should be made. This is the ideal; the circumstances and events on the ward will likely require some modification of this ideal.

1. Changing/Dressing/Bathing and Feeding (all wards).

a. Observe/rate Caregiver A for 5 minutes, then Caregiver B, then Caregiver C, and so on. Rate as many caregivers as possible but at least 2 (unless there is only one caregiver for this ward). It is better to have one set of ratings on as many caregivers as possible than to have two sets of ratings on fewer caregivers, but tTry to rate at least 2 episodes on each caregiver. In addition, it is desirable to have at least one rating on each of the three contexts for each caregiver who is rated. b. Repeat observations/ratings on Caregiver A, Caregiver B, Caregiver C, and so on. Caregivers do not have to be rated in the same sequence as above, but it is preferred. c. At the end of making all ratings for a single context, make ratings for Children with Disabilities Scales for the context. d. Repeat 2a, 2b, and 2c for the other context. e. Do not rate Child Behavior Scale in changing/dressing/bathing and feeding contexts.

2. Free Play

a. Make Indiscriminate Friendliness Rating – Indiscriminate Friendliness is only for wards where most children are 18+ months old. This rating is only done once per ward. This requires that a stranger enter the room during free play and stand in the middle of the children for 3 minutes. If the children are in free play when the Assessor first arrives in the ward, the Assessor can be the stranger and conduct this observation. If the Assessor arrives and children are being changed/dressed/bathed or fed, the Assessor makes ratings for these contexts and another Assessor must be the stranger later during free play.

b. On Caregiver-Child Interaction rating, observe/rate Caregiver A for 5 minutes, then Caregiver B, then Caregiver C, and so on. Rate as many caregivers as possible but at least 2 (unless there is only one). It is better to have one set of ratings on as many caregivers as possible than to have two sets of ratings on fewer caregivers, but tTry to get at least 2 episodes on each caregiver rated. c. Observe/rate Child Behaviors Scale for one 5-minute episode (only wards for children 18+ mos.) Do one CBS after the first two caregiver-child ratings; then move on to additional caregivers. This ensures that, even when there are a large number of caregivers, CBS still gets done. d. Repeat 3b and 3c as many times as Free Play permits. e. At the end of making all ratings for this context, make ratings for Children with Disabilities for this context if there is a designated special needs ward or there are numerous children present who are severely disabled or multiply disabled.

10 | P a g e Pre/Post Assessment Procedures Based on Minimum Data Collection Requirements

The table below makes the following assumptions about minimum data collection requirements. This procedure can be used if logistics or other factors prevent a prolonged observation of the entire orphanage or even most of the orphanage. The minimum data requirement can be satisfied within a continuous or segmented 3 – 4 hour window with 4 observers.

1. Sample at least 4 or 6 wards (pre or post assessment) if we cannot cover the entire CWI.

2. Sample at least 2 x 5-minute episodes x 2 caregivers per ward.

3. Sample at least one free play context and one task-driven context (feeding, changing, dressing, or bathing) per ward.

Assuming we have 4 assessors – A, B, C, D, this procedure can repeat itself if we are sampling more than 6 wards and more than 1 day.

Typically Developing Special Needs Half the Sky Programs (pre & post) (pre & post) (post-tests)

Time Period (up Infant Little Infants Toddlers Infants Toddlers to 1 hour Nurture Sister each)

Task A B C D n/a n/a Context

Free Play A B C D Context

Free Play C D A B Context

Ideally, the three time periods would be continuous (e.g., 1 hour breakfast  2 hour free play, or 2 hour free play  1 hour lunch). But if that is not logistically feasible, we can sample at different times of the day (e.g., 2 hour free play in the morning, 1 hour bathing in the afternoon)

11 | P a g e RELIABILITY PROCEDURES

The purpose of reliability procedures is to document that the four assessors rate caregivers and children’s behaviors in essentially the same way. This requires that pairs of assessors perform the identical five-minute observation and make ratings privately and without consulting with one another. These ratings are then compared between the two assessors after the entire morning observation is completed. This will be done in two orphanages selected for this purpose, not the target orphanages.

Reliability requires that each assessor is separately paired with each of the other three assessors more-or-less an equal number of times. Also, assessors should be distributed across different wards that have children of different ages—a single assessor should not only code rooms with infants but should also code wards having children of different ages. Each pair of assessors should rate caregivers-children in each context. The ideal of a perfect balance of an equal number of episodes for each pair of assessors and equal distribution of assessors across wards of different ages of children and under each context is never perfectly achieved, but we should do the best we can.

Assessor Pairings

Assuming four assessors, A=Ma Lang, B=Zeng, C=Graduate Student C, D=Graduate Student D If each assessor is paired with every other assessor, there will be six pairs: AB, AC, AD, BC, BD, and CD. Of course, at any one moment in time, there are only two pairs of assessors that can be operating simultaneously and there are only three sets of possible pairs: AB—CD, AC—BD, AD—BC. We will spend only one day in each of 2 orphanages for reliability purposes. A reliability schedule of assessor pairs might look like the following:

Paired- Assessor Reliability Schedule for One Orphanage Ward Assessor Pairing Approximate Times

Orphanage 1 Orphanage 2

1 (Infants) AB CD 2 (Toddlers) CD AB 8-10:30 am 3 (SN Infants) AC BC 4 (SN Toddlers) BD AD 10:30-13 pm 5 (Toddlers) AD BD 6 (Infants) BC AC 15-16:30 pm 7 (SN Infants) BD AB 8 (SN Toddlers) AC CD 16:30-18 pm This schedule balances well but not perfectly each assessor with each other assessor and each assessor across each age group. It also minimizes the amount of switching that assessors have to do between one pairing and another, because they will switch pairs only once during a morning and once during an afternoon. The alternative is for each pair to code five-minute episodes with each caregiver in

12 | P a g e one context, and then A and D stay in the same ward and B and C switch wards to do the second episode in that context. But this would mean that assessors would switch three times in a morning, once during each context activity. Scientifically, this is better, because each pair will rate the same caregivers in each context; but procedurally it is more complex and disruptive. Some procedure will need to be planned to get a “stranger” to enter the ward during free play to get reliability on indiscriminate friendliness.

A more conservative schedule, assuming that travel and CWI logistics would only allow us half- day observation versus a full day, is below.

Periods below are Special Needs Special Needs Infant Ward Toddler Ward 1 hour each Infant Ward Toddler Ward

Dressing/Changing AB CD and/or Breakfast

Free Play First Half BD AC

Free Play Second AD BC Half

Lunch and/or CD AB Bathing

Overall, we will conduct reliability in two different orphanages, spending one day in each orphanage. Be sure to note on your coding sheets the specific orphanage, caregiver, the specific ward, ages of children, date, the assessor making the ratings, and the assessor with whom you were paired.

We will conduct refresher training before doing post-intervention assessments, and we will repeat reliability documentation if a new assessor replaces an original assessor or is added to the group and in any case before the 12-month post-intervention assessment.

13 | P a g e