The School Action Effectiveness Study: Second Interim Report, Part II

DOCUMENT RESUME

ED 240 421 CG 017 250 AUTHOR Gottfredson, Gary D.; And Others TITLE The School Action Effectiveness Study: Second Interim Report, Part II. Report No. 342. INSTITUTION Johns Hopkins Univ., Baltimore, Md. Center for Social Organization o. Schools. SPONS AGENCY Department of Justice, Washington, D.C. Office of Juvenile Justice and Delinquency Prevention.; National Inst. of Education (ED), Washington,DC. PUB DATE Oct 83 GRANT 80-JN-AX-0005; 82-JS-AX-0037; NIE-G-80-0113 NOTE 435p.; Some tables are marginally legible because of small print. For the Second Interim Report, Part I, see ED 237-892; for the First Interim Report, see ED 222 835. PUB TYPE Reports - Evaluative/Feasibility (142)

EDRS PRICE MF01/PC18 Plus Postage. DESCRIPTORS Art Activities; Delinquency Prevention; Delivery Systems; Dropout Prevention; Educational Counseling; Educational Environment; Educational Improvement; Employment Counselors; *High Risk Students; Intervention; *Nontraditional Education; *Outcomes of Education; *Program Effectiveness; Program Evaluation; Program Implementation; Recreational Activities; Secondary Education; *Secondary School Students; Student Development; Youth Problems ABSTRACT This report, the second part of a longer study prepared by the Delinquency and School Environments Program, further describes interim results of the program's national evaluation of the Office of Juvenile Justice and Delinquency Prevention's (OJJDP's) Alternative Education Program. The report consists of evaluationsof 14 specific projects: (1) Compton Action Alternative School; (2) Project STATUS; (3) Project RETAIN; (4) The Milwood Alternative Project; (5) Project PREP; (6) The Jazzmobi-le Alternative Arts Education Project; (7) Otro Camino; (8) Project PATHE; (9) Virgin Islands Alternative Education Project; (10) Academy for Community Education; (11) Alternative Education for Rural Indian Youth; (12) Plymouth Alternative Education Project; (13) Educational Improvement Center--South Alternative Education Project; and (14) Jewish Vocational Services Alternative Education Project. The prOjects provide alternative education programs for high risk junior high and high school students and several include art, recreation,or vocational activities as well as education programs. The effectiveness of the school in promoting student development while preventing dropping out and delinquency is evaluated in these studies. (JAC)

*********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** THE SCHOOL ACTION EFFECTIVENESS STUDY:

SECOND INTERIM REPORT

PART II

Gary D. Gottfredson, Denise C. Gottfredson, and Michael S. Cook

Editors

Report No. 342

October 1983

This work is supported primarily by Grants No. 80-JN-AX-0005 and 82-J5-AX-0037 from the Office of Juvenile Justice and Delinquency Prevention, U.S. Depart- ment of Justice, andin part by Grant No. NIE-G-80-0113 from the National Institute of Education, U.S. Department of Education. The opinions expressed do not necessarily represent the position or policy of any agency.

Center for Social Organization of Schools The Johns Hopkins University 3505 North Charles Street Baltimore, Maryland 21218

U.S. DEPARTMENT OF EDUCATION NATIONAL INSTITUTE OF EDUCATION "PERMISSION TO REPRODUCETHIS EDUCATIONAL RESOURCES INFORMATION MATERIAL HAS BEEN GRANTED BY CENTER 'ERIC/ LI This document has been reproducedas / received from the person or organization originating it. 7(Minor changes have been made to improve reproduction quality.

Points of view or opinions stated in this docu- TO THE EDUCATIONAL RESOURCES ment do not necessarily represent official NIE INFORMATION CENTER (ERIC)." Position or policy. The Center.

The Center for Social Organization ofSchools hastwo primary objectives: to develop ascientific knowledge of how schools affect their students, and to use this knowledge to develop better school practices and organization.

The Center works throughthree researchprograms to achieve its objectives.

The School Organization Program investigates how school andclassroomorganization affectsstudentlearning and other outcomes. Current studies focus on parental involvement, microcomputers, use oftime in schools, cooperative learning, and other organizational factors. The Education and Work Program examines the relationship between schooling and students' later-life occupational and educational success. Current projects include studies of the competencies required in the workplace, thesourcesoftraining_ and_ experience that lead to employment, college students' major field choices, and employment of urban minority youth. The Delinquency and School Environments Program researches the problem of crime, violence, vandalism, anddisorder in schools andtherole thatschoolsplay in delinquency. Ongoing studies address the need to develop a strong theory of delinquent behavior while examining school effectson delinquency and evaluating delinquency prevention programs in and outside of schools.

The Centeralso supports a Fellowships in Education Research program that provides opportunities for talented young researchers to conduct and publish significant research andencourages the participation of womenand minorities in research on education.

This report, prepared by the Delinquency and School Envi- ronments Program, describes further interim results of the program's national evaluation of the Office of Juvenile Jus- tice and Delinquency Prevention's (0JJDP's) Alternative Edu- cation program. The first interim results were reported in CSOS Report No. 325, April 1982.

ot School Action Effectiveness Study Staff

The Johns Hopkins University

Gary D. Gottfredson, Project Director Denise C. Gottfredson, Associate Research Scientist Michael S. Cook, Associate Research Scientist Donald E. Rickert, Jr., Research Analyst Helene Kapinos, Data Technician Leis Hybl, Secretary Andrea Nuzzolo, Research Assistant Robert Kirchner, Student Research Assistant Abhijit Mazumder, Student Research Assistant Stuart Gavurin, Student Research Assistant Norm Ringel, Community Psychology Intern

Social Action Research Center

J. Douglas Grant, Project Advisor Deborah Daniels, Research Associate Jane St. John, Research Associate

Federal Project Officers

National Institute for Juvenile Justice and Delinquency Prevention

Barbara Tatem Kelley, Project Officer

Special Emphasis Division, Office of Juvenile Justice and Delinquency Prevention

Roberta Dorn, Project Officer Catherine Sanders, Project Officer

Technical Assistance Division. OJJDP

Mary Santonastasso, Project Officer

Technical Assistance

Polaris Research and Development

Noel Day, Project CoDirector David Sheppard, Project CoDirector Monseratte Diaz, Project Associate Michelle Magee, Project Associate Johannes Troost, Project Associate

4 ,ble of Contents

Part II: Interim Evalua s of Specific Projects

Compton Action Alternative :hool: Second Interim Report . 143 G. D. Gottfredson, D. C. Gottfredson, and M. S. Cook

Project STATUS: Second Interim Report 163 D. K. Ogawa, M. S. Cook, and G. D. Gottfredson

Project RETAIN, Chicago Beard of Education: Evaluation Report 197 J. St. John

The Milwood Alternative Project: Second Interim Report . . 227 M. S. Cook

Project PREP: Second Interim Report 267 D. K. Ogawa and G. D. Gottfredson

The Jazzmobile Alternative Arts Education Project 285 D. E. Rickert, Jr.

Otro Camino: Second Interim Report 323 J. St. John

Project PATHE: Second Interim Report 347 D. C. Gottfredson

Virgin Islands Alternative Education Project 393 J. St. John

Interim Quantitative Evaluation of the Academy for Community Education 407 D. E. Rickert, Jr.

Alternative Education for Rural Indian Youth: Lac Courte Oreilles 415 M. S. Cook

Plymouth Alternative Education Project: Second, Interim Report 439 M.S. Cook

Educational Improvement Center--South Alternative Education Project: Second Interim Report 471 D. E: Rickert, Jr.

Jewish Vocational Services Alternative Education Project: Evaluation Report 505 M. S. Cook Campton Action Alternative School: Second Interim Report

D. K. Daniels and G. D. Gottfredson

Abstract

The Compton Action Alternative School (formerly the Compton Action Center for Youth Development, CACYD) is a delinquency prevention demonstration project sponsored by the Office for Juvenile Justice and Delin- quency Prevention (OJJDP) as part of the OJJDP Program in Delinquency Prevention through Alternative Education. CACYD has evolved over the course of its first two years in ways that appear to have. strengthened it as a delinquency prevention project. Interim results based on student self-report suggest that the project has been remarkably effective in altering a number of student'characteristics that delinquency prevention theory implies must be altered to prevent delinquent behavior, and student self-reports of delinquent behavior are significantly lower than theself-reports of a control group. The self-report data must be interpreted with caution, however, because of some evidence of differential validity for treatment and control group members. Problems with the retrieval of some archival data on official delinquency and other outcomes limit the assessmentreportedhere. Newdatahavevery recently become available to strengthen the present analyses, and this report should be regarded as tentative. It is subject to revision as new results emerge.

The ComptonActionAlternative its second year,the Center placed School (CACYD) has gone througha emphasis on refining itsinterven- developmental sequencethat paral- tions designed to achieve affective lels the development its clients and behavioral objectives, and it must go through. That is,just as tried to teach its clients the its clients must first come to see behaviors associated with being stu- themselves as students and then dents,to have confidencein their learn the behaviors associated with abilities in the school environment, being students, CACYD first had to and to have confidence in the establish its identity as a school. staff's concern about their educa- Two years ago the CACYD was a youth tional progress.' and family counseling center called the Compton Action Center for Youth Development (hence the acronym) with a mini-school component providing only short-term services for Compton Unified School District (CUSD) students. Forthe first year it con- centrated its energies on becoming a This second interim report sup- school--and it attempted to teach plements an earlier report (Daniels, its clients that one of the things a 1982) which should be consulted for student does is come to school. In a basic overview of this project and an account of earlier activities.

1. In the third year (1982-83) the Center is emphasizing academics.

-143-

6 CACYD

Programmatic Changes in the Second As a part of the personalized Year educational approach, Individualized Education Plans (IEPs) were prepared The Center's clients are youths for all students by each instructor. who have not adapted well in a trad- The intention was for the plans to itional school setting. In all include an assessment ofeach stu- likelihood they were regarded by dent's skills at entry or the begintheir teachers in the public schools ning of the school year, individual as "slow" or "troublesome," student academic and behavioral objectives, characteristics generally regarded and plans to meet those objectives. as undesirable in public education. Theplans were to berevised and The Center's approach to these updated every five or six weeks. At clients is totellthem thatthey each periodic update, the plans were can learn. For those who are slow to specify whether the student was learners, instructors provide as meeting the objectives and to update much time and repetition of the the objectivesor the methodsand material as it takesfor them to materials to be used in the coming learn what is beingtaught. They ,,period to assist the student in also aim to provide enough opportu- achieving the objectives. The nities for them to succeed that they instructor was to inform each stu- can begin to realize they can learn. dent of the plan. A review of a Thisprocess is intended to lead sample of theseplansshowed that students to come to understand that the format did not include space for it is OK go be slow; and that it is behavioral objectives and that aca- important that they can learn demic objectives sought_and_plans_to nevertheless. To promote this achieve them often needed to be more learning, CACYD implements its con- precise and detailed. Most of these cept of personalized education, plans were updated regularly as This term means individualized edu- planned. Administratorsare aware cation with educational aims broadly that additional attention is needed construed, i.e., not limited to aca- to improve the implementation of demic concerns but also to the these IEPs. A revised form which is affective and behavioral domains. somewhat more specific in the infor- TheCenterassumes thatdifferent mation requested was introduced near motivators, teaching styles, and the end ofthe second semester for materials may be most effective for use in the third year. different students; and it assumes that different interpersonal styles Academic credit was granted may be effective for different stu- through the Compton Unified School dents.2 District. The curriculum included (a) reading, (b) English, (c) mathematics, (d) social studies, and (e) computer-assisted learning. Whereasin the first year survival skills were included as modules in each class, in the second year a 2. Unlike the first and second separate class was established that years,each student took a battery exposedstudents totasks such as of tests in the thirdyear (the thecompletion of job application Gates and California Achievement blanks, letter writing, completion Test). These testswere used to of rental application forms, and place students in achievement understanding gas and electric groups, a procedure also not used in bills. Unlike CUSD schools,where the first and second year. classes extend into the afternoon, 144 CACYD

at the Center classes are held until Justice System." 1:30 p.m.,including halfan hour for lunch. After the morning meeting,students begintheir academiccourse Interviews with staff, and direct work. Some, for instance, began observation, indicate that most work on tLe Plato (computerized instructors attempt to individualize instruction) system. The on-line instrtction, but additional work may Plato system used in the first year be needed to achieve individualiza- was replaced with micro-Plato which tion in all classrooms. Some staff uses discs. This smaller system have also expressed a desire for offers fewer programs than the lar- assistance in improving their teach- ger system, but eliminated problems ing methods,anda desireto make with phone lines that were experi- the curriculum more experiential. enced in the first yeai. A teacher supervises this activity, supple- The Typical School Day menting work on the computer with classroominstruction and practice The typical day starts off with a yin work books. Students spend time half-hour "morning meeting," or working with PLATO every day. assembly. This meeting was initiated as an incentive to get students Teaching Approach to school on time (and is also viewed asa vehicle for promoting No set formula for instruction is student participation and capacities provided for staff (three full-time to plan and identify problems). instructors, two half-time instruc- Staff members who rotate through tors, and a teaching aide), but gen- this responsibility share responsi- eral guidance has been provided. bility for running the morning meet- Four elements of effective teaching ing with two students. It is gener- have been communicated to the staff ally held asan open forum, where by the project director: students debate, make presentations, (a) "direct instruction," (b) time and plan for daily events. Issues to exercise the skills to be tran- to be taken up by the student smitted, (c) timespent in review government are developed in the and exercise of the skills, and morning meetings. Students planned (d) testing to checkfor mastery. andimplemented several activities In addition, several training ses- this year that were based on ideas sions for the staff in the Kawaida that arose inthese morning meet- method were held. The method ings. The largest such undertakings appears to be directed to enhancing were a prom andgraduation which self-esteem, identity, planfulness, they initiated and helped to plan, and responsibility on the partof and a week of debates known as "The students by encouraging instructors Great Debate." All students and to provide opportunities for the staff members participated in these mastery ofskills, recognition and debates. Thetopicsdebated were reinforcement of success, and accep- "Decriainalizing Marijuana," "Pre- tance. The method was developed by Marital Sex," and "The Criminal Gary and Maye Beale who drew heavily onan unpublished dissertation by Joseph White. The materials we have seenseem to draw heavily onthe 3. In the third year a student popular writingsof Glasser (1969; government has assumed the responsi- 1975). bility for running the morining meeting.

-145- CACYD

Additional Educational Features implement activities such as a fund- raiser for a field trip and the Several kinds of activities occur revision of roles that govern stu- either periodically or occasionally dent behavior, to supplement the day-to-day routine: Maintenancloyment_ activity. As one way to help solve a graffiti Career day. The Center's commu- problem during the second semester, nity advisory committee helped spon- two students rotated through jobs to sor a career dayin which a number keep the Center clean. Students of professionals, including a police must sign in, and they are expected officer, came to the school to dis- to learn conventional work habits. cuss their work with the stu' nts. They are paid for two hours work and The staff felt an important feature volunteer for tWohodrs of work. ofthis career day was that it was The students bolding these jobs are the first time that many profession- recommended by the staff,and this als had an opportunity to talk to work assignment Is sometimes used as so-called "bad" students in a posi- ,a reward for achieving a goal or as tive setting, and vice versa. an incentive to improve when a student is not achLeving goals. Informal student-staff inter.c- tion. As an approach to providing Mentors. Attempts were made rewardingsocial interactions bet- againto implement the mentor Pro- ween students and teachers that are gram thatwas unsuccessful inthe not provided intraditional school first year._ Agaih,this attempt was structures, staff are expected to not totally successful, although spend time with students by engaging some community involvement was in non - academic interaction. An achieved. Otte UCLA student con- example ofthis form of interaction ducted two projects with students, is the potluck lunches and dinners but attempts to recruit mentors from involving parents and students. other closer colleges failed,partially because ofcutbacksin work Formal interaction. Field trips study funds and Partially because of to an amusement park, a museum, some a reported lack of interest on the local colleges and universities, and Part of the college students. Prior tosports and entertainment events to cuts in CETA funding, some CETA were carried out. workers were employed by the Center to work with ytudents in the mentor A partialtoken economy. Near role. Although students apparently the end of the second year, the Cen- related better to CETA workers than ter began to implement a partial to College students, and students token economy system. Students undertook projects such as con- could earn tokens for class atten- structing Easter baskets for disa- dance and participation that could bled children, funding cuts pre- be redeemed in a recreation area to vented this activity from play pool or engage in other activi- continuing. ties. Task orientarion and atmosphere. Studentgovernment. Astudent Two additional features of the edu- government was formedwith eight cational environment were salient in elected members. The government students' responses to an interview initiated ad hoc committees involv- question. When asked what they ing other students to plan and would tell someone about the Center

-146-

9 CACYD if askedto describeit, the most that parents seem to respond better common responsewas that students to requests to participate in school have to get their work done every- activities than to requests to day. Student responses to interview attend meetings. Accordingly, questions also emphasized that they parents have been encouraged to felt good about themselves and the engage in activities such as the staff, that they felt relaxed, as career dayand a car wash. This though theycould learn,andthat appears to be a fruitful approach, they were surethe staff wascon- andsuggests thatparent involve- cerned about them learning. ment inschool activities might be fostered by providing them with Parent and Community Activities meaningful roles in the conduct of a school's activities. An attempt has Parents. Formal parent training also been made to -get parents was planned, and six workshops were involved in the studentcontracts held. The topics covered were "How involving educational plans, but the CUSD Functions," "A Candidates this has not met with great success Forum for Persons Running forthe ,nor been attempted as regularly as CUSD Board of Trustees," "Parenting had been envisioned. Adolescents," "Preventing Drugand Alcohol Use by Teens," "Gang-O- Community. A 13-member community riented Youth," and a "Family After- advisorycouncil hasbeen formed. noon." The last of these was a It includes a Compton attorney, pot-luckmeal shared by parents, business persons, representatives of students, and staff at which the the probation and narcotics parole workshop series was reviewed. The departments, a clergyman, a Congres- response on the part of participants sionalstaffer, and two students. was reportedly favorable, butonly We have not interviewed any of these six or seven parents participated in individuals about their roles,but all the workshops. (The first two thefunction of the committee is workshops drew approximately 35 par- reportedly to give advice,to pro- ticipants.) Considerable parent vide an indication of community sen- contact and involvement was achieved timent, and to helpdevelopgood in other ways. This included commu- will for the Center and help it to nication with parents when students solve problems. were absent,a parent council,and parent participation in Center The Center encouraged informal activities. community participation in other ways. For example, a community mem- Parents were contacted bytele- ber dropped in, volunteered to phone when students were absent. tutor, and did so for a number of Although this is a difficult activ- weeks. And a local liquor store ity to conduct, and although many owner contributed toandpartici- students of the Center live in non- patedin student-sponsored activi- traditional families, this was a ties. regular and recurring activity. Other Project Improvements A seven-member parent council held meetings about once per month Staff turnover was high inthe during the year. This council was Center's first year. Staffing was primarilya discussion and support stable in the second project year. group for parents. One teaching aide was dismissed becauseshe was associated with a The project manager has observed

-147-

10 CACYD gang, and the Center wantedto be its operation are decreasing somew- perceived as neutral with respect to hat and it is acquiring some of the competing gangs in the community. trappings of an institution. Quarterly staffing reports show that, exclusive of CETA workers, the The only noticeable opposition to project had 8.4 full-time equivalent the Center's activities has come workers through May,1982, with no from the police who feel that addi- turnover during the year until that tional security is required for the time. TLe projectstaff decreased Center to remain in operation in its by one person in May, one full-time present location. (There was one and one 70% time ,worker June, and serious gang intrusion into the cen- one person (the principal) in ter this year.) The Center's advi- August, 1982. Twofull-timeCETA sory committee has been of some help workers were lost in May when their in interacting with the police time allotment in the program department. In addition,the Cen- expired. ter's participation together with the police inthe. District's Clean The Center has engaged in plan- ,Sweep Program has increased respect ning using the PDE method more fully for the Center on the part of the in its second year of operation. It police,as well as other community has developed its activities accord- entitites. One CUSD school princi- ing to a more fully specified plan, pal has shown some opposition, but and it has involved more staff mem- this is not considered serious. bers regularly inthe formal planning process. Although all staff In the second _year, the_Center's members do not view the PDE method clientswerecarried on the CUSD as equally useful, nor do all use it rolls and received lunches through a in equal amounts, most staff members CUSD program. There are additional know about the Center's PDE plan and signs that the Center'srelations were ableto explain how planning with the district are becoming more following this method works. positive. Project managers now know more key people in the district and Difficulties with on-site evalua- the new acting superintendent (AS) tion activities experienced inthe is supportive of the Center's first year have ceased, and the on- efforts. An interview with the AS site evaluator is doing an excellent implies thathe viewsCACYD very job of implementing evaluation-re- positively and sees theCenter as lated activities. workingcloselywithhis schools. The Center's manager is a member of Progress Towards Institutionaliza- his Blue Ribbon Committee on School tion Change (chairing its subcommittee on

/ educational change),and hasbeen We define the institutionaliza- active in conducting surveys in CUSD tion of a program as a condition in schoolsintended to be helpful in which no special effort is required planningprograms to prevent CUSD to maintain its operation as a part teachersfrom alienatingstudents. of the status quo, and in which con- The Center's principal isa member siderable effort would be required of the district's disciplinary to end or modify the program. By review committee. The AS plans to this definition, the Center is seek CACYD assistance in implement- clearlynot yet institutionalized. ing and monitoring CUSD's efforts to It appears to be making progress increase individualization in towards institutionalization in the instruction. sense that obstacles to

-148-

11 CACYD

The Center managers andthe AS involuntary withdrawn (primarily both seetheCenter as an entity incarcerations), 15% of partici- that should operate outside the pants; left Compton, 12%; voluntary public school system. The Center is withdrawal (dropout), 4%; return to intended to have a role of its own regular school, 4%. butto operate in cooperation with the CUSD to accomplish some common Ofthe 91students for whom we goals. Although the CUSD has pro- have data who participated in CACYD videdsome servicesand furniture in the first year, 41 (45%) returned for CACYD there are no plans for the to the Center atsometime during district to provide any substantial the second year. Of these 41 retur- financialsupport. The AS reports nees, 66% were stillin attendance that he would help with funding if at the end of the second year, 22% he had the funds, as itis he sees wereinvoluntary withdiawals (pri- his role as helping the Center marily. due to incarceration), obtain outside funding. 5% left Compton, 5% voluntarily withdrew, and 2% returned to a regu- Decreasing funding through the lar school. This pattern of student OJJDP grant may influence the pro- attrition implies that few students ject's ability to operate. Further- are voluntarily dropping out of more cuts in work-study funding for school,and the largestsource of colleges in the area,and cuts in attrition is incarceration ofstu- CETA, have decreased the availabil- dents in this high-risk population. ity of low-cost supplementary workers. The poor economic climate and Methods decisions to reduce the Federal role in thesupport of social programs Assignment to treatment and con- will almost certainly work to reduce trol groups. Except for returning the likelihood that the Center will students (N = 41), students eligible achieve institutionalization. for participation were assignedto treatment or to an untreated control group. Because of pressure from. Interim Summative Evaluation OJJDP to rapidly increase the number of participants in the school, how- Problems experienced in imple-/ ever, assignment was not straight- menting the evaluation in the first forward. Pressure to rapidly bring project yearhave been resolved. the number of students served up to Project personnel have zea- an acceptable level required the lously pursued evaluation activities following procedure: Between the with the result that the evaluation start of the schoolyear and 14 design is strong and has been well December 1981, a computer generated implemented. In all 80 students list of pseudorandomnumbers was participated in the Centerduring provided tothe on-site evaluation the 1981-82 school year. Of these coordinator. The list contained 48 (60%) were still in attendance at lines a random 80% of which carried the end of the school year.4 Attri- a code for treatment condition and tion isaccounted for asfollows: the remaining 20% a code for control condition. As youths became eligible for the program, theirnames were alphabetized andwritten in 4. Our best estimate of the percen- that order on the list. This proce- tageof participants who persisted dure was i-tended to randomly assign until the end of the 1980-81 school 80% of el 7e youths to treatment. year is 58%.

-149-

12 Some Ito Coster woe approachise the A difference was also found mentor geettetleste required by beton* tteatemot god control groups She ersomser. the esti* of treatises* fa umbers of police contacts bet- to seletrel osetammeete om the list imes 16 September 19110 amd 22 August Mtge rowersed=00% were aseigmed to (i.o., the year Woe assign- the owed grow nand 202 to the meet).The trestmeat group (N 38) 441104,404 gO0414 Is ell, $ teethe had a mese of .16 police contacts in were smetgood from a pos4 et ellgti. that year sod the control group (X Mae to perttsipste to the %eater. 32) bad a mesa of .60 police con- and 14 tote deetgeel to as seltreated tacts. Treatment students had fever isootrs4 grasp Swim the first prior police contacts than control 044410MOAt period described above, a youths in both the early and late total et 23 paths wore wiped to assigemest periods, and the differ- treeivemt had 10 palm were ence is statistically significant (e etstemod tae the mourn group, %r- t .004) eve* when assignment period ime the wooed aseteemeet period, 14 is statistically held constant. A paths gore aesigmed to treatises' close examination of the diatribu- anus 44 tore asetsmod to antral. ,tlone implies that this difference Ibie depertwee free stool* realm can mot be accounted for by outl i- esoLgomeist complicate*the desks ers. forthe treatment group the eamewhet4 number of prior police contacts rafted from 0 to 4, with 55.3% having no prior contacts Le the period 4041451112k0 tollto to oil:: examined. For the control group the police _contacts 4441004 t4 04f the peel 44 eligibles *ember , of , prior, *my here differed be early and ranged from 0 to 7, with 47% having late aseigameet portfolio. (These so prior contacts in the period exa- eherhe wee mesoosety beams* treat- mined. The treatment and control meet Mateo to of comae associated groups differed somewhat in ethnic with ported of assignmeet.) Tarte- convention but this difference hlee so ibteh molter end later appear, due to a difference in the speep* differed,eatme treatment composition of the eligible pools in tem, ere give* (with g levels the early and late assignment por- to preethesee) to thefolly/lag ted.. The treatment group was 78% Wets withdtawel from school (.02), Stack, and the control group was 91% Wise 6004$444 ditteth000t to black. This difference is not sig - tweets (.00), SOW Wert (.05), eificant when assignment period is when amino; (.05), Parental %de- statistically controlled. salt** (.fl), a somposite lodes of Sibeel Ormatteadsocethat is par- We do not know to what to attri- ttelly redeadoot with *then undue bute the observed difference in (454 Sehelliees Mom* (.0%), pre - intervention police contacts. sod lavolvemoot (.01). faplot* One possibility is that insufficient iloglish, the pool el eligibles dif- time was alloyed to elapse between fered is impertaat weys between the the dam a police contact may have early mod late *Mesons period. occurred andthetinethepolice This mess that valets the assign - records were searched for evidence It is possible that as period is Wife into account, ofarrests. treatimmet and mitre' groups would because a disproportionate number of be etas to differ due to the control youthi entered the experi- lays they were emaposed is addition Mont in the late assignment period, toaoy differeesce due to trestmext. more of their arrests werereflected in the data base because a longer

-ISO-

13 CACYD period oftime had elapsed between This instrumentis described else the arrest and the time the files where (Gottfredf,on,Ogawa, Rickert, were searched. On the other hand, & Gottfredson, 1982), and readers the difference was observed for should consult that description for early and late assignment periods. an account of some of its psychome We willcontinue to explore this tric properties. This questionnaire issue and conductnew searchesof was administered by the onsite the police contact data base to evaluation coordinator in small check on this hypothesis. groups or individually as necessary. The administration was closely Analyses. To assess the effects supervised and resulted in high res of participation in the Campton pro ponserates' for bothtreatment and ject in the most direct way possi control groups, and a low percentage ble, a comparison was made of stu of item nonresponse. Control group dents who were assigned to members were offered $5.00 for com participateand thecontrol group ingto a questionnaire administra taking into account the assignment tion. A total of 74% of treatment period during which they entered the ,and87%of control students were experiment.5 Where deemed appropri administered the questionnaire. ate, statistical adjustmentswere also made for prior police record. In addition, the onsite evalua tioncoordinator collectedseveral Measures. A singleinstrument, other pieces of information from the School Action Effectiveness school and police records. This Study Student Questionnaire was used information included police records to collect information on both and school attendance and withdrawal treatment and control students. information. The measures examined

5. In technical terms, the approxi sary to search for a main effect of mate equivalent ofa 2 by 2 facto treatment onlyafter adjustingfor rial design with unequalcells was assignment period because assignment accamplished using theSPSSANOVA period was significantly associated procedure with option 10 on release with a number of outcomes even when 9 ofthat package (Nie, Hull, Jen treatment was "controlled" out. We kins, Steinbrenner, &Bent, 1975; regard this procedure as a conserva Hull & Nie,, 1981). This procedure tive test of treatment effects involves a regression approach, because, as a practical matter, the where the statistical test for main period during which students are effects involves the incremental assigned to treatment is associated contribution ofeach factor net of with theduration of exposure to prior factors (in this case the treatment. Thus, applying a statis assignment period duringwhich a tical control for period of assign person entered the treatment or con ment tends to work against finding trol groups) and covariates (prior treatment effects if effect siie"is--- police record when appropriate). associated withduration ofexpo The interaction effect (treatment by sure, as might be expected. Depen assignment period) is tested after dable differencesassociatedwith main effectsareenteredinto the bothtreatment condition and per model. If an interaction effect iod of assignment have been assigned significant at the p = .10 level or to period in the analyses. This is less was found, data were analyzed unfortunate, but protects the separately by period. It was neces- believability of the statistical -151

14 CACYD in this report are briefly described trouble, do not like school, etc. below: Attachme4 to school. This is a Withdrawal from school. Withdra- scale based on student reports that wal from school, either voluntary or they like the school. It is involuntary,is coded 1; any other intendedto measurean element of observed outcome (left Campton, Hirschi's (1969) social bond. returned to school, or persisted in CACYD) is coded 0. Belief in rules. This is a scale based on student reports that taking Any police contact. Any evidence advantage of others and breaking of policecontactforanyoffense rules or laws is not OK. during the period 16 September 1981 through 1 August 1982 according to a Interpersonal competency. This search of arrest records in the is a questionnaire-based measure of Hall of Records auto- psychological health based on Hol- mated information 'reftieval system land and Baird's (1968) scale with is coded 1. No evidence of police .,the same name. contact during that period is coded O. Access to these records was Positive self-concept. This gained through a court order. scale measures students' self-esteem and conception of themselves as pro- Number of alleged offenses. This social, law-abiding citizens. is the total number of offenses alleged during the period 16 Septem- Self-reporteddelinquent behav- ber 1981 through 1 August 1982. ior. This scale is based on 19 items asking whether the respondent Attachment to parents. This is a has committed various kinds of del- questionnaire-based measure of inquent behavior during the past "attachment," i.e. feeling close and year. wanting to be like parents. It is intendedto measurean element of Self-reported substance use. Hirschi's (1969) social bond. This is a subset of the previous scale, limited to the use of drugs, Negative peer influence. This is alcohol, and inhalants. a scalebased onstudentreports thathisor herfriends get into Self-reported serious delinquent behavior. Thisis a subset ofthe self-report delinquent behavior scale limited to the most serious of conclusions drawn. the offenses in that scale.

Multivariate analysis of variance punishment. School This scale is was precluded because ofexcessive based on students' reports that they missing data. Our inability to pro- have been punished in various ways tect the univariate testsbyper- in school for their recentschool forming multivariate, analysis of behavior. variance first means that some differences may be expected by chance School rewards. Thisscale is alone. We interpret_univariate dif- based on students' reports that they ferences nonetheless, and this len- have been rewarded in various ways iency seems to us to counterbalance in schoolfor their recentschool the conservative handling of nonor- behavior. thogonality in the design.

-152- CACYD

Victimization. This scale Suspension. This is a measures recent in-school victimiza- self-report of suspension from tion experiences. schoolduring thecurrent(Spring) term. Invalidity. This scale is intended to detect nonveridical res- School non-attendance index. ponding. Reporting that one reads This is a two-item measure of class several whole books every day and cutting and school cutting. that it is easy to get along with nasty people earns a high score. Rebellious autonomy. This scale is intended to measure an immature School effort. This is a questi- attitude of autonomy. A high score onnaire-based measure of effort is earned by reporting that it is no expended at school work. one'sbusiness how the respondent spends his or her money,for exam- Practical knowledge. This isa ple. 7-item self-report measure of knowledge of several life competencies, Involvement. This is a question- such as how to balance a check book, naire-based measure of participation arrange a trip out of town, or apply in same common school activities. for a job. Results Alienation. This is a self-report measure of feelings of Treatment-control group compariL estrangement from the social order. sons are presented in Table 1. Of 25 comparisonsbased on question- Self-reported grades. This sin- naire measures, 18 significantly gle-itemmeasure is self-explana- favor the treatment group.6 One of tory. these significant differences favors the control group. In one case a Self-reported reading ability. statistical interaction was found, This single-item measure is self-ex- meaning that the results differ planatory. depending on whether the youths were assigned to treatment or control Educational expectation. This is conditions in the early or late the response to a questionnaire item assignment period. asking how far the person expects to go in school.

Work forpay. Thismeasure is scored "1" if the respondent reported that he or sheworked for pay during the past week,and "0" if 6. These significance tests are not the respondent indicatedthat he or independent. Indeed, someof them she did not. are largely redundant (specifically the three self-report delinquency Regular bob. This indicatoris measures, two of which are based on coded"1" if the respondentindi- subsets of items in the overall mea- cated that he or she holds a full- sure). Therefore thesignificance or part-time job, and it iscoded levels shown in the table should be "0" if the respondent indicates that regarded as nominal rather than he or she does not. actual levels. Some of these significant differences may arise due to chance alone.

-153-

16 CACYD

No significant difference occurs result implies that the credibility for any of the measures basedon of the self-report delinquency mea- archival evidence shown in Table 1 sure ia,highly questionable. (withdrawals, police contact, number of police contacts, suspensions, or Several other checks on the val- grades).7 The treatment and control idity of self-report for the treat- groups were vastly different in ment and control groups are not officially reported suspensions, reassuring. These checks are sum- however. CACYD reported only six marized in Table 2. Percentage of suspensions in the entire year. In daysin attendance calculated from contrast, 81.2% of control group archival datasubmittedfor CACYD youths were suspended at least once participants for the final quarterly according to school records. reporting period by the Center cor- (Because of differences in data col- relates .31 (N = 29, Q < .10) with lection methods this resultis not the School Nonattendance index based tabled, and we have not calculated a on self-report. By comparison, significance test, but none is total days of absence during the necessary. The difference is strik- ,year based on CUSD records corre- ing.) lates .23(N = 41, n.s.) with the School Nonattendance index. Self- A threat to interpretability. reported reading ability correlates The one significant difference -.07 (N = 29, n.s.) with scores on favoring the control group presents a reading test the project used to an important threat to the interpre- assign students to levels in the tation of the questionnaire results. Plato system and_.05_(N = 24) with Treatment students scored higher CACYD grades. No reading test was than control youths on the invalid- available for the control group, but ity scale, a measure of non-veridi- self-reported reading ability corre- cal reporting. Accordingly addi- lates .16 (N = 36) with CUSD grades tional checks of the validity of the for this group. Self-reported self-report measures were made sepa- grades correlates with grades rately for the treatment and control according to officialrecords -.21 youths. For the control group, (N = 25) and .10 (N = 37) for treat- the self-reported delinquent behav- ment and control groups, respec- ior scale correlates .54 (N = 34, 2 tively. Finally, CACYD reported no < .001) with the number ofpolice administrative removals at all dur- contacts in the 81-82 period. For ing its last reporting period, but the treatment group, this corela- 28% of CACYD participants (N = 32) tion is -.01 (N = 23, n.s.).° This reported having been suspended during the last school term. It is not

7. There is reason to believe that some errors may have been made that data. Due primarily to selective led to an underestimate of the num- non-response to these particular ber of withdrawals for control group items, delinquent behavior self-re- youths. We are continuing to ports were available only for 59% of explore the completeness of the data treatment students (N = 39) and 64% in this regard. We are unable to of control group members (N = 54). compare treatment and control groups Thisdifference is nonsignificant. on archival attendance data at this In contrast, police records were time. We are also continuing available for almost all treatment attempts to explore these data. (97.4%)andcontrolgroup (98.1%) 8. TheN's vary due to missing youths.

-154- CACYD possible,of course to calculate a inspection of the questionnaire correlation when there is no varia- booklets imply that they were care- tionin oneof the variables. Of fully filled out and few items are control students, 81.2% (N= 48) skippedor markedin obvious pat- were suspended at some time during terns. Furthermore, the differen- the 81-82 school year, and 78% (N = tial validity found for treatment 45) reported having been suspended and control groups for question- during thelast schoolterm. The naire-based measures for which there correlation between thesetwo mea- existsa parallel in thearchival sures for the control group was only data accords with a hypothesis that .04 (N = 43, n.s.). treatment students attempted to create a good impression or other- In short, the evidence supporting wise responded in nonveridical the validity of self-reports for ways.9 And, the questionnaires were both treatment and control groups, administered by a member of the Cen- but especially for the treatment ter staff who had frequent contact group, is weak. In the case of sus- with the students and whom they may pensions, both self-report and ,not have wanted todisplease. On archival data converge in implying the other hand, respondents faking much lower suspensions among the good might be expected to have CACYD participants, although the reported fewer schoolpunishments. level of suspensions implied by the The opposite result was found: two data sources differs. Scores on the School Punishment Scale are significantly higher for Discussion of Interim Summative the treatment group. These,, and a Results number of other interpretations, are possible. CACYD staff, for example According to the questionnaire havesuggested anexplanation for measures, participation in CACYD has the low correlation between actual remarkablypositive outcomes. The grades and self-reported grades that resultsbased oninformationfrom involves the way student perceptions archives are not so positive. of their performande vary over time. Until further work is performed, Limitations. Several limitations many such interpretations are possi- are important in assessing these . ble. Onbalance, the self-report results. The first, and most obvi- results should be interpreted with ous, is that the administration of caution. the questionnaire by a member of the CACYD staff may have created a subtle and unintentionalbut powerful demand for treatment youths to present themselves in a positive light. 9. Hindelang, Hirschi, and Weis This demand may have been absent or (1981) found differentialvalidity less powerful for control group for self-report measures similar to youths. Evidence suggests that this those examined here in their study is a plausible interpretation of the of Seattle youths. Specifically, results. Treatment students' scores they found near zero validity coef- on the Invalidity Scale were signi- ficients for Black males who were ficantly higher than control youths officially delinquent. The pattern scores. This scaleis intended to found here suggests limited validity be sensitive to careless responding for the control group (many of whom or to an attempt to fake good. The were officially delinquent) and even elevated scoresdo not seem to be lower validity for for the treatment due to carelessresponding--visual group.

-155-

16 CACYD

A secondlimitation stemsfrom Implications. Although the the finding that treatment and con- interpretation of the self-report trolyouths differedsignificantly results is hazardous, the interim in the number of policecontacts summative results based on the during the year prior to the study self-report measures suggest not period. The same pattern (control only that CACYD participation has youths being more officially delin- been remarkablyeffective in pre- quent) holds for both the early and venting delinquency,but also sug- late assignment periods. This out- gest some explanations of its effec- come couldoccur by chance given tiveness. Several known correlates random assignment, or it could be a of delinquent behavior were affected product of a breakdown in the ran- by the CACYD treatment. These domization process." include belief in rules, attachment to parents and school,"self-concept A third, andless threatening, (esteem), and involvement (measures limitation is due to the assignment of the social bond according to of different propOrtions ofeligi- social control theory); negative bles to the treatment condition at ,peer influence (a measure of differ- different time periods. Despite the ential association); and school application of a statistical control grades. Furthermore, the greater forperiod of assignment, because amount of school rewards and punish- the composition of the pool of eli- ments reported by treatment students gibles'differed between the two time suggeststhat supervision and the periods some residual spurious asso- consequation ofbehavior maybe a ciation between youth characteris- major wayin which_ these _outcomes tics and outcomes may existin the can be achieved. Although site results. An examination oftreat- visit observations and interviews mentand controlmeans separately with participants are by themselves for each time period suggests that generally untrustworthysources of this threat to interpretation is not evidence, observations and inter- highly plausible as an explanation views with students participating in of the results. the program nevertheless lend support to this interpretation. Like- wise, observations of phone calls to parents when students are absent and of deliberate attempts of the staff 10. The application of a statisti- to strengthen interpersonal ties cal control for prior police records with the students also support this does not alter the statistical con- interpretation. clusion: netof assignment period and of prior police record treatment Discussion of Other Observations youths report fewer kinds of delinquent behavior than do control Our observations suggest some youths. Police records are not, ways the project might be strength- however, stable covariates. The ened. First, the individualized correlation of 80-81 police contacts education plans we examined can be with 81-82contacts is low. The strengthened greatly by(a) making application of a statistical control themmore concrete and specific, for prior grades does not alter the (b) reviewing and updating them more statistical conclusion: net of frequently, (c) providing specific assignment period and of prior consequences for meeting or failing grade-point average, treatment to meet short-term objectives, youths report receiving higher (d) ensuring that the objectives and grades. consequences are clearly understood

-156-

19 CACYD

by students,staff, and parents or 4. The relationship between the guardians. Center and the Compton Unified School District was strengthened. Second, althoughtheprinciples of effective teaching suggested by 5. The Center began the proce- the project manager accord with some dure of starting its school day with literature on effective teaching a youth initiated activity--a crea- (e.g., Brophy & Evertston, 1974; tive and interesting innovation. Crawford & Stallings, 1978;Gage, 1978) a formal structure for ensur- 6. Studentreportsimplythat ing thatthese principles are fol- they felt positive about the alter- lowed precisely should be imple- native school andthat they knew mented to strengthen theacademic they were expected to learn. component. 7.. Warm interpersonal relations Third, observations and inter- between students and staff were views with staff suggest that developed. improvements in curriculum and teaching technology could strengthen 8. Despite serious problems with the academic components. The use of gangs in the Compton community, stu- well engineered educational technol- dents from rival gangs generally ogies such as DISTAR (Becker & Car- co-existed peacefully in the Center. nine, 1980; Becker & Gersten, 1982) or various approaches to increasing Recommendations the effectiveness of classroom reward structures (Slavin, 1977, Our observations and analyses to 1982,in press) could add strength date suggest the following recommen- in this area. dations:

Fourth,the intiation ofa par- 1. Greater use of previously tial token economy appears to be a developed educational interventions useful step. This step couldbe and curricula to increase learning more fully elaborated along the would probably strengthen the pro- lines developed in Achievement Place ject's academic component. (Wolf et al.,1972; Levitt et al., 1981). 2. The development of additional strategies to retain clients in the Brief Summary of Achievements in the program for a longer period of time. Second Year 3. More active steps to secure 1. Staffing was stableduring sponsorship forthe Center in the the second year, with no person future must be, taken if it is to / leaving the Center until May of that continue to evolve. year. 4. Administration of student 2. Of students who participated questionnaires in the Spring by a inthe Centerduring the 1981-82 person unknown to the respondents, school year, 60% were still enrolled especially for the treatment group. at the end of the school year. 5. Closer replication oftoken 3. The collection of information economy systems developed and tested by the on-site evaluator was tho- elsewhere to strengthen this project rough and timely. component, if it is to be continued.

-157-

20 CACYD

6. Strengthening the serviceprovision do not conflict individualized education plans by with the demands of an orderly eval- much more frequent review and uation. greater specificity to increase their faithfulness to the character- 8. In future efforts in this istics of such plans described ear- area, making arrangements for ran- lier. domization to be conducted off-site withina feasible time line, and 7. In future efforts in this allowing a longer period of time to area, making arrangements in advance elapse before searching archives for such that demands from a sponsor for arrests.

-158-

21 CACYD

References

Becker, W. C., & Carnine, D. W. Direct instruction: An effective approach to educational intervention with the disadvantaged and low performers. In B. B. Lahey & A. E. Kazdin, Advances in clinical child psychology (Vol. 3). New York: Plenum, 1980.

Becker, W. C., & Gersten, R. A follow-up of follow through: The later effects of the direct instruction model on children in fifth and sixth grades. American Educational Research Journal, 1982, 19, 75-92.

Brophy, J. E., & Evertson, C. M. Process-product correlationsin the Texas Teacher Effectiveness Study: Final report (No. 75-21). Austin, Texas: Research and Development Center for Teacher Education, University of Texas, 1974.

Crawford, J., & Stallings, J. Experimental effects of in-service teacher training derived from process-product correlations in the primary grades. Paper presented at the Meeting of the American Educational Research Associ- ation, Toronto, March 1978.

Daniels, D. K. Compton Action Center for Youth Development: Interim report. In G. D. Gottfredson (ed.), The School Action Effectiveness Study: First interim report. Baltimore: The Johns Hopkins University, Center for Social Organization of Schools, 1982.

Gage, N. L. An experiment on teacher effectiveness and parent-assisted instruction in the third grade: Background and rationale. Paper presented at the Meeting of the American Educational Research Ass6ciation, Toronto, March 1978.

Glasser, W. Schools without failure. New York: Harper & Row, 1969

Glasser W. Reality therapy: A new approach to psychiatry. New York: Harper & Row, 1975. (First published in 1965.)

Gottfredson, G. D., Ogawa, D. K., Rickert, D. E., Jr., & Gottftedson, D. C. Measures used in the School Action Effectiveness Study. In G. D. Gottfred- son (ed.), School Action Effectiveness Study: First interim report. Bal- timore: The Johns Hopkins University, Center for Social Organization of Schools, 1982.

Hirschi, T. Causes of delinquency. Berkeley, Calif.: University of Califor- nia Press, 1969.

Hindelang, M., Hirschi, T., & Weis, J. Measuring delinquency. Beverly Hills, Calif.: Sage, 1981.

Holland, J. L., & Baird, L. L. An interpersonal competency scale. Educa- tional and Psychological Measurement, 1968, 28, 503-510.

-159- CACYD

Hull, C. H., & Nie, N. H. (eds.). SPSS update -?721AXPAcedures and facil ities for releases 7-9. New York: McGrawBi11, 1981,

Levitt, J. L., Young, T. M., & Pappenfort, D. M. Alievesiet_..qs1.ace:Thecl teachin famil treatment model in arou home setti.gg (Reports of the National Juvenile Justice Assessment Centers). Washington, D.C.: National Institute for Juvenile Justice and Delinquency PreventioA, Office of Juven ile Justice and Delinquency Prevention, 1981.

Nie, N. H., Hull, C. H, Jenkins, J. G., Steinbrenner, K., & Bent, D. H. Sta tistical acka e for the social sciences (Second edition). New York: McGrawHill, 1974.

Slavin, R.E. A student team approach to teaching adolegeetIts with special emotional and behavioral needs. .PsysilsIcla_kn the Schools, 1977, 14, 77-84.

Slavin, R. E. Critical dimensions of clapsruom organization: A strategy for researchbased instructional improvement. Baltimore: Johns Hopkins Univ ersity, Center for Social Orgnization of Schools, 1982. (An earlier ver sion of this paper was presented at the annual meeting of the American Edu cational Research Association, New York, 1982.)

Slavin, R. E. Cooperative learning. New York: Longman's, in press.

Wolf, M. M., Phillips, E. L., & Fixen, D. L. The teaching Anew model for the treatment of deviant child behavior in the community. In S. W. Bijou & E. L. RibesIresta (eds.), Behavior modification. New York: Academic Press, 1972.

160 Table 1 Overview of Tests for Differences between Treatment and Control Conditions

Criterion Difference variable favors F df

Archival sources Withdrawal control 0.08 1,67 .78 ns Any police contact control 0.23 1,89 .23 ns Number of treatment 2.69 1,87 .10 as police contacts Grade-point average control 0.84 1,89 .36 ns Self-report Attachment to parents treatment 6.84 1,68 .01 ** Negative peer infl. treatment 0.27 1,68 .6O ns Attachment to school treatment 46.81 1,74 .001 ** Belief in rules treatment 5.30 1,70 .024 * Interpersonal treatment 5.60 1,72 .021 * competency Positive self-concept treatment 2.04 1,65 .16 ns Self-reported treatment 7.62 1,54 .008 ** delinquent behavior Self-reported treatment 5.46 1,56 .023 * substance use Self-reported serious treatment 7.44 1,54 .009 ** delinquent behaviors School punishment treatment 6.00 1,73 .017 * School rewards treatment 6.94 1,74 .010 ** Victimization control 1.83 1,72 .18 ns Invalidity control 4.21 1,71 .044 * School effort treatment 8.45 1,71 .005 ** Practical knowledge treatment 10.55 1,74 .002 ** Alienation treatment 4.88 1,67

Self-rep. grades treatment 1.28 1,74 .26..(;13 n: Self-reported interactionc reading ability Educ. expectation treatment 1.69 1,74 .20 ns Work for pay treatment 6.59 1,74 .012 * Regular job treatment 1.94 1,74 .17 ns Suspension treatment 24.08 1,73 .001 ** School non-attendance treatment 4.31 1,71 .042 * index Rebellious autonomy treatment 20.81 1,70 .001 ** Involvement treatment 17.31 1,71 .001 **

Note. Based on analyses that test for an effect of treatment net of period of assignment to treatment or control groups (see text footnote 5). The withdrawal variable may not be comparable for treatment and control groups (see text footnote 7). a Part of total self-reported delinquent behavior. Treatment group reports more school punishment. Nature of effect depends on period examined. * Statistically significant ns Not significant ** Highly statistically significant as Almost significant

-161- 24 Table 2

An Examination of Differential Validity of Self-Report for Treatment and Control Groups

Self-report Archival measure measure r N

Treatment group

Delinquent behavior Number of police -.01 23 total contacts

School nonattendance Number of unex- . 31 29 index. cused absences, ,,April-end of year

Reading ability Plato reading -.07 29 pretest

CACYD grades .05 24

School grades CACYD grades -.21 25

Control group

Delinquent behavior Number of police .54 34* contacts

School nonattendance Number of absences . 23 41 CUSD, 81-82

Reading ability CUSD grades . 16 36

School grades CUSD grades .10 37

*p < 01 Project STATUS: Second Interim Report

D. K. Ogawa, M.S. Cook, and G. D. Gottfredson

Abstract

Project STATUS (Student Training Alternatives Through Urban Strate- gies) is continuingits efforts at Eliot Junior High and Muir High School in Pasadena, California toreduce classcutting, disruptive behavior, dropout from school, and lack of student and parental involvement in the school. The project's five interventions: a) the Options class, b) the Youth Committee and Leadership Training Class, c) project training, d) parent involvement, and e) the Action/Advisory Committee, are designed to provide students with a more meaningful educational program. More specific project objectives are todecrease alienation, increase attachment to school, self-esteem, and academicsuccess.

Non-equivalent control group designs are used toassess the effectiveness of the Option classes and Youth Committee at Eliot, and the Options classes at Muir. No evaluations are currently available for the other program components. Results show students in Eliot's Options class to be significantly lower in alienation, make higher ratings of their own reading ability, report more fairness and clarity of school rules, and have fewer withdrawals from school than students in anon-e- quivalent control group. It apparently also increases absenteeism for project students. No evidenceof .effectiveness was found for Muir's Options class or for the Youth Committees at Eliot or Muir. We cannot as yet judge the effectiveness of parts of the project; numerous prob- lemsin implementing some project components mayhave weakened the effectiveness of the planned interventions. Next year's efforts will include consolidating the Youth committee and Leadership Training Class to intensify the leadership treatment.

Goals and Rationale interventions--parent involvement, project training, and the action/ad- Project STATUS (Student Training visory committee--are intended to Alternatives Through Urban Strate- involve parents, community members, gies)is continuing its efforts to and project staff in project actvi- provide targeted junior and high ties. All of these interventions schoolstudentsin Pasadena,Cali- are designed to address five prob- fornia, with a high interest curri- lems found at Eliot Junior High and culum (Options Classes) and leader- Muir High Schools: (a) class cut- ship training (Youth ting; (b) disruptive behavior; Cammittee/Leadership Training Class). Three other

ton including the project's Program Development Evaluation plan, conversations with the project director, a This description of the project's report by the Project Director goals, rationale, and objectives (Reiss, 1982), and an earlier draws on several sources of informa- account of the first year's interim

163

26 STATUE

(s) lack of student involvement; clear. This departure from the (4) dropouts at Muir highschool, standards set for the evaluation of aced (e) lack of parent involvement. the alternative education program resulted in a relatively weak evalu- These problems are thought to ation of Project STATUS. For exam- said becauee students are not ple, the major objective of the lovely,* in relevant and meaningful Youth Leadership Training Class, to educational programs. Consequently, increase students' leadership students experience academic ba- skills, is lostamida number of ler*, loss of eelf -esteem, aliens - process objectives. No measure was tiesand less commitment towards specified for this important objec school. This accords with empirical tive. studies, which have shown corrals - ties§ between academic failure, low Planned Intervenkions commitment and attachment to school, aod student misbehavior (Stilt - Project STATUS interventions are chcembe, 1964; lilberberg i Silber - based on the notion that intervening berg, 1971; Elliot6 Voss, 1974; ,only at the individual level or only Mirschi, 1969). at the school level cannot solve studentmisbehavior. The overall to addition to experiencing aca- approach is designed to meet indivi- demic failure,manystudents with dual student needs as well as natural or traditional leadership involve students, school personnel, abilities are seen by the project as parents, community members and 14ckicg the appropriate leadership organizations_ in_ affecting .school skills to effectively influence climate. their school environment. This, along with the perception that the aptions Ulassea school rules are not fair and that there are no mechanisms for student Sixty 7th graders and 60 9th input and involvementin decision - gradersof allacademicabilities making, is believed to alienate stu- volunteer to participate in alterna- dents from school. tive educational English and Social Studies classes. These classes uti- Eased on STATUS' theory, several lize a curriculum focusing on cop- objectives havebeen specified to ing with authority, responsibility, achieve their goals. Table 1 sum- the school, the family, and friends. marize* each goaland the corres - Audio-visual presentations, field pealing objectives. This table trips, guest speakers, role-playing, males clear the extent to which the simulations,and action activities evaluation of this project deviates are used to enrich thecurriculum from the Program Development Evalua- content. Thisintervention,which tion model. The objectives include attemptstoprovidestudents with process objectives or critical relevant and appropriate student- beochmarhs for the implementation of centeredmaterials, is primarily theproject, andthe measures of aimed at STATUS' objectives of goals and objectives are not made decreasing alienation and increasing academicachievement and self-es- ontare teem. evaluation of this project (Carlton, 1982). The Carlton report provides a more complete description of first year activities and plans.

164

27 STATUS

Youth Committee (YC) essential for changeand that as representatives of the school, their Approximately 50 students identi- policies and actions are important fied astraditional (e.g.,student in improving student attitudes and council members, team captains, perceptions (Reiss, 1982). etc.) and natural (e.g., gang leaders) leaders are referred by school In addition to these training personnel to participate in the sessions, project staff participate Youth Committee. This intervention in on-going informal training is designed to provide students with throughout the year. Utilizing opportunities toidentifyproblems weekly observation, the project in the school and seek solutions in director gives feedback to to pro- order to improve the climate of the ject teachers, aides, and counselors school. By channeling the Youth in the areas of individualized Commitee members' leadership abili- instruction, effective use of aides, ties toward effective organizational and classroom participatory manage- change, project implementers theor- ment. ize that these studentswill not only feel moreinvolved withand Parent Training committed to theschool, but will also positively influence other stu- Parents of students participating dents. in the project wereto meet ona regular basis to identify training Leadership Training Class (LTC) needs and toreceivetraining in maximizing parent input in school A leadership training curriculum decision-making practices. The was devised to provide students from intent of this intervention is to the Youth Committee with intensive get parents involved and to get individualized training in develop- parents to accept responsibility for ing decision-making,conflict man- what goes on in school. This inter- agement, and communication skills. vention was not well implemented Approximately halfof the students during the 1981-82 school year (see from the Youth Committee participate below). dailyin this class for one semester; then the rest of the committee Action/Advisory Committee takes the classduring thesecond semester. The purpose of this class The Action/Advisory Committee is is to allow students to develop made up of parents, citizens, commu- theirleadershipskillsandapply nity service agencies, and other them to theproblems addressed in organizations from the Pasadena com- Youth Committee meetings. munity who identify and provide resources for project activities. Staff Training In addition, members serve as role models and provide project students Project staff, school administra- with vocational guidance and tors, and teachersparticipate in resources. Examples of activities training. sessions prior to the fall include making in-class visits, pro- semester to help them facilitate viding vocational intern experi- student development and involvement ences, and organizing on-site visits in organizational change. The to various agencies. As with parent theory behind the training is that training, this intervention is school staff must be aware that designed to get the community positive student attitudes are involved with the school.

165

28 STATUS

Project Environment and Staffing (2%),office operating costs (7%), and indirect costs (21%). The project's environment, i.e., theimplementing organization, the Implementation of Project STATUS targeted schools, and the school district, has remained the same as Lastyear'sproblems in imple- last year (see Carlton, 1982, for a menting the program(see Carlton, completedescription). There have 1982) resulted in differential been, however, some changes in phases of implementation of the staffing. At the school sites, intervention components for this three new teachers' aides replaced year. For example, the Options the three aides from last year who class is more developedthan the left the program. One of the new parent training component. Whereas aides also left during the semester. the Options class is at the imple- Consequently, one of the high school mentation/fine-tuning phase,parent teachers didnot. have a teaching training is closer to an initiation/ aide. adoption phase. Although this situ- ,ation is true at both schools, At caF (Constitutional Rights Tables 2 and 3 indicatethat the Foundation),there was no replace- degree of implementation of the ment this yearfor the community interventions varied at the two coordinator position. One reason schools. Table 2is based on the for no active recruiting was the un- SAES (School ActionEffectiveness certainty of the third year budget, Study) student questionnaire and and the inefficiency in training shows that Options students_at. Eliot someone for a one-year position. significantly differ from non-equi- However, a new administrative assis- valent controls on measures of indi- tant was hired to replace the previ- vidualized instruction, field trips, ous one. Since there is no on-site student-teacher interactions, and evaluator and since the school dis- student influence in decision-mak- trict's director ofevaluation has ing. Their are no differenceson other commitments, it is important Student Degredation (a scale measur- for the administrative assistant to ing negative behavioral treatment of handle many of the evaluation activ- students). On the other hand, ities. Optionsstudentsat Muirdiffered significantly from their non-equiva- Program Expenditures lent controls only in terms ofthe number offield trips. This find- Figure 1 shows how STATUS appro- ing, however, contradicts project priated its $405,971 budget for the reports that nofield trips were 1981-82 school year. Salaries, taken at Muir(see Table 3). The including contractual services (con- datain Table 3 should beinter- sultants, stipend for training ses- preted withcautionsince project sions, etc.) comprise the largest staff recorded the extent and nature single expediture. This 60% of the of program activities for each stu- budgetdoes notinclude thetotal dent at the end of the school year. salaries of project teachers and Inaccurate records and retrospective staff at the schools. A large per- reporting may degrade the quality of centage oftheir salaries are paid thedata. Other project-developed by the Pasadena School District. rating systems were also examined to Program materials and equipment provide information as to the amount account for a small portion ofthe and quality of the Options and LTC budget, 7% and 2% respectively. classes. Other expenditures include travel

166

23 STATUS

Options Classes a highinterestcurriculum. Is a 38% affirmativeresponse to "more The Options classes revolve interesting" good or bad? We do not around a curriculum that aims to be know. As another example, about 40% of high interestto the students. of the students report doing work Each unit of the Options class cur- outside of class--without some com- riculum is based on law-related parison group it is impossibleto materials and execises, and attempts say whether the curriculum is to deal with a significant aspect of inspiring more students to do more the student's life, e.g., the homework, or less. CA comparison family,schooland thelaw. Each group was available for the self-re- topic is intended to address impor- ported School Effort measure in the tant project objectives such as SAES questionnaire. No statisti- increasing attachment to school and cally significant difference was decreasing feelings of alienation found between treatment and compari- from schooland society.1 However, son students.) Project staff becausecurriculumunits werenot report,however, that the results ready until the week of use in the ,did provide feedback to the curricu- classroom, teachers had little time lum writer so modifications could be to review the materials and prepare made for the upcoming year.2 for class beforehand. Other activities ofthe Options Eliot Junior High. The 54 stu- classes included cooperation between dents and two teachers involved in Options and YC/LTC teachersin the the Options class rated each curri- planning of activities, and staff culum unit after its presentation. coordination of a mock trial compet- Ten rating categories were used (see ition for the Options students which Table 4). These ratings were car- includednon-project eighth grade ried out using project-developed students as jury members. materials. Table 4 shows the results forthe students' ratings. Muir High School. Implementation Two of the ratings most critical for problemsat Muir began early when the assessment of project objectives counselors mis-assigned nonproject are ratings of curriculum relevance students to the Options classes at and interest. For each unit, over the beginning of the semester. Pro- half (54-70%) of the students felt ject staff had a difficult time cor- the material was relevant and bet- rectly rescheduling the project stu- ween 38-70% felt the class was more dents because a) it was difficult to interesting than others. We have no re-arrange the mis-assigned stu- norms for these ratings, no project- dents' schedules and b) nonproject generated standard against which to students assigned to the Options compare the ratings, nor are there classes enjoyed the classes and ratings of curriculum made by a com- wanted to remain enrolled in them. parison group experiencing common Ultimately,about 25% of the stu- curricula. These ratings do not dents who took the Options classes necessarily demonstrate that the during the first semester were ori- curriculum that was implemented was ginally designated as nonproject

1. For a description of the curri- 2. Options class teachers also culum, send requests toElizabeth rated the curriculum units, but res- Salzman, CRF, 1510 Cotner Avenue, ponses are based on only one or two Los Angeles, CA. 90025 teachers.

167

30 STATUS students, i.e., theydid not meet tions and 7 participated in school the original criteria for selection committees. At Muir,17 outof 77 into the program. membersparticipated in presentations and 15 participated in school Another implementation problem committees. This suggests that stu- was that one teacher did not have a dents may have been attendingthe teaching aide. In addition, the meetings only to get out of class. project director reports that alternative education techniques have not In addition to problems in sche- been completely adopted at Muir, and duling YC meetings, not all YC mem- teachers there do not spend as much bers could be scheduled into the time planning together as teachers LTC, an elective course. This at Eliot. course was only offered one period perday and often coalicted with Table 5 shows the results of the other courses. Some of the YC mem- students'ratings. ofeach unit of bers who did take the LTCsubse- the Options curriculum as taught at quently dropped out of the YC. The Muir. Depending on the unit, bet- ,scheduling and drop-out problem ween 65-79% ofthe students agreed resulting in only a few students that the information presented was receiving training and experience in relevant. Of the students respond- the use of leadership skills, per- ing, 77-89% felt the class was more haps subverting the YC/LTC as a interesting than others. Project school change intervention. Accord- staff reported that the ratings ing to the project's theoretical proved useful in modifying the cur- model, systematic _changes_ _cannot riculum. occur withinanorganization with only a small number of trained indi- Youth Committee/Leadership Training viduals. Class Another problem in implementing The YC/LTC was only weakly imple- the YC/LTC was that the two project mented in 1981-82. There are sev- paraprofessional counselors, who eral reasons why. As was the case coordinatedthe YC/LTC activities, during the first project year there may have taken on too many responsi- were problems in scheduling YC meet- bilities for their half-time non- ings. Meetings could not be sche- professional status. duled after school because this conflicted with other activities. Besides overseeing the YC/LTC, Lunch period was too short. Teach- the paraprofessional counselors also ers objected to having students served as school counselors for the attend YC meetings during class Options and YC/LTCstudents. The time--despite thesestudentsbeing program design called for the para- identified as leaders, many lack professionalcounselors to arrange academic skills and cannot afford to parent conferences andmake home miss class. However, for lack of a visits if students were failing aca- better time, meetings at both demically, cutting class, or behav- schools were held during class time. ing badly, but almost no conferences Consequently, YC meetings were held or home visits took place. Records infrequently. A second problem was showthat atEliotonly one home that few of the YC studentsgot vist was made all year, and at Muir, involved in project activities: out two. At Eliot, 25 students received of 50 YC members at Eliot,only 4 on-going counseling from the para- students participated inpresents- professionals. At Muir,the para-

168

31 STATUS professional counselor contacted all leadership skills. of the Options students at least once at the beginning of the semes- One resultin the ratings that ter. During the course of the year, may be important for program devel- 15 students at Muir were called in opment efforts isthat the ratings once for behavioral problems; at Eliot or generally lower than another two students were called in those at Muir: nineteen of twenty- twice. five ratings favor Muir. Even this comparison should be interpreted A further problem with the imple- very cautiously because of the small mentation of the YC/LTC is that the number of students reponding to the counselors hadlittleexposure to survey. Percentage agreement scores the leadership training curriculum. are unstable when the n's are as low as they are here. To summarize the efforts ofthe paraprofessional counselors, they Muir High School. The 77 members only worked half-time, were unfamil- of the YC addressed issues such as iar with the leadership training class and school cutting and student curriculum, providedlittle or no "pride. The LTC was held daily for documented counseling, and made a 16 YC members each semester. Eight- total of only three home visits. It y-two percent of the second semester is likely that unless they engaged LTC students rated the information in other,undocumented activities, presented in class as relevant; 85% theyhad littleimpacton project found the class more interesting students. than other classes at Muir. All of the students- responding--reported Eliot_ Junior High School. A that they had developed the ability total of 50 students participated in to lead a group (see Table 6). the YC during the year and addressed Again, without a control group, such issues as security and safety there is no clearcut wayto inter- at school, longer lunch periods, and pret these ratings. more elective courses for 8th graders. Twenty-eight of the YC members Staff Training took the LTC. Scheduling difficulties led to a late start-up date for During the school year, several the class. As a result, first sem- training sessions, wereheld with ester LTC students remained in the project/staff, teachers, and school class for the entire year, with no administrators. A total of 17 for- new students added the second semes- maltraining meetings with project ter. teachers, counselors, and aides were held. Over 40 individual meetings Table 6 shows the results for the were held with project teachers and studentratings of the leadership the project director to discuss the training curriculum. Ratings of in-class observations made weekly by relevance, interest, and leadership the project director. training received are of particular interest to project staff. Overall, Parent Training 83% of thoseresponding felt the information presented was relevant, Two meetings at each schoolfor 77% found the class more' interesting the parents of STATUS students were than others,and 91% felt they had heldthis year. Approximately 50 developed leadership skills. Unfor- parents were involved in these meet- tunately, we do not know whether the ings which were run bythe project students did in factimprove their

169

32 STATUS

director and principals of the pro- difficulty was the ignorance of project schools. An overview of the jectstaff aboutthe goals ofthe program was given and an description project. It is only with the start of how parents could help the pro- of the third project year that other ject and how the project could help project staffbesidesthe project parents. The program design called director havebegun to understand for the holding of parent training the goals and objectives of the pro- workshops, but none were held. No gram. Finally, a lack of direct parent committees were established. project control over selection and supervision ofin-:school staff, and Action/Advisory Committee a shortage of personnel to coordinate allaspects of theinterven- This year the action and advisory tions also contributed to poor pro- committees werecombinedinto one ject implementation. committee. Meetings were held every six to eight weeks with an average Interim Outcomes attendance of 15. The committee revised and distributed the commu- Interim evaluation results have nity resource guide. The committee two potential uses: they can pro- also held a public relations meeting vide information as to the effec- at the superintendent's house to tiveness ofa project to date, and expand the project'sbase in the mayserveas material forfurther community. program development. The extent to which the information is useful Summary depends inlarge part on the ade- quacy of the evaluation design. For For the Options and YC/LTC inter- the CRF project, strong evaluations ventions, obstacles internal and were agreed to, but were considera- externalto the projectinfluenced bly weakened by non-random assign- implementation. A major external ment to treatment and control,and force was scheduling: project stu- multiple treatment for project stu- dentscould notbe scheduledinto dents. project classes, non-project students werescheduled into project Evaluation Design classes, and convenient meeting times for the Youth Committe could Options Classes. Potential not be found. An internal obstacle Options participants were recruited to faithfulproject implementation by project staff prior to the fall was the reluctance of project staff semester. Interested students were to adopt teaching and counseling required to submitan application styles advocated by the project and acquire parental permission to director. In addition, project participate in STATUS. Fromthis implementers were inexperienced with pool of applicants project staff the curriculum developed for the selected students who were represen- Options classes and the Leadership tative of the school population, Training Classes. Thus, project i.e., participants included students staff did not adopt project educa- of all academic abilities, races and tional techniques, and were not behavioraldispositions. The pro- experienced in project curricula. ject agreed to randomly assign stu- It appears, therefore, that these dents to treatment and control from components were not implemented as thisreferralpool. Randomization designed. was not adhered to at either Eliot or Muir. A further implementation

170 STATUS

At Eliot, students were randomly We do notknow howcomparison assigned to treatmentandcontrol group students were selected for the from the pool of screened referrals. Eliot and Muir Youth Committees: Roster updates collected in October students from the referral pool and indicated little relationship bet- consent pool had originally been ween the original random assignment randomly assigned, but different and the students designated as students were ultimately designated "treatment" and "control" by project by the project as treatment and con- staff. The reason for this is trols. Students assigned to control unclear. What is clear is that the became treatment,and new students evaluation of Eliot's Options appeared in both treatment and con- classes involves a comparison bet- trol groups. To compound these ween treatment students and a non-e- problems,at Muironly 6 controls quivalent control group accompanied were identified. Nosensible com- by an unmeasured selection bias. parison is possible between approximately 50 treatment students and 6 The experimental evaluation of controls. Therefore the Muir YC the Optionsclassesat Muir broke ,cannot be evaluated. down when not enough students received parental permission to par- Summary of Evaluation Designs. ticipate. The non-equivalentcon- Forthe Options classes, quasi-ex- trol group at Muir therefore largely perimental non-equivalent control consists ofstudents whose parents group designs describe the evalua- did not wish them to receive alter- tions at both Eliot and Muir. The nativeeducation. The differences evaluation ofthe YC atE_ liot can between such students and those also be characterized as a non-equi- whose parents did grant permission valent control group design. No cannot be known with any certainty. evalution design was implemented for the Muir YC, or for either school's The YC. As with the Options LTC. classes,YC participation is on a voluntary basis and is open to all Measures. Measures come from students. Faculty, administrators, several sources: a) the SAES ques- parents, and students referred stu- tionnaire, b) school records, c) dents to the YC. The criteria for project questionnaires, and d) pro- referral wasthatthestudent was ject records. The majority of mea- identified as a natural or potential sures are scales and individual leader. LTC students were non-ran- items from the SAES student questi- domly selected from among those stu- onnaire. Test scores, attendance dents recognized as the most promis- and grades were taken from school ingleaders in the YC. Selected records. Project questionnaires students also had to be able to fit include student and teacher evalua- the LTC class into their school tions of the curriculum, student schedules. Thusthe LTC students development of leadership competen- are a select group, and no convinc- cies, and a teacher observation ing control group for them could be checklist. Information from project developed--LTC students will not be records will beused to determine evaluatedseperatelyfrom YC, stu- the number of parent meetings held, dents at large. andthe number of policy changes initiated by parents and students.

171

3 STATUS

Results 'tistically significant, these students also reported themselves as Equivalence of the Control cutting classes and school more than Groups. Tables 7 and 8 give pretest did the controls. Options students comparisons between the various at Muir do not significiantly differ treatments and their available con- fromtheir non-equivalentcontrols trols. The treatment and non-equi- on any attendance measure. No sig- valent control groups appear fairly nificant effects on attendance were similar on the measures taken (these found for the YC at Eliot. pretest measures were scored in a different--but equivalent--metric Nosignificant differences were than similarly-named outcomemea- found in the amount of disruptive or suresreported later). TheEliot delinquent behavior between Options Options students do not differ from and non-equivalent control students their controls on any of the 12 psy- at Muir and Elliot, nor between YC cho-social measures,and the Youth students and non-equivalent controls Committee students differ from their at Eliot (see Table 10). controls only on a measure of Attachment to Parents: YC students There was a significant differ- are more attached to their parents ence between Options and non-equiva- than the non-equivalent controls are lent controlstudentsat Eliot in to theirs. AtMuir, the Options terms of the numberwho withdrew students differ from their controls from school (see Table 11). Signi- on measures ofPractical Knowledge ficantly fewer Options students and Rebellious Autonomy: The treat- withdrew from school (p<.01). Par- ment students report more practical ticipants in the YC at Eliot and the knowledge, and are less rebellious. Options classes at Muir did not dif- Both of these comparisons, there- fer from their non-equivalent con- fore, favor the treatment group. trols in the number of withdrawals.

The evaluationdesigns for the With no design, we do not know Eliot Options classes and Youth Com- how Muir's YC affected the project's mittees, and the Muir Youth Commit- goals. tees are non-equivalent control designs. They are not true experi- Unlike last year, the Youth Com- ments. Besides the measured differ- mittees at Eliot and Muir were able ences that do exist, other, unmea- to get students involved in school sured, differences may well exist. decision-making. This year, there were three student-initiated policy Project Goals. Tables 9 through and procedural changes put into 11 show treatment and non-equivalent practice. However, the project was control group comparisons on several unable to increase parental involve- measures ofSTATUS' goals for both ment in school decision-making prac- Eliot and Muir. Three measures were tices at eitherschool--noparent used to assessclasscutting(see initiated policy or procedural Table 9). Options students at Eliot changes were put into practice. had significantly more unexcused, i.e., non-illnessand non-medical, Project Objectives. The results absences than non-equivalent control of treatment and non-equivalent con- students. Statistically, the prob- 1 trol group comparisions on STATUS' ability of this difference occurring objectives as measured by the SAES by chanceis less than 5 times out student questionnaire are shown in of 100 (p< .05). Although not sta- Tables 12 and 13. Options students

172

35 op* STATUS at Eliot are significantly less Parents, Parents' Emphasis on Educa- alienated (p<.05) than controls, and tion, Belief in Conventional Rules, are more likely to report that LifeCompetency,InternalControl, schoolrulesare fair (p<.01)and Negative Peer Influence, School clear (p<.01; see Table 12). There Effort, Involvement in Conventional are no differences on scales measur- Activities, Rebellious Autonomy, ing Self-esteem or Attachment to Victimization, School Punishments, School. MuirOptions students do School Rewards, or Invalidy (a mea- notdiffer from their controls on sure of truthfullness in reporting). any of the above measures. Eliot YCmembers report them- The YC at Eliot produced no mea- selves to be more Involved in Con- surable effects on measures of ventional Activities than dotheir Alienation,Self-Esteem, Attachment controls. This is probably an to School, or the Clarity or Fair- implementation measure, and suggests ness of School Rules. We cannot that the Eliot YC members are becom- assess the effectiveness ofthe YC ing involved ina wider range of at Muirat meeting theprogram's ,,exeriences. The YCstudents also psycho-social objectives. report signficantly lower attachment to their parents. Why this should In terms of academic achievement, be the case is unknown. Although it Options students at Eliot reported is conceivable that the program is themselves to be higher in reading causing the students not to care for ability than did the non-equivalent their parents, the more likely controls (p<.01; see Table 13); they explanation_ is that_this isa pre- did not report, however, getting existing difference arising from the higher grades, or receive them. No non-equivalent nature of the design. measurable effects on school None of the other measures on Table achievement were found for the 14 show any differences between Options program at Muir, or the YC Eliot YC treatment and controls. at Eliot. We cannot judge the impact of Muir's YC on academic The Muir Options class does not achievement. differ from its designated non-equivalent control on any ofthe four- Outcomes of Theoretical Interest. teen psycho-social outcomes found in Table 14 shows the results of treat- Table 14. ment and non-equivalent control groupcomparisons onoutcomesnot Summary. The interim outcome directly addressed by project STATUS evaluation may be summarized as fol- but which are of theoretical impor- lows: tance totheprevention of delinquency. These outcomes also provide 1. Comparisons between Eliot a check on any unanticipated nega- Options class treatment students and tive or positive effects of the pro- non-equivalent controls suggest that ject. the class results in fewer withdrawals from school, decreased aliena- Options class students at Eliot tion, greater interpersonalcompe- differ from the non-equivalent con- tency, higher perceived fairness and trols only on a measure of Interper- clarity of schoolrules,but more sonal Competency: Options class unexcused absences. This interven- students report themselves to be tion may be also be increasing stu- more interpersonally competent than dents' perceptions of their own do the controls. They do not differ reading ability of students, but it onscales measuring Attachment to

173

36 al STATUS

is not affecting grades, or disrup- resultant non-equivalent control tive behavior. groups. All results (and non-results) must be interpreted in this 2. The evaluation ofthe Muir light. Options class provides no evidence that it has any effects on its par- What Accounts for the Results? ticipants--no differences were found between the treatment and non-equi- Four factors may account for the valent controls on any of twenty- lack ofeffectiveness of the Muir seven outcome measures. Options class as compared to the more positive effects found for the 3. Comparisons between Eliot YC Eliot Options class. First,it may students and their non-equivalent be that the curriculum is more controls show that YC students effective on younger students, i.e., report more Involvementin Conven- 7th graders as opposed to 9th grad- tional Activities (a program imple- ers. mentation measure), but less attachment to their parents. The latter Second, an undetermined number of effectis probably a pre-existing, control group students received pro- between group difference. The ject services at Muir. This isa interim evaluation found no evidence potential explanation ofa lack of that this intervention is affecting differences between the control and attendance, academic achievement, treatmentstudentsonthe outcome disruptivebehavior, or psycho-so- measures. To examine this possibil- cial adjustment for participating ity, analyses were conducted using students. only non-equivalent controls who received no project services. This 4. No convincingevaluation is liberal analysis still uncovered no available forthe Muir YC, or the differences between the treatment parent involvement,project train- and "no-treatment" controls at Muir. ing, and action/advisorycommittee components of the project. A third reason why Muir's Options class appears to be less successful 5. Implementation dataimplies than Eliot's, is that Eliot's class that the Eliot Optionsclass was may have been somewhat better imple- more strongly implemented than the mented (see Tables 2 and 3). Muir Options class. Nevertheless, student ratings suggest that the A fourth possibility, of course, Muir students enjoyed the class is that the few significant findings more. (both positive and negative) for the Eliot Options class are due to pre- Discussion existing differences between treatment and non-equivalent control stu- Caveats to the Evaluation Results dents. Because we do know how treatment and controls were selected As discussed earlier, there were we cannot conclusively attribute significant breakdowns in the origi- program effects (or the lack of nal experimental designs for the them) to the program. Eliot Options classes and YC, and the Muir Options classes. Despite The null findings for the YC on their similarity on the pretests individuallevel measures at Eliot available, unmeasured differences, may be misleading because the empha- especially motivational differences, sis of this intervention is on may exist between the treatment and

174

37 STATUS school-level changes, unlike the Program refinements for 1982-83. Options class, which focuses on Because the results suggest that the individual-level changes. It is YC and LTC interventions asimple- possible, although unlikely, that in mented in 1981-82 didnotaffect a school climate sense, the YC may students, program changes have been affect the control students as much made for the 1982-83 yeaL The YC as the treatment students. and LTC will be combined next year into a single intervention,which The puzzling negative effect that will serve only 30 students. Fewer the Eliot Options class appears to studentswill be served, but the be having on attendance may be the intervention will be intensified by resultof thepositive effect the having a smallnumber ofstudents program is probably having on with- meet everyday to receive the leader- drawalsfromschool. The program ship training and practice in appli- may be successfully retaining stu- cation. We are hopeful that this dents in the program that would have consolidation may eliminate the pro- dropped out; these students may gramimplementation andevaluation still skip school more than students ,design breakdowns encountered this that would have remained in school year. without the program's support. In other words, the "negative" effects Modifications were made for both onabsencesmay be a paradoxical the Options and LTC curriculums, resultof keeping Options students based on the feedback from the stu- enrolled in school. dent and teacher ratings collected by the project. A greater emphasis Implications of the Results will be made on addressing STATUS' objectives in each curriculum unit. Plausibility of STATUS'Theory. Although we cannot unequivocally Evaluation Strategies for 1982-83 attribute the positive effects found for the Eliot Options class to the The structure of the PDE plan has class, the program does appear to been modified this year to make it have met certain project objec- more useful to the project staff. tives--decreasing alienation, and Inaddition,clearerspecification increasing the use of individualized ofthe project's action theory has instruction. It also appears to resulted in more coherent objectives increase students' perceptions of and helped guide intervention stra- their reading ability. It also tegies. probably is reaching a project goal of decreasing withdrawals from As with last year, a quasi-exper- school. Also, even the Eliot imental design will beimplemented Options class has had no measurable next year to assess the effective- effect on delinquency. In short, we ness of the Options and Youth Lead- do not know as yet whether the ership Training class components. theorybehindSTATUS as a delin- We were largely unsuccessful at quency prevention project will ulti- changing the procedures for place- mately prove useful. ment of students into treatment and control conditions forthe 1982-83 The theory underlying the YC is school year, but an early assessment untested in results to date. No of theequivalence of the groups evidence is available to suggest shows no major differences between that this portion of the project is treatment and controlstudents on effective; it was not implemented in the few outcomes we analyzed. strong form.

175

38 STATUS

Problems Project STATUSis Encoun- interpersonal skills(although, as tering yet, we have been unable to verify such changes with hard data). Stu- As discussed earlier, Project dents have given presentations at STATUS has experienced major prob- board meetings and have competed in lems in implementation. A second a. Mock Trial Competition. These majorproblem concerned evaluation successes have helped turn initial issues. One issue is that the pro- skepticism of the program by school ject director feels there is a and district staffs to support. As trade-off between implementation a consequence, STATUS has been writ- activities and evaluation activi- ten into the Muir School Improvement ties. STATUS's small staff makes Plan, funded by the state to improve this problem particularly acute. the school environment. In addi- Another issue is the District evalution,a description ofthe project ator's attitude toward the national has also been included in "A Compen- evaluation. He feels the evaluation dium of Character Education Programs effort is larger 'thanthe program in California Public School..." itself and hasstrongly questioned the validity ofthe evaluation and Institutionalization the use ofthe SAES questionnaire. This resistancemay havediverted Even before it was known whether energies from program implementation the project would be funded by OJJDP to evaluation issues. A final eval- for the 1982-83school year, the uation issue is providing data to Options classes atboth Eliot and the national evaluators in a timely Muir were _included_ in_ the _master_ _ fashion. Besides a lack of manpower schedule for the upcoming year. The to collect the data, lags in updat- English Department at Muir iscon- ing school records createsa data sidering the adoption of project management problem. Plans have been materials to be used in other made for the upcoming year to classes. Other schools in the dis- address these obstacles. trict have also expressed interest in STATUS. Successes Project STATUS is Encoun- tering Although next year willbe the last year of federal fundingfor The treatment and non-equivalent Project STATUS, there are some prel- control comparisons indicate that iminary indications that project the Options class at Eliot is proba- STATUS orsome aspects ofthe pro- bly successfully achieving some of ject mey be adopted bythe project its objectives. School administra- schools -nd other schools as well. tors, teachers, and parents have The Options class shows potential. reported marked improvement in Careful implementation and monitor- Optionsclasses participants; they ing of other aspects of the program feel that these students have newly will be necessary to realize the acquired academic leadership and program's optimal strength.

176

39 STATUS

References

Carlton, R. Student Training Alternatives Through Urban Strategies (Project STATUS): Interim Report. In G. D. Gottfredson (Ed.), The School Action Effectiveness Study: First Interim Report. Baltimore: Johns Hopkins University, Center for the Social Organization of Schools, 1982.

Elliot, D.S. & Voss, H. L. Delinquency and dropout. Lexington: Heath & Co., 1974.

Gottfredson, G. D. & Daiger, D. C. Disruption in 600 schools: The social ecology of personal victimization in the nation's public schools (Report No. 289). Baltimore: Johns Hopkins University, Center for Social Organi zation of Schools, 1979.

Hirschi, T. Causes of delinquency. Berkeley: University of California Press, 69.

Reiss, D. Project STATUS: An overview. Paper presented at the annual meet ing of the American Educational Research Association, New York, 1982.

Silberberg, N. E. & Silberberg, M. D. School achievement and delinquency. Review of Educational Research, 1971, 14 (1), 17-33.

Stinchcombe, A. L. Rebellion in high school. Chicago: Quadrangle, 1964.

177

4;0 Figure 1

Projoct STATUS' Budgeted Expenditures for 1981-1982

4 Total Personnel-CRF Staff $127,746 Program (31%) Materials $27,439 (7%) Office Operating Travel $10,135 (22 Costs $27,054 (7%) Equipment $7,945 (2%) Indirect Contractual Services Costs $119,595 $86,057 (29%) (21%)

-178- 41 Table 1

Project STATUS' Goals, Objectives, and Measures

Goal/Objective Measure

I. Coal: Decrease class cutting. I.A. SAES student questionnaire B. Attendance data

Objectives:

1. Increase use of individualized instruction plans. 1.a. Count of individualized educationalplans for student b. SAES student questionnaires

2. Increase use of appropriate student centered materi- 2.a. Teacher evaluation of CRF material als. b. Student evaluation of CRF material

3. Increase use of high interest curriculum. 3.a. Teacher evaluation of CRF material b. Student evaluation of CRF material

4. Increase self-esteem. 4.a. SAES Student questionnaire

5. Increase academic achievement. 5.a. GPA b. CTBS scores c. SAES student questionnaire

6. Decrease alienation. 6.a. SAES student questionnaire

7. Increase attachment to school 7.a. SAES student questionnaire

II. Goal: Decrease disruptive student behavior. II.A. SAES student questionnaire

Objectives:

1. See objectives 1-7 above.

42 Table 1(cont.)

Goal/Objective Measure

III. Goal: Increase student involvement in decision making III.A. Number of student initiated policy and procedure levels. changes put into practice

Objectives:

1. Decrease student alienation. 1.a. SAES student questionnaire

2. Increase classroom participatory management. 2.a. Student self-report b. Observations of classroom procedures by project director

3. Increase leadership competencies of youth committee 3.a. Student self-repOrt members.

4. Increase perceptions of fair, and clear school rules. 4.a. SAES student questionnaire

IV. Goal: Decrease drop-out rate. IV.A. Attendance data

Objectives:

1. See Goal II, objectives 1-7.

2. Increase parents' awareness of their child's capa- 2.a. Number of times parents participate in meetings bilities. with project teachers and counselors

d3 Table 1 (cont.)

Goal/Objective Measure

V. Goal: Increase parental involvement. V.A. Number ofparent initiated policy changesputinto practice 8. Number of parent initiated procedure changes put into practice

Objectives:

1. Increase number of opportunities for parents to par 1.a. Number of parent meetings held ticipate. b. Number of parent committees c. Number of parents attending meetings

2. Increase parents' skills to participate meaning 2.a. Number of PET meetings held fully. b. Number of parents attending training sessions

/1 3. Increase community involvement. 3.a. Number of Advisory/Action committees held b. Number attending meetings

Note. Goals and objectives were extracted from the Program Development worksheets and Lrom, Project STATUS: An overview, a paper presented by David Reiss at the annual meeting of the American Educational Research Association in New York, 1982. Table 2

Trmtment and Control Comparisons on

F.ovoral l!Ip!tlionnairo Mvo,tow; a rniim hplmontallan Objectives

Individualizea d Student - Teacher Student a n b b Treatment Instruction Field Trips Interaction Degradation Influence

and Croup M Sd N N Sd N II Sd N M Sd N N Sd N ELIOT JUNIOR HIGH SCHOOL

Options

Treatment 2.68* 1.90 59 1.59**1.04 59 1.52** 1.37 60 2.18 1.57 57 2.71* 1.89 56

Control 1.84 1.86 51 .49 .81 55 .90 .99 51 1.88 1.90 51 1.83 2.04 48

YC 6 LTC

Treatment 1.89 1.86 71 .94** .96 50 1.48 1.05 48 2,80 1,75 45 2.36 2.00 45

Control 1.61 1.59 49 .47 .73 74 1.16 1.16 79 2.47 1.69 72 2.21 1.81 71

Co MUIR IITCH SCNOOL tN)

I Options ** Treatment 2.33 1.93 54 1.61 .98 54 1.60 1.26 52 2.10 1.60 51 3.02 1.97 51

Control 1.67 1.75 54 .42 .66 55 1.17 1.19 53 1.57 1.45 54 2.34 2.22 50

Note: Due to small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth Conatta members and controls at Muir High School

`Individual questionnaire items

h Individual level sttident scales *r.05 **p.01

46 Table 3

Average Number of Times Project Status

Students Participated in Program Activities

Semesters in Options Semesters in LTC YC Meetings Task Forces Field Trips

Group M. SD N M SD N M SD N M SD N M SD N

ELIOT JUNIOR HIGH SCHOOL

Options .91 .94 114 .09 .41 114 .45 1.92 114 .11 .49 114 .08 .36 114

YC/LTC .10 .35 140 .36 .74 140 2.70 3.97 140 .64 1.07 140 .43 .70 140

MUIR HIGH SCHOOL

H Options .91 .95 108 0 0 108 .39 1.29 108 .01 .10 108 0 0 108 w YC/LTC .03 .23 91 .25 .44 91 2.84 2.35 91 .07 .29 91 0 0 91

Note. The information presented here is based on end of the year project reports on the nature and extent ofprogram activities for each student. This includes both treatment and control groups since some control students participated in other aspects of the program.

47 Table 4

Options Students Ratings of High Interest Curriculum

at Eliot Junior High

Unit I Family Unit II - Law Unit III - Person Unit IV - School Percent Percent Percent Percent Item Agree N Agree N Agree N Agree N

Understood purpose 94% 127 100% 42 93% 42 100% 40 Information was relevant 70 125 54 42 66 38 61 41 Eajoyed activities 82 129 88 41 93 40 93 42 Did outside work 39 126 43 42 37 41 39 41 Read and understood materials 94 123 95 41 92 42 100 40 Speakers, films helpful 92 114 88 42 93 42 88 41 Would recommend class to friends 59 127 55 43 71 41 66 41 Class more interesting / i H than others 54 126 70 43 39 41 63 41 '42 Materials different than

1 uther classes 90 123 93 43 . 90 41 90 40 Class lived up to expectations 65 124 85 40 66 35 73 37

Note. Rating forms were given to both English and social studies Options students at the end of eachunit.

49 Table 5

Options Students' Ratings of

High Interest Curriculum at Muir High School

Unit II - Law Unit III - School Unit IV - Job Unit V Life Percent Percent Percent Percent Item Agree N Agree N Agree N Agree N

Understood purpose 94% 112 93% 102 97% 33 100% 18 Information was relevant 55 110 66 101 79 33 65 17 Enjoyed activities 91 111 95 102 88 32 94 18 Did Outside work 27 107 23 104 52 33 56 18 Read and understood

materials 96 110 96 103 94 33 94 18 Speakers, films helpful 96 110 89 102 94 33 89 18 Would recommend class to

1 friends 93 1-' 110 90 104 88 33 94 18 Class more interesting

1 than others 77 110 87 104 78 32 89 18 Materials different from

other classes 88 109 88 104 88 32 100 18 Class lived up to expecta-

tions 79 105 74 98 84 31 89 18

Note. Rating forms were given to both English and social studies Options students at the end of eachunit.

52 Table 6

Student Ratings of LTC Curriculum at Muir High and Eliot Junior High

Muir Higher Eliot Junior High Item Percent N Percent N Agree Agree

Understood purpose 100% 13 96% 23 Information was relevant / 82 11 83 23 Enjoyed activities 100 13 91 23 Did outside work 85 13 52 23 Read & Understood materials 92 13 87 23 Speakers, films useful 92 12 86 21 Would recommend this class to friends 100 13 91 22 Class more interesting. than others .. 85 13 77 22 Materials different from other classes 92 12 91 23

Involved in rule development 83 ,, 12 71 21 Involved in class constitution dev't. 33 12 48 21 Involved in planning 50 12 76 21 Involved in making suggestions 75 12 82 22 Class lived up to expectations 100 13 45 20 Involved in selecting school issues 73 11 75 21 Expressed opinions 100 13 67 21 Class conducted by students 85 13 - ______11___21 Different points of view heard 100 13 77 22 Decisions based on general aggreement 62 13 82 22 Students responsible for o n behavior 83 12 52 21 Students respect different opinions 92 13 50 20 Little wasted time 85 13 45 22 Students decide plan 100 13 73 22 Equal group representation 69 13 55 22 Developed ability to lead group 100 13 91 22

Note: Leadership training class students were given course evaluations at the end of the year.

aResponses are from second semester students only.

-186-

53 Table '1

Means and Standard Deviations

on Fall '81 Pretest Measures- -

Eliot

Treatment

Options Class Youth Committee

Treatment Control Treatment Control

Scale M SD N M SD N M SD N M SD N oppamorjammirww.m..mm.

Alienation 92 1.08 44 .84 .83 35 1,23' 1.18 48 1,14 .94 71

Attachment to Parents 1,28 2.49 42 1.94 3.06 37 .61* 2.80 48 -.71 4,07 71

Negative Belief (Non-Belief in 1,41 1.37 42 1.81 1.34 38 1.60 1.29 48 1.55 1.29 72 Rules)

7.17 School Effort 1.69 42 7.42 1.60 36 6.79 1.67 46 7.07 1.63 64

1!, Negative Peer Influence 1,80 1.78 40 1.81 1.86 33 1.89 1.88 44 1,65 1.92 69

Interpersonal Competency 3,88 1.00 42 3.91 1.05 38 4.02 1.09 49 4.05 1.05 72

High Internal Control 3.07 1.30 43 3.30 1.02 38 2.84 1.08 49 2.90 1.18 72

Practical Knowledge 8.57 2.78 42 9.11 2.89 38 9,53 3.29 49 9.81 2.86 71

Positive Self-Esteem 1.75 4.08 28 1.54 4.86 24 .99 5.06 29 1.77 4.93 43

Attachment to School 3.81 5.46 42 3.04 3.35 36 1.02 6.78 47 1.91 5.45 69

Involvement -.53 5.27 40 1.00 5.80 33 1.67 6.66 46 1.41 6,27 61

Rebellious Autonomy 1.58 1.05 43 1.64 1.12 38 1.85 .92 48 1.81 1.00 72

'fit -test for difference between treatment and control is statistically significant at the p<.05 level, indicating source of non-equivalence of groups at pretest.

**cutest for difference between treatment and control is statistically significant at the p<.01 level, indicating source

4. of non-equivalence of groups at pretest.

Note, Refer to Char ter 4 of the First SAES Interim Report for description of scale construction. Item number 32 was

omitted from the 'Attachment to Parents" scale, item number 45 from the "Negative Peer Influence" scale.The scale "High Internal Control" is not discussed in chapter 4. It includes the following three items from the pretest:

--Luck is more important than hard work.

--Getting what I want has little or nothing to do with luck. --Every time I try to get ahead, something or someone stops me. 54 55 Table 8

Means and Standard Deviations on Fall '81 Pretest Measures- - Muir Options Class

. Treatment Control

Scale M SD N M SD N

Alienation 1.10 1.01 51 1.14 1.14 23

Attachment to Parents -.46 3.13 51 .15 2.10 24

Negative Belief (Non-Belief in Rules) 1.25 1.22 48 1.42 1.26 19 School Effort 7.14 1.61 50 7.33 1.56 24 Negative Peer Influence 1.82 2.07 47 1.17 1.68 23

Interpersonal Competency 3.99 .88 50 4.00 1.00 23 High Internal Control 3.38 1.02 50 3.09 1.20 23

Practical Knowledge 9.49**2.51 51 7.69 .2.67 24 Positive Self-Esteem 1.77 3.39 40 2.08 3.58 21 Attachment to School 3.84 4.19 47 2.77 3.29 22

Involvement -1.11 5.18- 46-_-1.34-5.69_23 _ Rebellious Autonomy 1.68* .98 49 2.23 .87 22

*t-test for difference between treatment and control is statistically significant at the p<.05 level, indicating source of non-equivalence of groups at pretest.

* *t -test for difference between treatment and control is statistically significant at the p<.01 level, indicating source of non-equivalence of groups at pretest.

56 -188- Table 9

Treatment Control Group Comparisons on Several

Measures of the Project Status Goal, "Decreasing Class Cutting"

b Treatment Class Cuttinga School Cuttinga Unexcused Absences

and group M Sd N M Sd N M Sd N

ELIOT JUNIOR HIGH SCHOOL

Options

Treatment .61 1.19 56 .43 1.08 56 1.56*3.15 64 Control .36 .86 33 .09 .30 32 0.58 1.33 55,

1 ,Youth Committee M 0 Treatment .71 1.11 42 .29 .74 42 0.43 1.07 54

Control .50 1.05 72 .36 .92 72 0.99 2.04 86

MUIR HIGH SCHOOL

Options

Treatment .91 .97 47 .57 1.16 47 1.42 2.83 55

Control .93 1.00 40 .68 1.10 40 1.95 4.55 55

Note: Due to the small number of non-equivalent controls (nr.:6) meaningful comparisons cannot be made between Youth Committee members and controls at Muir High School.

a Class cutting and school cutting are individual items on the SAES student questionnaire.

b Measure based on school attendance records. r. *p<.05.

vu 57. To1111, 10

Treatment and Control Croup Comparisons on

Several Measures of the Project Status Cool,

"Decreasing Disruptive behavior"

!,11.tep.o.ted Sen-reporte Sll-rcporied Sll-reported 'Fall 001, lii 1081v Classrnor Treatment '01..petolon4 Oelinquuncy Drug nor serious delinquency Suspensions ExpolsIons Disrupt i and Croup 4 Sd N H Sd N 8 Sd N M Sd N M Sd N M Sd N M Sd

ELIOT JUNIOR 4ICH SCHOOL

Options

Treatment .24 .43 49 .17 .22 48 .20 .28 48 .14 .22 49 0.0 0.0 64 0.0 0.0 64 .50 .51

Control .25 .44 32 .14 .18 31 .20 .27 31 Jo .14 31 0.0 0,0 55 0.0 0.0 55 .33 .47

Ymth Cc!':;'111u.'

Treltrav111. .11 .32 35 .18 .19 30 .29 .29 37 .11 .19 39 0.0 0,0 54 9.0 9,0 54 .40 .50

Control .2h .44 62 .19 .22 64 .27 .30 65 .13 ,24 64 0.0 0.0 86 0,0 0,0 86 .49 .51

V) 01 III 11104 5011001. C)

I Options

Treatment .09 .30 42 .21 .20 41 .34 .31 40 .13 .18 41 ,:n.o 0,0 55 0,0 0.0 55 .35 .48

Control .03 .16 39 .16 .16 37 .33 .34 31 .08 .11 37 0.0 0.0 55 0,0 0.0 55 .27 .45

Note; Due to the small number of non-equivalent controls (n.6) meaningful comparisons cannot be made between

Youth Commilto.. rwrbers and controls at Muir High School .

individual Item on SAES student questionnaire.

b Individual-level student scale

C,1 hosed on school records.

60 Table 11

Treatment - Control Group Comparisons on a Measure of the Project Status Goal, "Decreasing Dropout Rate"

Treatment a and group Withdrawal from School M Sd

ELIOT JUNIOR HIGH SCHOOL

Options Treatment 0.13**0.33 64 Control 0.35 0.48 55

Youth Committee Treatment 0.06 .23 54 Control 0.13 .34 86

MUIR HIGH SCHOOL

Options Treatment .09 .29 55 Control .20 .40 55

Note: Due to the small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth C.ommittee members and controls at Muir High School. aWithdrawals were collected from school records and include reasons such as transferred, graduated, involuntary, and voluntary. The reported mean combines involuntary and voluntary withdrawals.

**p<.01. Table 12

Treatment and Control Comparisons on a Several SAES Questionnaire Measures of Project Status' Objectives

Positive Attachment Fair Clear

Treatment Alienation Self-esteem to School School Rules School Rules

and Group M Sd N M Sd N M Sd N M SD N M SD N

ELIOT JUNIOR HIGH SCHOOL

Options

Treatment .31* .25 47 .72 .17 42 .71 .25 51 2.70**1.81 53 3.42**1.76 57

Control .45 .23 28 .72 .19 26 .67 .24 32 1.48 1.66 46 1.77 1.87 47

,110.11 CummitC0

I Treatment .43 .30 34 .71 .15 31 .58 .28 37 2.09 1.66 46 2.83 1.69 46 f-i Control .36 .27 55 .73 .18 53 .62 .25 63 2.33 1.69 67 2.79 1.77 73 UD N

MUIR HIGH SCHOOL

Options

Treatment .36 .27 35 .72 .19 32 .74 .23 44 2.51 1.64 47 2.98 1,76 49

Control .31 .31 38 .73 .16 33 .75 .23 39 2.33 1.86 51 2.63 2.01 54

Note: Due to small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth Gumittee members and controls at Muir High School. All measuresare individual level student scales.

a Sce section on objectives for more complete description.

*p<,05 **p<.01

63 62 Table 13

Treatment and Control Group Comparisons on Several Measures of the Project STATUS Objective,

"Increase Academic Achievement."

Treatment Self-reported Gradesa Self-reported Readinga Fall GPAb Spring GPAb and Group Ability

M Sd N M Sd N M Sd N M Sd N

ELIOT JUNIOR HIGH

Options

Treatment 2.50 .84 54 1.96**.79 52 2.34 .66 53 2.29 .61 54

Control 2.41 .61 32 1.38 .80 26 2.28 .62 41 2.30 .68 36

1.4

Treatment 2.53 .83 43 1.70 .79 40 2.36 .77 46 2.39 .74 50

Control 2.80 .80 71 1,94 .87 69 2.53 .85 80 2.50 .89 74

MUIR HIGH SCHOOL

Options

Treatment 2.62 .74 47 1,74 .80 46 2.31 .91 48 2.50 .87 47

Control 2.88 .85 40 .1.92 .76 37 2.68 .91 43 2.72 .92 44

Note: Due to small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth Committ members and controls at Muir High School,

aAn individual itemon the SAES student questionnaire.

b Measure based on school records.

**p4.01

65 6,1 Thhlo 14

trpaimeni and rolorol cnnwri.mn1 un

Sevoral IllitrouloN of Theofeliedi 101 oreqt

Talent'. ,rih.rik on Altarhment to IOW In II (Ii Internal Negative parents Conventional folio; Compotonoy Competency Control Veer Influec Ireltnnott Ii i ii and Croup Sd N Sd N M Sd N N Nd N M Sd N M Sd N H Sd

EMT JUNIOR HICH SCUM,

options 46 1,40 .48 .47 Treatment .71 .29 48 .64 .26 ' 55 .65 .24 46 ,84* .18 43 .28 46 .23 .21 Control .72 .28 29 .63 .25 33 .59 .25 25 .73 .19 27 1.26 ;43 26 .53 .24 26 .19 .17

Mall Con-ltice Treatment .81 ,26 34 ,481* .31 42 .70 .22 33 .79 .22 33 1.41 .50 32 .40 .26 33 .23 .21

.64 .29 70 .64 .25 56 .80 .21 56 1.38 ;43 '55 .46 .24 55 .24 .21 1 Control .80 .32 98

I-4 V) MUIR 111011 SCHOOL

Options .76 .23 34 1.348 .32 33 .44 .26 36 Treatment .61 .29 37 .49 .29 46 .59 .26 33 .20 .20 38 .76 .22 38 1.22 .41 38 .37 .29, 36 Control ,58 .30 38 '11 .23 40 .65 .26 .16 .17

Made Note: boo to mall number of non-mivalent Youth Committee controls (n4) meluingful conyarisons cannot be between Tooth Committee members and controls at Muir HO School. All outcomes are individual-level student scales from

the SACS questionnaire.

a At pretest,'pt Ins studonts were significantly higher than controls (pc.01).

ET COPY ARILABLE 66 . TABLE 14

cont'd

Rebellious

Treatment School Effort Involvement Invalidity Autonomy Victimization Punish Rewards and Croup M Sd N M Sd N M Sd N M Sd N M Sd N M Sd N M Sd

ter,.`.,. ELIOT JUNIOR 11011 8Q14.Q1,

Options

Trest rra .60 .33 56 .26 .20 51 .16* .24 45 .41 ,37 43 .20 .25 51 .31 .30 53 .36 .32 .27 Control .64 .24 31 .21 .19 33 .27 .24 27 .57 .28 25 .18 .21 30 .31 .25 30 .27

'Youth tionitter .36 .33 ?wont .55 .34 41 .350 .17 40 ,18 .25 34 ".66 .32 30 .15 .22 36 .29 .31 36 .31 .34 67 .34 .29 f7nttel .61 .32 72 .26 .18 66 .19 '.20 56 , .64 .35 55 .17 .23 64

I.`, .

P**1 ' VI r "' i1UIR RICH SCHOOL 141,--- .1 . r Optimum .63 32 .14 .21 43 .17 .24 44 .27 .29 Treatment .51 .32 46 .246 .17 46 .21 .24 34 .36 .11 .14 38 .11 .18 38 .21 .28 Antrul .66 .31 39 .19 .14 34 .11 .17 37 .63 .33 35 I

1When excluding controls who participated in the Youth Committee, Options were significantly higher than

1 controls (11(.01) Project RETAIN, Chicago Board of Education: Evaluation Report

J. St. John

Abstract

Responsive Education through Alternative Instructional Networks (Project RETAIN) was an alternative education project of the Chicago Board of Education sponsored primarily by a grant from the Office for Juvenile Justice and Delinquency Prevention (OJJDP) as part of its Program in Delinquency Prevention through Alternative Education. Operating in nine schools in the complex Chicago public school system,RETAIN soughtto useindividualized learning plans andthe attention ofspecial teachers to reduce -problems ofpoorschool attendance, disruptive behavior, and low student achievement. The evaluation provides no evidence that this project achieved its goals or the results sought by the OJJDP., This report describes the project and its evaluation.

RETAIN (Responsive Education absenteeism in the elementary Through Alternative Instructional schools. Disruptive behavior Networks) wasa project undertaken included minor classroom orschool bythe Chicago Board ofEducation disruption as well as serious with supportfrom the Office for assaults. Achievement problems were Juvenile Justice and Delinquency evident in the results of standard- Prevention (OJJDP) as part of that ized tests. The project considered office's Program in Delinquency Pre- poor attendance and low achievement vention through Alternative Educa- to be both individual- and school- tion (OJJDP, 1980). level problems.

Project RETAIN Plans Rationale

Goals The project's staff listed many causes of these problems, and RETAIN was aimed at the problems divided them into five categories: of poor attendance, disruptive (a) Parental and domestic causes, behavior, and low achievement in (b) school staff causes,(c) school Chicago public schools. Attendance administration causes, (d) community problems included class-skipping and causes, and (e) student causes. absenteeism in the high schools and Parentaland domestic causes range from child abuse,to keeping chil- dr.an home to babysit, to an inability to meet the school health requirements for enrollment. School This report supercedes an earlier staff causes include ineffective and report (St. John, 1982) on this pro- inappropriate teaching styles, cul- ject. It covers the entire two tural insensitivities and negative years of project operation. Donald biases, and self-fulling prophecies. E.Rickert, Jr., analyzed the data School administration causesrange for the second year of the project's from attendance policies (locking operation, and wrote the section school doorswhen the tardybell describing those results.

-197- 70 RETAIN rings) to unfair disciplinary Interventions as Planned and Modifi- actions and the escalation of minor cations incidents into major episodes. Com- munity causes include discrimina- First program year. During the tion, unsafestreets,and lack of first program year the RETAIN staff community support for the school. specified the following major inter- This lack of support occurs because ventions: (a) increasing parental the school "does not value community awareness, (b) linking families with participation" and because the "com- needed services, (c) developing munity does not view schools as a individualized curricula, communitystructurethat they are (d) advocacy, (e) peer groupcoun- involved in influencing and cooper- seling, and (f) continued in-service ating with." Student causes might training for.RETAIN teachers. These be economic concerns that conflict components were discribed in an ear- with school attendance, lack 'of lier report (St. John, 1982), and

motivation, not . valuing school, were modifications of the interven- negative attitudes toward the tions specified in the original pro- school, poor peer relations, lack of posal(Chicago Board of Education, social skills andself-discipline, 1980). low self-esteem, orlack ofbasic skills. The primary focus of the project became the implementation of Indivi- Although RETAIN did not discuss dual LearningPlans(ILP's). The delinquency as a problem or set the project's staff reported that ILP's specific goal of delinquency preven- developed .in _the_ early_ _stagesof _ tion, the project's theory contains implementation were not used effec- explanatory elements similar to tively. Accordingly, they decided those in Cloward and Ohlin's (1960) that more in-servicetraining for and Cohen's (1955) theories of del- RETAIN teachers was needed to assist inquency. them vita developing and using this approach to education. Plans for Objectives the in-service training were carried out in September, 1981, just as Although the project revised its school opened. list of objectives several times during the 1981-82 school year, the Second program year. The list of list eventually produced resembled a RETAIN interventions continued to list ofinterventions more than it evolve during the second program resembled objectives (intermediate year. By the end of the second pro- outcomes resulting from interven- gram year,project managers listed tions,as we understand the term). the following major project compo- In other words, the list of objec- nents: (a) parental awareness tivesproduced may have resembled activities, (b) efforts to establish objectives of the "management by linkages with Chicago Board of Edu- objectives"sortmore thanobjec- cation socialservice agencies and tives in Program Development Evalua- other community agencies and use tion sense. these services, (c) individual learning plans for attendance, Although the project's theory behavior, and achievement problems, specified the system's numerous con- (d) career education (individualized tributions to the problems at hand, and not for all participants), and the project's goals implied (e) computer-assisted instruction, school-level outcomes, no school- (f) summer '82 RETAIN program, level objectives were specified.

-198-

es, 71 RETAIN

(g) in-servicetraining forRETAIN boration of plans for evaluation was teachers, (h) staff intervention performed not because managers saw analysis meetings, and (i) evalua- evaluation tasks as essential to tion tasks including student selec- their project, but because the tion, questionnaire administration, national evaluators had observed data collection, and case study evaluation tasks were not always reports. These components are sucessfully carried out in the first described together with information year (see St.John, 1982). It was onthe degreeto whichtheywere hoped thatcarefulplanningwould implemented in a subsequent section result in a stronger evaluation. of this report. Intervention Analysis Meetings Many of the changes from year to were added by managers as a means of year were the result of attempts to increasing the quality andeffec- reorganize plans according to Pro- tiveness ofspecific actions taken gram Development Evaluation (PDE) for individual participants. definitions of interventions, objectives, strategies (see Gottfredson, Shifts among_ PDE categories. In 1982); some changes reflect the the first year, Advocacy (advocating amount of effort needed for success- for the rights and needs of partici- ful implementation; and some reflect pants) was considered an interven- major changes in plans. These first tion. However, a more detailed year to second year changes can be explanation revealed that this was a grouped into two categorieg, strategy used in the implementation (a) interventions dropped and added, of three components: Parental and (b) shifts among PDE categories. Awareness, Linking Families and Ser- vices, and ILP's. Interventionsdropped oradded. Onlythe Peer Counseling component Computer- Assisted Instruction was was dropped from overall project listed as a strategy associated with plans. Onefactor contributing to the ILP's in the first year, but in the discontinuance of the component the second year, was categorized as was scheduling. RETAIN students an intervention because of the con- were scheduled for project services siderable planning effort needed to such that often only one or two stu- make the computers operational. dents received services at a given time. Not allof the participants Career Awareness was shifted from wanted to be in peer counseling the objective category to the inter- groups and the chances of those who vention category because this was an did being scheduled to attend RETAIN activityplanned forparticipants. at the sametime were slim. Alt- This could be considered another of hough managers abandoned efforts to the ILP strategies, but since major implement this component at each efforts, especially in the summers, site, it remained the primary were devoted to this activity, man- approach used by one RETAIN teacher. agers decided to classify it as an Seven of the remaining eight RETAIN intervention. teachers reported using something they called peer counseling with The Implementing Organization some students in their school. Project RETAIN can best be under- Managers added extensive plans stood in the larger context of the for the successful implementation of Chicago Board of Education (CBE), their complex evaluation. The ela- the organizationthat was awarded

-199-

7 RETAIN the grant toconductthe project. However, GFP assigned a program Members of Board are mayoral appoin- monitor to ProjectRETAIN in the tees. Through a long chain of com- second program year. mand, the Board oversaw implementation of Project RETAIN (see the It was also unusual that the organization chart in Figure 1). administrators overseeing implemen- The General Superintendent of tation and project teachers had Schools, appointed by the Board, has direct contact. Ordinarily, project responsibility for implementing all management would only have indirect school board policies. The project contacts with teachers. Due to the itself was under the Deputy Superin- nature ofthe project's activities, tendent for Field Services, who contactbetween project management delegated direct responsibilities and RETAINteachers wasarranged; for overseeing implementation to the and formal structures- were main- Director of Field services (DFS), tained by communicating project Central. The project was placed in plans and activities to intervening Field Services because of its plans district superintendents and princi- to include non-academic services to ,,pals. participants and their families. The Department of Personnel hired a staff to work with the DFS to manage CBE Environment the project. The project management staff oversaw nine RETAIN teachers Project RETAIN operated in a com- who workedin nine Chicago public plex forsefield because of its schools. Eight of these teachers affiliation_with the: Board._ CBE is were selected earlyin the project currently underpressurefrom the according to Board guidelines,and Department of Justice to implement a one, whom a principal especially satisfactory desegregation plan. wanted in her school, was hired sev- During the 1981-82 school year, the eral weeks later. The third major courts and school board reached an division of the Board, Instructional agreement and theearlystages of Services, Government Funded Projects the desegregation plan went into (GFP), was involved because the pro- effect. This public schoolsystem ject was funded by the government; is one of the largest in the country this division oversaw budgetary and well over 50% of itsstudents procedures. are minority. Declining enrollments for the last several years have This arrangement was unusual for forced schoolclosings and teacher the Board: Most government grants lay-offs. A sizeable budget deficit are completely under the authority had to be immediatelyameliorated of the Deputy Superintendent of because state laws make it illegal Instructional Services, Government for public institutions to operate FundedProjects. The arrangement at a deficit. The Board of Educa- introduced unexpected elements into tion is under fire from parents and the project's forcefield as the two citizens groups, each with its own divisions of the Board developed particular(and often conflicting) workingrelationships. Both divi- concerns. Very few days pass with- sions had to be kept informed of the out news articles about the system project's progress. Initially pro- and its problems. ject managers expected that project planning and implementation would be During the spring of 1981, a new reported toand approved byField General Superintendent of Schools Services, and fiscal management mat- was appointed by the Board of ters would bethe purview ofGFP.

-200-

73 RETAIN

Education from outside the system, History of the Project causing uncertainty about steps that would be taken to solve the problems The proposalfor Project RETAIN of the system. Many new administra was developed in part in response to tors were appointed in 1982, includ public pressures on the school sys ing a new Deputy Superintendent for temto more adequately respond to Field Services. This change had the needs oftroubled students and very little effect on RETAIN opera schools. The Board of Education tions. assigned priority to alternative education and commisFioned the Pro A critical incident. The General gram Development Manager of the Cen Superintendent sought additional ter for Urban Education to develop funding forspecialprogramsas a such a programandsecurefunds. means of addressing problems in the Prior to thisproposal, oneother system. As part of this fundseek unsuccessful attempt at funding had ing activity she called on officials been made. at OJJDP to seek support for future delinquency prevention programs. The system had never before During her visit,she was informed implemented an alternative education that her systemcurrently had a projectclosely resembling project large grant from the agency, and was RETAIN,butthere have been other toldthat both theevaluation and projects withsimilar goals. The project activities needed strength ABC Project and the Family Guidance eningbefore morefunds could be Center were both aimed at increasing considered. OJJDP officials student achievement and reducing reported that she was apparently absente' 1-m and disruption. TAPI unaware of the project RETAIN--a was im to reduce truancy in circumstance that is not surprising schools plagued by this problem, and becausethelongchainof command worked with younger students identi keptthe project atsome distance fies as potential truants. Title I from her. prog. ams were planned to be imple mented in eastschool, but implemen On returning to Chicago the Geri'- tatica did r3t reach the final eral Superintendent met with project stages. TAP 1was revitalizedas managers, and subsequently appointed part of the desegregation planin the GFP monitor. She also approved 1981-2. the administration of the SAES stu dentquestionnairefrom which the Project RETAIN Staffing and CBE had censored important items in Resources the project's first year. CBE man agers were invited to brief high Project RETAIN had four manage level CBE administrators about their ment positions, nine teachers, and n project and its evaluation. Fora computerspecialist. The Direcf,.....,r time after these activities, the (not funded by the grant) had pr RETAIN manager' interest in PDE mary responsibility for communicat planningand projectdocumentation ing with other CBE administrators. increased. The Project Coordinator oversaw all project staff and the implementation of each aspect of the project. The coordinator worked directly with the teachers and kept the Director informed of progress and :.ssues. The Management Specialist wa3 pri

201 RETAIN marily responsible for fiscal feeder elementary schools (grades management and evaluative tasks, and K-8) or middle schools. These reported to both the Coordinator and schools were identified as having Director. The ActivitySpecialist serious problems with one or all of was responsible for working directly the project's target problems. with the RETAIN teachers in developing and implementing activities, and Elementary school level. Grades reported to the Coordinator. A Com- 7 and 8 are, for the mostpart, puter Specialist had responsibility housed in elementary schools in the for securing, installing,and pro- Chicago school system. These grades gramming a microcomputer system at are organized into classes which the project schools and for provid- stay together for the entire school ing in-service training in the use day mostly in one classroom with one of the computers. Teachers reported teacher. Some elementary schools directly to the Activity Specialist. have "walking" reading and math pro- Total funding for two years of programs, which mean the students ject RETAIN was $1.3 million, with change classrocms to attend classes most beingallocated to personnel ,,with specialty teachers. These costs. (The two-yeargrant from schools are usually small with 2 or OJJDP accounts for about $1.1 mil- 3 'lasses in each grade level. In lion; the balance was a CBE contri- some schools, each grade level fol- bution.) lows the same basic schedule during the day. TheChicago school system provided Project RETAIN with nine This highly structured _environ- teachers(whose salaries were paid ment helped structure the RETAIN from grant funds) who were to imple- project. At all elementary school menttheproject at each of nine sites, the RETAIN teacher had a room sites, including three high schools with a telephone set aside for and six elementary schools. The RETAIN activities. The RETAIN project managers conducted pre-ser- teachers generally visited other vice training in 1980-81 to acquaint teachers each morning before school thoseteachers with the goalsand started. This was done to check on objectives of the project and to participant attendance, achievement, assist them in developing new and classroom behavior from the pre- approaches tothe students who are vious day. They returned to the often neglected bythe system,and RETAINclassroom to stand in the ongoing inservice training in hall and greet students arriving for 1981-82 for continued assistance. the school day. When a participant was not seen, other locations in the school were checked, and if the stu- The RETAIN Structure dent could not be found, the RETAIN teachercontacted the parents by Project RETAIN served three clus- phone. In this way, the project ters of three schoolseach: The teachers monitored the attendance of Northside (School District 3), and had daily contact with partici- Southwest(School District 12) and pants. Southeast (School District 19) communities. These clusters were cho- Participants usually attended sen because they represent three RETAIN in small groups ofthree or different population groups from the four during the school day. Small city and are located in high crime numbers allowed RETAIN teachers to neighborhoods. Each cluster is com- work with students either in small I-Jsed of one highschool and two

-202- RETAIN groups or individually and provide period. Students who had not either academic instruction or coun- reported in the morning and who were seling. The results of some stu- not in their classes were thought to dent's learning style inventories be absent and thier parents were to suggested that small group instruc- be called. Students whowere in tion was appropriate for them; oth- attendance but not in their classes ers needed individualattention or were to be counseled for class skip- to work mostly on their own. RETAIN ping. teachers attempted to schedule studentsfor whom small groupswere Participantswere scheduled to mostappropriate at the sametime attend RETAINeither during their period. But scheduling constraints study hall time or before school did not always allow for this--only started in the mornings. The stu- certain time periods were available dent's schedules determined when for attending RETAIN both because of they could participate in RETAIN and student schedules and because RETAIN how often during each week. If a teachershadotherproject duties student were selected and his or her eachday. RETAIN teachers hadto schedule conflicted with thetimes have large blocks of time each week the RETAIN teacher had available for for making home visits and for classroom work, arrangementswere attending every-other-week project sometimes made tochangethe stu- staff meetings. Project staff dent'sschedule. When all routes reported that these constraints also were blocked the RETAIN teachers and resulted in a catch-as-catch-can participantsworked out the best schedule resulting in varying possible arrangement for early or amounts of contact with RETAIN par- late inthe daycontacts. RETAIN ticipants. teachers tried to schedule students with similar problems and learning High school level. Chicago high styles at the same time, but because schools are large schools that of scheduling problems they were not include grades 9through 12. Most always successful. of these schools have staggered start-up times for each class level. Participants. The project plans The daily schedule involves changing specified that 15 students would classes every 52 minutes for 7 to 10 receive direct services at each class periodseach day. Students school. Students were to be chosen arenot necessarily scheduledfor randomlyfrOmpools of referrals. instruction each class period. School staff were asked to refer students with attendance, achieve- The school organizationhelped ment,or behavior problemsfor any structure the high school, level combinations ofthese problems) tp RETAIN project. Each RETAIN, teacher the RETAIN teacher. Parentalcon- had a phone and classroom,or area sent was required for participation. of a classroom, reserved for RETAIN activities. Generally attendance Length of participation. Parti- was monitored by the having partici- cipants were to remain in the pro- pants come by the RETAIN classroom ject until the RETAIN teachers before reporting to their homeroom. judged thatthey no longer needed Later in the day, some RETAIN teach- alternative education. The project ers went on what they called didnot specify how muchcontact "rounds." This activity involved each student would have with RETAIN, checking the classrooms students nor did it establish guidelines for were supposed to be in at that time the termination of participation.

-203-

76 RETAIN

Forcefields Other schooladministratorsand the Board supported the project, but Eachteacher had responsibility the project managers spent much time to clear with the principal any working through bureaucratic proce- alternative procedure or policy the dures. project planned to implement in the school. For instance, in most pro- Cross site variations. Extensive ject schools, the RETAIN teachers cross-site variations in the imple- wererelieved of hall monitoring mentationof theprojectresulted responsibilities because the project when the project caved in to princi- required them to leave the campus to pal demands. At Bowen and Gage Park contact families. Regular responsi- high schools and Bontemps Elementary --bilitiesintheschool would have School, the principals required that hindered the implementation of RETAINteachers service morethan important project interventions. the 15 selected participants. At Project RETAIN managers reported Thorp elementary school the princi- that not all principals were suppcir- pal used the RETAIN classroom as the tive of the project efforts in their ,site foran experimentalin-school schools. suspension project. The RETAIN teacher had responsibility for the Problems in implementing some of suspended students and generally the program strategies at some involved them in RETAIN activities. schools, including the advocacy role The necessityto beat school all for RETAIN teachers and release time day for this non-RETAINactivity for RETAIN teachers' homevisits, prevented this RETAIN teacher from were never overcome. The principal making home visitsduring working at one of the schools reported that hours. At Blaine Elementary School, he suspected the teacher originally the RETAIN teacher was often called hired to implementRETAIN at his upon to counsel non-RETAIN students schoolwasabusingthe off-campus referred tothe office for discip- arrangements for making home visits. line reasons. Several RETAIN teach- In the Fall 1981 school semester, he ers reported that principals called blocked implementation until the on them to contact the families of teacher wasreplaced with someone non-participants because allother more to his liking in January 1982. attempts had been unsuccessful. At Bontemps, the RETAIN teacher This action (at Sheridan) created explained that one ortwo students another problem. The project Activ- had been placed in RETAIN sothat ity Specialist had to spend a great the families could be contacted to deal of time training the new staff give consent to assigning the stu- member. Furthermore,the principal dent to the Special Education track. continued to disallow home visits. RETAIN teachers repOrted that the The special activities and status difference between the RETAIN parti- of the RETAIN teachers could have cipants andnon-participant reci- led to tense inter-teacher relations pients of services was that no ILP's if the project's emphasis on helping were developed for the non-partici- students with special needs were not pants, and contacts were less regu- accepted byallteachers. Faculty lar for most of these students. At contacts were maintained as a means most sites,principal demands were of building support among peers for handled as additional responsibili- the project. ties. Managers reported these unanticipated variationsdid notharm

-204- RETAIN

RETAIN implementation. Only at was implemented is based,then, on Sheridan Middle School was it neces thedocumentsfurnished byRETAIN sary to drop a component. managers, observations at each school, and commentsmade during What Was Implemented? interviews with RETAIN staff.

The projectdid not participate Parental awareness. Conferences in ProgramDevelopment Evaluation withparents continued to be the (PDE) reportingprocedures in the primary method of making parents way we had intended. A management aware of the educational problems plan was notcompleted during the their children experienced and first PDE planning conference,nor involving parents in the solution of was much systematic planning docu these problems. Therole of the mented. In the first year, general RETAIN teacher in these-sessions was reports of successes were often that of an advocatefor the stu given. These reports usuallytold dents. Many conferences took place of necessary successes in procedural in the homes of the students. matters within the bureaucracy. Implementation in the schoolswas At Nightengale Elementary School not documented, and so can not the RETAIN teacher organized a regu really be described. lar weekly parents meeting. Parents at this school were assisting in the During the second program year project and three parents wereat projectstaff struggled with'docu the school theday I visited. I menting programmatic efforts in also saw pictures in the RETAIN their PDE plans. Although we classroom of social events sponsored attempted to clearly describe expec by the project for both participants tations, managers reported a lack of at that school and for three RETAIN understanding of the expectations projects in nearby schools. Project for planning and reporting. Several managers indicated that the teacher timesthey commented onthe bulky was successful with the council plan that would result if they docu because ths,1 neighborhood had long mented projectactivities at each time residents who had always been school. Such documentation would active in school activities. At have been necessary because each Blaine and Bontemps Elementary component had some exceptional vari Schools, the RETAIN teachershad ation at one or more ofthe sites. implemented individual counseling PDE plans were never developed; with several of the students' fami other documents were substituted. lies in an effort to improve parent/child relationships. Some Site visits were conducted as a times the teachers helped families means of gathering information about begin counseling at local community the project operations atthe nine mental health centers. RETAIN man schools. All RETAIN teachers were agers also developed three newslet interviewed about how they executed ters and sent them to the parents of their understanding ofthe project participants. plans, a sample of ILP's were reviewed, and teachers were observed Linking families and services. while working with students. Seven The Chicago Board of Education is a of the principals at the implementa city agency. This status could link tion sites were interviewed. the Board with other divisions of city government such as law enforce- The following description of what

205

78 RETAIN ment, the judiciary, health and based on this information. The human services, and political bodies teachers were also expected to det- such as the City Counciland the ermine their students' individual Mayor's Office. Project RETAINis learningstyles and planteaching therefore potentially linked to methods that matched student learn- these organizations through the ing styles. Once thesedata were Board. gathered, students and teachers were expected to develop a contract that The Board of Education maintains specified long term goals, short many social and supplemental educa- term objectives, the activities tionalservices through its three planned for reaching the objectives, Pupil Service Centers, and by virtue and the means of regular evaluation of the project being a part ofthe of the students progress. For exam- Board of Education, theseservices ple, a student who hadbeen chroni- were automatically linked. Project cally Absent might have as a goal to managers reportedmaking arrange- attend school every day he or she is ments for quicker processing of not ill. The short-term objectives requests for services from these ,might be tofirst begin attending centers. three days a week, then four,and finally five, with attendance RETAIN teachers and staff estab- reported by thestudents' regular lished relations with community ser- classroom teachers as the means of vice organizations and businesses evaluating whether or not the objec- near some oftheir schools. Three tives were reached. schools reportedly established informal RETAIN Parents Councils Individualized 1-31-ani -were- to that included members from community cover any individual counseling organizations. RETAIN managers also needs withwhich RETAIN teachers acquired a list of service organiza- could help. The plans were also to tions categorized by geographic area specify individual career interests, and type of service and distributed plans for increasingawareness of the lists to the RETAIN teachers. jobs in fields of interest, instruction needed for particular jobs, any Although project managerspro- job seeking-skills that may be vided little in the wayof guide- needed,and appropriate job place- lines for determining whether or not ments for eligible students. Compu- to refer families for services, the ter- assisted instruction was planned teachersapparentlyfollowed their as one aid to implementation of hunches about family circumstances learning plans. and consulted other people acq- uainted with the students. Clearly The project managers encountered somedecisions needed little gui- difficultiesin theirattempts to dance--one teacher reported she had standardize the ILP recording format referred a family to the local men- across schools. Despite the manag- tal health center because the father ers' honest efforts to devise a com- was an alcoholicwho Abusedhis mon format bythe end ofthe year child. there were as many recording formats as there were RETAIN teachers. The Individualized Learning Plans Project Coordinator reported that (ILP's). RETAIN teachers gatiL.slred this was because each teacher felt social andacademic historiesfor more at ease with hisor herown the participants and devised indivi- recording format. dualplans for achievement,attendance, and behavioral objectives

-206-

.0 7(9 RETAIN

RETAIN teachers varied in the equipment. Later, arrangements for degree of specificity of both long- electrical outlets and overnight term goals and short-term objec- security slowed down installing the tives. Several reportedconfusion equipment. Then difficulties overthese details. Despite these acquiring software (computer pro- problems, eachcurrentparticipant grams) were encountered when project ateachschool had a filefolder managersfound tha$: the Board of containing records of goals and Education alsohad guidelines for objectives contracted for, and lists what could and could not be of actions taken by eachteacher, installed on the computers. The dates the action was taken, and some computerexpertonthe management report of results of the actions. team developed a few small programs for thecomputers so teachers and Students reported they knew about students could begin using the their plans and contracts; several equipment, and he conducted on-site could state some of their goals and workshops in the use of the the reasons they thought working to machines. The software was eventually acquired and installed, but one achieve the goals would be benefi- I, cial. Students reported they were of the schools had its computer sto- aware of most of the actions taken len. Parentsat thisschool we're on their behalf by the RETAIN teach- working to replace the computer when ers, and many reported they appreci- I visited the school in March. ated the caring and help received--even when the help involved At some schools the RETAIN teach- the dreaded parent contacts. ers regularly used the computers to give additional instruction in math Career awareness. Project staff and English. One student reported reported that records of specific that themachine wasfun to use career awareness activities for the because it divided complex problems individual students appeared on intolittle stepsand immediately their ILP's. Some of the approaches reported to the student whether or included the use of CVIS available not each step had been properly exe- on the Board of Education's downtown cuted. Another studenthad begun computer, speakers, visits to job writingsimpleprograms that made sites, study of occupation-specific the computer print out geometric preparation units before visiting designs. The computer kept indivi- sites, help with job seeking skills, dual loge of student use that and the "World of Work" component of included amount cf time, units com- the 1981 summer program. pleted, and degree of mastery in each unit. Teachers -4housed the Computer-Assisted Instruction. computers recorded the computer log This component met with obstacle information on students ILP's. after obstacle and, although planned to begin in the first year of imple- Inthe beginning of theschool mentation, did not get off the year project managershad planned ground until the fall of the second forstudents tobe tr:',1sported to year. The first unanticipated DePaul University for additional obstacle was that the Board of Edu- computerinstruction. These plans cation guidelines required that the had to be abandoned because ofthe purchase of the equipment becon- high fees that would have been ducted through a competitive bidding charged, large amounts of time that process. This unanticipated would be 'needed r:o transport requirement delayed obtaining the dents to theUniversity and 't,ack

-207- RETAIN again, and because University and (d) determine individual stu- personnel expressed some reserva- dents' optimal learning styles. tions about having groups of young Teachers were alsoto learn about students using their facilities. family services available through theBoard of Educationand, other Summer program. During the sum- community agencies. mer of 1981, the project implemented a special program to work with all For the most part, project manag- RETAIN students for the entire day ers did not make specific plans for at the nine school sites. The sum- specific types of in-service train- mer program emphasized career educa- ingafter the initialpre-service tion and remedial instruction. They training. The management style was had hoped to employ the RETAIN high one of informally surveying teachers schoolstudentsastutorsfor the to find out what areat of respon- elementary students with CETA fund- siblity they thought they could use ing but were unable to do so. someadditional trainingin then trying to identify resources for Earlyin the 1981-82 year, pro- delivering the training. Often the ject managers had not decided managers learned of particular work- whether they would implement the shops thatwere planned elsewhere summer program fora second year. and spontaneously arranged for The final decision rested on whether release time This lack of specif- or not the Board of Education plan- icity became clear when project man ned to have any schools openfor agers asked Polaris,the technical summer school. We do not know if a assistance contractor for the OJJDP summerprogram wasimplemented in initiative, to come and train teach- the summer of 1982. ers in effective peer counseling methods. The impression ofthe TA In-service training. The forego- contractors was that the project ing list of c*ponents planned as managers were unclear about how the direct services to students is indi- proposed training in peer counseling cative of the variety or skills the fit into their project plans. Alt- on-site implementers must have had hough peer counseling was initially in order to successfully serve stu- planned asa component of the pro- dents as planned. Because teachers ject, it had, by thattime, been selected to implementthe project abandoned fornumerous reasonsat had, for the most part, been class- all sites. Theproject managers' room teachers, project managers felt decision to seek trainingin peer that many hours of in-service train- counseling when they did was somew- ing would be required. Project man- hat surprising. agersbegan the training by acq- uainting the teaching staff with the Interventionanalysismeetings. project goals and planned interven- By January of 1982, project managers tions prior to student selection for added a new component for their participation in the first program teachers called Intervention Analy- year. They also thought in-service sis Meetings. This was planned to trainingshould occurperiodically be a means of helping teachers to, throughout the implementation period solve problems through the presenta- so teachers could acquire skills and tion of case studies at staff meet- information needed to (a) implement ings. Other members of thestaff thecareer developmentcurriculum, were expected to analyzethe case (b) improve student reading skills, based on the presented materials and (c). use computers as an instruc- suggest possible interventions. We tional aid, (c) counsel students,

-208- RETAIN have boon unable to determine the questionnaires were lowinthe whether thiscomponentwas imple- first year's administration. Manag- mented. ersindicated that principals saw the questionnaire as a disruption to sFYs'1jg1Lcti.vities. At the their school and as having little or August 1982 planning conference, no value. Managers therefore made project managers developed extensive plans for helping principals by plans for carrying out each task of developing specialreports on some their evaluation. They had been of the questions and reviewing the unable to gain approval in the first results they thought principals year to administer the SAES questi- would find to be useful information. onnaire containing the self-reported Thesetasks were executed, but it delinquency items. The Chicago had little effecton the response Board of Educations censoring these rates in the second year. items from the questionnaire resulted in no assessment of the Another plaguing problem was the primary result sought by the spontor likely breakdown of the experimental of this delinquency prevention pro- design atsome schools. Post -ran- ject because the Board also blocked 'domization checksshowed that ran- attempts to acquire official delin- domization had failed in several quency datafrompolice or court schools in the project's first year. sources. Managers reported that Managers decided to resolve this failure in this area was primarily problemby using a random number due to a fear on the part of admin- table to make the selections at the istrators and principals that RETAIN central office. This proved parents would object or cause prob- to be an unsatisfactory-solution-to lems for principals and the school the problem when evidence of another system if they knew the project was breakdown in randomization began to part of a delinquency prevention accrue. initiative, or knew about the sensitive nature of some of the questions Because the project also had being asked of their children. school-level goals, plans were made for reporting school level data to In the second year, project man- the national evaluation team. The agers set a timeline for contacting projectidentifiedseveral control high-level administrators to gain schoolsto usein the comparative support forthe questionnaire, but analysis, and decided to administer theirplans did not include any the questionnaire in oneof these strategies for helping the adminis- schools. These plans were not fully tration cope with irate parents executed. No school-level data were should thisunlikelyevent occur. reported for any of the schools. As it turned put, the matter became moot when the questionnaire was Problems also surrounded indivi- brought to the attention of the Gen- dual-leveldata collection activi- eral Superintendent after her meet- ties. During thefirst reporting ing with OJJDP officials. We wrote period of the project's second year to the Superintendent requesting staff at t1 schools who collected approval of the standard student and reported data on the treatment questionnaire for administration in and control groups misunderstood Chicago. The Superintendent reluc- some printed instructions. At the tantly approved the questionnaire. endof the first reporting period this problem was discovered, and by Overall school response rates on

-209-

82 -4,.4 RETAIN the last data reporting period codes comparison groups at two schools. werebeingproperlyusedand all In one of these schools, the control instructions were followed. group contained a high proportion of special education students. Random Other components. Managers gave ization may also have broken down in oralreports aboutspecial activi a third school. A significant dif ties which some of the RETAIN teach ference in the number of siblings ersimplemented at their schools. reported by treatmentandcontrol At Blaine Elementary School, the students was found in that school. RETAIN participants wrote and per As a result, project managers com formed a Play for their school. The mitted themselves to conduct the Northside cluster took a field trip randomization process and use a ran to Evanston forthe experience of dom numbers table for the selections attending a professional perfor in the second year, and the place mance. ment of any student in the project by any other means was to be docu Effectiveness mented.

Evolution of Evaluation Design Second year. Although we urged that randomization be conducted at First Year. The project managers Johns Hopkins, project managers established procedures for random demurred. Several indications imply selection of students at each that the experimental designalso school. More than 15 students were broke down in the secondproject referred at each school; thus selec year. Some students who had parti tions had to be made in some fash cipatedin theprogrampreviously ion. Randomization was agreed upon were reenrolled in the program. because it seemed administratively And, project managers usedrandom simplest. Principals, counselors, number tables to select new partici and RETAIN teachersthen made the pants for the second yearbefore- selections by drawing names of stu obtaining parental permission for dents referred to RETAIN out of participation in RETAIN and the L,istsLists of RETAIN participants, evaluation. Many students selected controlstudents, and waiting stu for the control groupwere then dents were developed. The project excluded from the study when paren reported that there were only a few tal permission could not be students in thenine schoolswho obtained. We experienced considera were placedin the project byany ble difficulty in determining method other than drawing names from exactly what happened to the design. hats. When we discovered that implemen This method of randomization did tation had been delayed in one of not produce equivalent groups of the project's schools (Sheridan), we students. Post randomization checks again requested to conduct randomi revealed significant differences in zation at Johns Hopkins. This achievement (standardized test request generated some ill will, but scores and grades) for the was acceded to. A onesemester true experiment was definitely conducted at Sheridan, although project imple mentation was weak in this school. 1. Alas,they were told to"draw names from a hat" by the evaluation Both years. As with all projects field worker initially assigned to in the Alternative Education Pro work with the project. gram, project staff were expected to 210

83 RETAIN

take all necessary stepsto secure In one project high school information pertinent to the evalua- (Bowen) RETAIN students were absent tion from school and other archival significantly more often than con- records, and to administer the trol students (6.11daysvs. 2.67 School ActionEffectiveness Study idaa).on average, respectively, student questionnaire each spring. In Blaine Elementary Archival data were to be collected School, Interpersonal Competency for RETAIN participants and control scores were higher for RETAIN than group youths, and questionnaires for control students (4.07 vs. 3.21, were to beadministered to these p<.05).2 (This is the school in youths and to probability samples of which RETAIN andcontrol students students from participating schools. differed in number of siblings.) Survey administration was to be Several differences between RETAIN guided by administration manuals and control students were found in supplied to the project. Bontemps Elementary School. RETAIN students were significantly lower in Interpersonal Competency than the Results First Program Year controls (2.96 vs. 4.30, p<.05), higher in Rebellious Autonomy (2.21 The clear evidence of a breakdown vs. 1.37, p<.05), higher in self-rein randomization in two schools, and ported suspensions (.76 vs..28 per the possibility that it failed in a student in thepastyear, p<.05). third, made us wary of assuming that Finally, in Thorp Elementary, RETAIN randomization was implemented in the students scored higher than control remaining schools. Because we did students in Attachment to School not supervise the randomizatioa, we (4.19 vs. -3.69, p<.01).- have no direct knowledge of the procedures used. Despite these wor- These interim results were inter- ries, some significantdifferences preted with caution for several rea- between RETAIN and control students sons. First, we were uncertain that were found andmerit discussion. the RETAIN and control groups were Interpretation should be tempered by truly randomly equivalent. Second, awareness that (a) randomization may the available data were dredgedto have broken down inseveraladdi- detect significant differences, and tionalschools, if not in all of when many significance tests are them, and (b) interventions were conducted a few may be "significant" implemented in limited forms, and for by chance. Third, the project only a short period of time. implemented its interventions in weak form. Although detailswere No statistically significant dif- sketchy,ILP's were apparently not ferences were found on any cognitive regularly updated, the parent variables, after the two schools where evidence of non-random assignment involved cognitive variables were excluded from analyses. Mea- 2. Descriptions of the measures sures of delinquent behavior were used in the School Action Effective- not available for analysis: Nearly nessstudy during year one maybe all self-reported delinquency items found in thefirstinterim .report were censored from the questionnaire (Gottfredson, 1982) and descriptions bythe Chicago Board of Education, of the slightly modified measures and the Board refused to seek offi- used during year two may be found in cial delinquency data. the overview section ofthe second interim report (Gottfredson,Gott- fredson, & Cook, 1983).

-211-

84 RETAIN component did not go as expected, administration was poorly imple and the intervention was brief. mented in several schools. Propor Questionnaires were administered tions of questionnaire data missing only 8 to 10 weeks after implementa foreachof theproject'sschools tionof the projectbegan. These are shown in Table 4. The question results apply only to the interven naire, although a posttreatment tions, as implemented, not to the measurement instrument, contains interventions intended. measures on several measures,such as age, sex, race and parental edu cation that are considered to be Second Year Results; background measures or measures of stable personal characteristics that Duringthe 1981 82school year, could probably not be affected by a 163students in ProjectRETAIN'S treatment. They are thus usefulin nine target schools were identified assessing the pretreatment equiva as being control students, and 145 lence of treatment and comparison were identified as treatment stu groups. dents. Table 1shows the number of treatment andcontrol students in Analysis proceeded in several each school. Usable pretreatment steps. First, treatment and control data froze records forthe 1980-81 students were compared on available schoolyear were provided on stu pretreatment and Student Question dents' gender, age,fourth quarter naire background "variables. These English grades, Iowa Test of Basic comparisons wereperformed at the Skills (ITBS) scores in reading and project level (i.e. all project math (for elementary school students treatment students vs. allproject only), number of high school credits controls) and were attempted at the earned (for high school students school level (i.e., treatmentvs. only), and suspensions. An unfortu controls in each of the target nate amount of data were mis schools). Comparisons at the target sing--ranging from. 21% missing for school level are largely uninterpre English grades to 35% missing for table due to small N's and many suspension data. As Table 2 shows, missing data. there was considerable variation across schools on the degree of -If the projecthad substantial missing data. effects, this should show up in the projectlevel comparisons, where, Outcome data from records for the despite the unknown bias due to 1981-82 schools year were provided missing data, the N's are large for attendance, suspensions, Eng enough to allow a reasonable chance lish, math, science and social stu of detecting statistically signifi dies grades, and withdrawal from cant differences if they exist. school. The amountof missingor Comparisons of treatment and control incomplete data ranged from 0% for groups were madefor outcome mea withdrawalsto 60% for attendance. sures from school records as well as As was the casefor pretreatment outcomesmeasured by the Student data, there was considerable varia Questionnaire. For outcomes for tion across schools (see Table 3). which the difference between treat ment and control students was signi Completed or partially completed ficant, subsequent analyses examined Student Questionnaires (administered the data to determine whether treat Spring, 1982) were returned for 65% ment by school interactions were of the treatment students and 52% of present. A significant interaction the control students. Survey 212 RETAIN would suggest that the project was tical controls wereapplied in a more effective in one school or sec regressionmodel,3 no significant of schools than in others. differences between groups remained, except for seli'-reported employment Because the randomization process and dropout in school 1240 only. It had broken down, for treatment-con- turns out thw.the difference bet- trol group comparisons in which ini- ween treatment and control students tial analyses found significant dif- on dropout ratein school1240 is ferences, statistical controls were due to a very high rate among con- applied in a stepwise regression trol students.4 These results may be analysis to "control" for pre-exist- interpreted assuggesting that the ing differences between groups. observed differences between RETAIN and comparison youths can plausibly Post randomization checks. When be attributed to pre-existing dif- all project treatment students were ferences; they may not be attributed compared to all control students on to the program's influence with any available pre-treatment and back- confidence. ground data, no significant differences were found (see Table 5). Discussion Furthermore, differences between the two groups were small in comparison The present results are subject to the individual differences within to several important limitations. groups (compare the differences bet- The most important limitation is the ween means in Table 5 with the stan- overall poor quality of the data. dard deviations, which are indica- Missing and incomplete data, and the tors of the extent of individual variations across schools in quality differences). Comparisons on pre- may introduce bias of an unknown treatmentand background variables were also performed school-byschool. This time, several signifi- cantandnearsignificant differ- 3. A control variable is a variable encesappeared, despite the small that is correlated with the outcome N's (see Table 6). Because randomi- variable of interest which is some- zation wasto occur atthe school times introduced into an analysis to level, thisset of results implies adjust for pre-existing differences that the evaluation must be regarded between groups when the groups are as involving a non-equivalent con- not randomly equivalent. The fol- trol group design, because randomi- lowing variables were controls for zation was not effective. the regression ofdropoutrate on treatment: age, suspensions in Outcomes for treatment and com- 1980-81,and absences in 1980-81. parisongroups. Post-intervention For the regression parental emphasis comparisons between treatment and on education on treatment, the fol- control group students are summar- lowing variables were utilized as ized in Tables 7 and 8. These are controls: parental education, dummy simple raw comparisons, with no sta- for school 4440, and the dummy for tistical controls. Threeoutcomes race (1=white, 0=other). For the favored the treatment group. These regression of practical knowledge on were: dropout (p<.01), practical treatment, these variables were used knowledge (p<.05), and self-reported as controls: sex, dummyforrace employment (p<.01). One result (1=white, 0=others) and the dummy favored the control group--parental for school 1240. The control varia- emphasis on education. When statis- bles used for the regression of

self-reported employment were : sex

-213- RETAIN nature into statistical analyses. elsewhere. (d) The activities at It also weakens analyses by decreas- Bontemps should be have been care- ing statistical power, i.e., fully scrutinized to s:L., if changes decreasing the chancethat differ- could have been made tc -,void what ences that may exist will be appeared to be negative,*sects on detected bya statisticaltest for interpersonal competency, lllious significance. A secondlimitation autonomy, and possibly sui:p,sions. is the non-equivalence of groups (e) The activities atThori, ,bould prior to treatment(SeeTable 5). have been examined to learn w;.,?.77 Groups wereto have been randomly project activities that contrit,...i.i assigned to treatment andcontrol to student attachment to sclloo conditions, but this was either not couldhay.i. been identified and pvt

accomplished or was unsuccessful. into practi . elsewhere. Finally, we judge the implementation asnotparticularly a strong one, Possibly, of course, thcstl and therefore statistical results results were cttributable to should probably be interpreted 'as breakdown of r:ldmization. Ther- 'representing the results for a ,fore, we sugge,t--- ,2 that the proj...ct rather weak set of interventions. carefully review -bow students vere selected for participation in RETAIN,and consider whether .these Results of First Program Year results may Lave been dueto some artifact of the selection procedure. Despite the limitations, the Although project managers made some implications of the first year attempt to improve their evaluation, resultsfor the project were sev- there is little evidence that any of eral. Earlier (St.John,1982) we these suggestions were followed up listed several suggestions: with much zeal. (a) Steps should have been taken to assure that randomization was implemented. (b) The project should have Second Program Year Results examined activities at Bowen to determine whether any aspects of RETAIN Taken Logether,the results for activities contributed to stude,v- the second year provide little evi- non-attendance. (c) The project dence that RETAIN accom:Lished should have examined its activit. either its own goals or the results at Blaine to learn whether project sol,ht by OJJDP. This outcome may activities that contributed to be the result of (a) a weak evalua- interpersonal competency could be tion, or (b) a weak program. identified and put into practice PDE and The Proiect

Project RETAIN managers did not andthe dummies forschools1240, fully participate in a Program 6180 and 5090. Development Evaluation of th,s kind 4. Forcontrol students in school we had envisioned would be imple- 1240, the sample mean for dropout mented. Theywerecoopera-ive in rate is .55 with an SD of .51 the summative aspects of evaluati...... , (N=18). For the treatment group the electing toattemptimplementation sample mean is .13(SD=.35, N=15). of a true experimental design and to See Table 7 for the sample means and report school record data on indivi- standard deviations forthe entire duals in a timely and frequent fas%- sample(i.e.all treatment vs.all ion. They did not, however, furnis: control students).

-214- 87 RETAIN systematic implementation data to us outcome as an objective of their as requested. We do not fully interventions because they said they understand the reasons we were una did not want to be held accountable bleto obtain implementation data, for raising selfesteem, a rela the reasons for the extensive miss tively stable characteristic that is ing data problems, or the reasons difficult to change. for the failure of randomization.5 Project managers appeared some Goals. Project managers partici times to use the word "objective," pating in the firstPDE planning in ways other than that intended in workshop were clear about the prob the PDE method. In other planning lems the projectwas designed to models, "objective" is often used address. Therefore they were clear synonomously with a process or about the goals ofthe project and activity. PDE requires abandoning the target population that was that .usage and learning a more appropriate for the direct services research oriented meaning of objec component. tive. Activities or processes are called "interventions" inthe lan Theory and objectives. The pro guage of PDE. ject's rationale pointed to many sources ofthe problems RETAIN was intended to address, and some of the The Match between Theory and Inter impliedcorrective activities were ventions incorporated into planned activi ties. For the most part, the plans The project's theory described developed failed to clearly specify the contributi,us or students, the intervening outcomes that would parents, community, andschool to have to be achieved.We sensed some the problems that students experi reluctanceonthepart of project ence. Itfollows,therefore, that managers to make clear statements in objectives and interventions should thisarea. Forexample, although be aimed at system chwles, with at managersstatedthatone cause of least as much vigor as the indivi the problems that RETAINwas to dual focus. Although plans ini address was ravaged selfesteem, tially included several intermediate they would notstatethis obvious outcomes and project activiti,:s that were related to chances in the sys tem, nearly all were dropped. Only at Thorp was there anexperiment 5. It may be ofsome significance with an inschool suspension project that, after extensive discussion of initiated by the principal. Except the advantages and disadvantages of insofar es the use of ILP's and the alternative evaluation designs at attendance methodology were viewed the January 1981 workshop where the as a pilot or demonstration of these experimental' approach was decided techniques, RETAIN may then be char upon, the reason project managers acterized as a project aimed at chose to implement randomization assisting individuals in,arsisting rather than alternatives (e.g. despite the system. If the pro matched control groups, regression ject'stheory wascorrect, RETAIN discontinuity, time series) was that wasprobably not able to achieve they judged it to be administra dramatic effects when no system tively simpler to accomplish. In changes were made. The focus on 15 actual practice, maintaining the individualsmay not be a strong randomized treatmentcontrol group enough interve,.:tion to affect a design was not simple. whole school--even a small one. 215

88 RETAIN

Schoollevel. From the outset Government Funded Programs monitor project staff stated schoollevel in the second year did not help to goals ofincreasing attendance and resolve the management difficulties. reducing disciplinary incidents. Although the schoollevel interven Teachers in the schools tried tions were dropped,the goals were valiantly toimplement the project clung to as if the project's success tothe bestof their ability,but were riding on having reached these lacked guidance and training in the goals. many skills needed. Students at all sites seemed to be attached to the The project managersexplained RETAIN teachers and grateful for this was because Project RETAIN was their help, but management practices itself an experiment with new. poli resulted in nine different programs ciesand procedures. They-ft argued evolving (one in each of the that implementation ofthe' project schools) rather than the development was therefore evidence that the of a single cohesive program. goals were met. As for r1 creating broad changes in school policies or The large Chicago Board of Educa practices,the staff reported that tion bureaucracy is a highly politi this would have taken years of cal organization. Managers were effort, involving every level of the sensitive to the politics of the Board in planning and approving new system. They appeared less sensi policies. Furthermore, they were tive to the demands of developing a not policy makers, but project strong set of interventions, and had implementers responsible for a pro difficulty in working with the pro ject serving 15 students at each ject's complex forcefield to develop school. They did indicate that the the project. The very nature of the project was being carefully watched project meant that the system had to by administrators andthat if the be modified at severallevels to alternative policies used in the accommodate it. Project activities project appearedto be successful, and organization required that new policy makers would consider adop lines of administration be developed tion in the system. to assist in the direct contact bet ween RETAIN teachers and managers. Conclusion Without these arrangements, the question of who was in charge--prin Project RETAIN suffered from man cipal or project managerswas left agementdifficulties. The CBE is wide open, and the project took on a large enough that these problems different character at each school. apparently were undetected (just as The project dropped entirecompo the Project itself operated without nents at some schools, andadded the knowledge of the General Super additional activities at others,in intendentuntil hercommunications large part in accommodation to the with OJJDP) despite each project principals ofthe severalschools. modification being cleared by a DFS, Newpolicies were needed for the an administrator placed rather high implementation of many planned in the system. Although the manag activitiessuch as campus release ers had all worked witht this DFS, a time for home visits. formal hierarchy in decision making appeared to be therule, perhaps It may have been helpful if pro accounting for the minimal awareness ject managers had been empowered to of Project RETAIN at higher adminis work withprincipals to planand trative levels. The assignment of a implement experimental policies and procedures that would have affected their schools. Managers did not perceive they had this power. 216

SY4 RETAIN

References

Chicago Board of Education. Project RETAIN (A proposal to the ()ffice for Juvenile Justice and Delinquency Prevention). Chicago: Author, 1980.

Cloward, R. A., & Ohlin, L. W. Delinquency and opportunity: A theory of del inquent gangs. New York: Free Press, 1960.

Cohen, A. K. Delinquent boys: The culture of the gang. Glencoe, Ill.: Free Press, 1955.

Gottfredson, G. D. A theoryridden approach to program evaluation: A method for stimulating_ researcherimplementer collaboration (Report No. 330). Baltimore: The Johns Hopkins University, CSOS, 1982.

Gottfredson, G. D. (ed.). The School Action Effectiveness Study: First interim report (Report No. 325). Baltimore: The Johns Hopkins University, CSOS, 1982.

Gottfredson, G. D., Gottfredson, D. C., & Cook, M. S. (eds.), The School. Action Effectivness Study: Second interim report. Baltimore: The Johns Hopkins University, CSOS, 1983.

Office for Juvenile Justice and Delinquency Prevention. Program announcement: Delinquency prevention through alternative education. Washington, D.C.: Author, 1980.

St. John, J. Project RETAIN: First interim report. In G. D. Gottfredson (ed.), The School Action Effectiveness Study: First interim report. Bal timore: The Johns Hopkins University, CSOS, 1982.

217- Figure 1: Aspects of Chicago Board of Education Organizational Chart

I IState Legislature

Chicago Finance Authority (Budget Authority) 1 Chicago Board of Education

General Superintendent Superintendent for Finance 1 of Schools j------T------'4"."-I-61 Deputy Superintendent: Deputy Superintendent Deputy Superintendent for for Field Services for Instructional Serviced Maintenance Services -,

Director, Director, Director, Government Pupil Pupil Pupil Funded Personnel.] Service Service Service Programs Center Center 'Center South Central North

Curriculum

District District District Pupil Personnel Superintendent Superinten Superinten- Services and District dent dent Special 19 District District Education 12 3

f3 Principals 3 Principals 3 Principals

3 RETAIN 3 RETAIN Teachers Teachers Table 1

Numbers of Treatment and Non-Equivalent Control Students, 1981-82 School Year, in Each Project School--Project RETAIN

Treatment Status

Non-Equivalent School Treatment Control

la 1240 15 18 1340 14 15 1430 18 25 2300 16 20 4440 15 16 5090 18 17 5750 18 20 5880 16 17 6180 15 15

-219- 92 Table 2

Average Proportion of Missing Pre-Treatment Data from 1980-81 School Records in Each Project School, 1981-82 School Year, Project RETAIN

Treatment Status

School Treatment Control. Overall

1240 .33 .14 .23

1340 .30 .33 .32

1430 .27 .72 .53

2300 .08 .02 .05

4440 .05 .20 .13

5090 .17 .26 .21

5750 .06 .12 .09

5880 .58 .51 .54

6180 .02 .05 .03

Project as .24 a whole Table 3

Average Proportion of Missing Post-Treatment Data from School Records in Each Project School, 1981-82 School Year, Project RETAIN

Treatment Status

School Treatment Control Overall

1240 .32 .67 .51

1340 .65 .63 .64

1430 .60 .82 .73

2300 .16 .22 .20

4440 .11 ,, .36 .24

5090 .24 .10 .18

5750 .33 .39 .36

5880 .88 .88 .88

6180 .10 -.19 .14

Project as .44 a whole

-221: 94 Table 4 Average Proportion of Missing Student Questionnaire Outcome Data for Each Project School, 1981-82 School Year, Project RETAIN

Treatment Status

School Treatment Control Overall

1240 .63 .79 .72

1340 .93 1.00 .97

1430 .35 .77 .59

2300 .14 ,, .24 .20

4440 .21 .48 .35

5090 .19 .16 .18

5750 .44 .38 .40

5880 .62 .46 .53

6180 .14 .36 .25

Project as .46 a whole

222 Table 5

Comparison of Background Characteristics of Project RETAIN Treatment and Control Students (Entire Sample) During 1981-82 School Year

Group

Non-equivalent Treatment Controls

Variable M SD N M SD N

Sex (coded 1-2) 1.36 .48 107 1.37 .49 105

4th Quarter English .58 .73 117 .59 .68 130 Grade--1980-81

1980-81 ITBS Reading Scorea 6.33 6.90 96 5.41 5.75 103

1980-81 ITBS Math Scorea 6.46 6.56 95 5.36 6.68 102

Credits Egrned Through 3.68 3.27 28 2.54 2.82 48 June 1981

Total Days Absent During 21.83 18.20 101 23.85 22.97 110 1980-81 School Year

Total Days Lost Due to .18 .52 102 .27 .74 97 Suspension: 1980-81

Age (from 1982 13.44 1.66 95 13.14 1.70 86 Student Survey)

Grade Level (from 1982 1.78 1.45 95 1.54 1.33 83 Student Survey)

Parental Education (from 2.08 1.20 72 2.16 1.15 58 1982 Student Survey) aVariable is available only for students who were in elementary school during the 1980-81 school year. bVariable is available only for students who were in high school during the 1980-81 school year.

-223- 96 Table 6

Background Differences of Project RETAIN Treatment and Non-Equivalent Control Students During 1981-82 School Year, by School

Group

Non-equivalent Treatment Controls

Variable SD N M SD N Prob.

School 1240

Total Days Lost Dud to .25 .46 8 .93 1.07 14 .05 Suspension: 1980-81

School 1340

Total Days Absent During 39.15 19.18 12 55.0420.64 13 .06 1980-81 School Year

School 1430 _

1980-81 ITBS Math Score 12.08 13.23 7 .63 1.51 12 .06

Credits Earned Through 5.39 3.86 11 1.58 3.05 18 .01 June 1981

School 5090

Age (from 1982 13.31 1.14 16 12.33 1.84 15 .09 Student Survey)

Grade Level (from 1982 1.53 .64 15 .78 .80 14 .01 Student Survey)

School 5750

Total Days Absent. During 16.28 18.73 16 6.4210.62 18 .08 1980-81 School Year

School 5880

Age (from 1982 12.28 1.11 7 13.30 1.06 10 .08 Student Survey)

Note. These comparisons suffer from problems of missing data.

-224- Table 7

Comparison of 1981-82 School Jeer Outcome Variables from School Records for RETAIN and Control Students

Group

Non-equivalent Treatment Controls

Outcome Variable M SD N M SD N

Ratio of Days . 10 .14 101 .10 .13 80 Absent to Days Enrolled: 1981-82

Ratio of Days .03 .04 68 .03 .04 65 Excused Absence to Days Enrolled: 1981.-82a

Ratio of Days .03 .05 61 .02 .03 62 Unexcused Absence to Days Enrolled: 1981-82

Ratio of Days Lost . 00 .01 78 .01 .02 73 Due to Administrative Removal to Days Enrolled: 1981-82

Ratio of Times Suspended .00 .01 83 .00 .01 59 to Days Enrolled:1981-82

Average English Grad .89 .66 95 .86 .69 84 (1-4 scale): 1981-82°

Average Math Grade .88 .67 87 .94 .60 77 (1-4 scale): 1981-82°

Average Science Grads .96 .70 64 .91 .60 64 (1-4 scale): 1981-82

Average Social Studies .93 .64 70 .89 .57 67 Grade g-4 scale): 1981-82°

Dropout . 15** .36 145 .35 .48 163

Note. The statistically significant outcome for dropout favoring the treatment group dissappears when appropriate statistical controls are applied.

aVariable is applicable only for project elementary schools. b Average of the four quarterly grades in the particular course.

** I) < 01

-225- 98 Table 8

Comparisons of 1981-82 School Year Outcome Variables from Student Survey for RETAIN Treatment and Control Students (Entire Sample)

Group

Non-equivalent Treatment Controls

Variable M SD N M SD N

Parental Emphasis on Education .53* .31 87 .62 .26 78 Attachment to Parents .54 .26 92 .58 .28 85 Negative Peer Influence .31 .23 88 .31 .21 86 Attachment to School .57 .27 87 .54 .30 81 Belief Scale Scored Positively .57 .24 83 .55 .28 74 Interpersonal Competency .74 .23 84 .77 .23 74 Positive Self-Concept .66 .17 75 .65 .19 59 Self-Reported Delinquency .18 .23 81 .19 .25 70 Scale Self-Reported Drug Use .28 .36 83 .21 .33 72 Serious Delinquency .13 .21 81 .15 .25 71 Punishment Index .25 .29 86 .25 .25 77 Rewards Index .28 .31 85 .27 .27 78 Victimization .19 .25 86 .17 .21 78 School Effort .49 .33 95 .47 .31 83 Practical Knowledge 1.46* .35 83 1.33 .45 75 Internal-External Control .55 .22 83 .53 .23 75 Alienation .42 .26 81 .47 .22 75 Self-Reported Grades 1.91 1.08 93 1.93 1.04 84 Reading Ability, ..,elf Rating 1.11 .90 89 1.14 .81 77 Days of School Cut in 4 Weeks Preceding Survey .56 1.09 95 .38 .72 85 How Often Cut One or More Classes .98 1.41 95 .74 1.23 86 Educational Expectation 2.89 1.85 95 2.93 1.90 85 Did You Work for Pay Last Week? .65** .48 95 .36 .48 86 Regular Part-Time or Full-Time Job? .37 .58 95 .28 .57 86 Suspended from School During Term .43 .50 86 .33 .47 78 School Nonattendance Index .72 .84 94 .62 .77 85 Rebellious Autonomy .68 .33 83 .63 .30 73 Involvement .27 .21 87 .23 .20 81

Note. Two of the statistically,significant differences in these raw comparisons (i.e. Parental Emphasis on Education, Practical Knowledge) are not significant when appropriate statistical controls are applied.

* p<.05

** P< 01

-226- The Milwood " 'native Project: Second Interim Report

M.S. Cook

Abstract

The Milwood Project is one of several projects funded by the Office of Juvenil, Justice and Delinquency Prevention (OJJDP) as part its Program in Delinquency Prevention through Althernative Education. The Milwood Project operatesin Milwood Junior High Schoolin Kalamazoo, Michigan. Project components during 1981-82 included an in-schoolsuspension room,a home- school liaison,an attendance clerk, a Sk lls Lab class for low-achieving students, a school-within-a-school for eighth grade students (the Milwood Alter- native Program), a studentcouncil, a projectadvisory, andan intramural/activities program.

School climate comparisons with South Junior High School(the control junior high) indicate that between the 1981 and 1982 school years school climate improved slightly at Milwood, and declined substantially at South. Milwood is maintaining a positive school climate in the face of financial cutbacks in Kalamazoo programs. Basic skills scores improved significantlyat Milwood, while remaining stable at South. We cannot conclusively attribute these positive effects to the Milwood Alternative Project.

Chronic non-attendance--skipping several consecutive days of school--is also down at Milwood as compared to South. The in-school suspension room reduced the number of exit suspensions, but does not appear to have affected the overall number of suspensions. We are unable as yet, to judgethe effectiveness ofthe other project interventions. For 1982-83 an Intensive Study Program has replaced the Skills Lab,and a reading diagnosis and tutoring program has been added.

Major Program Changes in 1981-82 grade students,(b) the Skills Lab, a basic skills class, (c) a student The Milwood Alternative Project council, (d) a student project advi- continued implementing the major sory--a student board with input programinterventions described in into project activities, (e) a pro- thefirst interim report (Carlton ject advisory utilizing community and Cook, 1982). Briefly, these members, (f) an attendance monitor interventions included (a) the Mil- and home-school liaison, and (g) an wood Alternative Program,a schoolin-school suspension center. The within-a-school for sixty eighth only modification of note was in the teachingstafffor the Skills Lab class. Because ofteacher layoffs and teacher union issues related to seniority, the original teacher for Thisreport coversroughly the the Skills Lab (around whom the period of September 1981 to June class was designed),was replaced 1982.

227

i00 Milwood Project first with substitute teachers, then force set up to evaluate the with a school counselor appointed by Kalamazoo Public Schools' discipline central administration who had never policies)has praisedthe Milwood taught in a classroom. Finally, in project's in-school disciplinary January the position was filled by a policies,and recommended thatthe teacherselectedby Milwoodstaff policy be implemented system-wide. from a pool available for seniority- Favorable reviews of the project based layoff recall. This person appeared in the local newspaper, on did not have specific expertise in the T.V. news, and representives of workingwith low-skills children, the project were invited to discuss and was not involved in program or their work on a local television project planning. talk show. The Milwood project has also received favorable press Major Forcefield Changes in 1981-82 through a UPI news release.

Four major forcefield changes A negative force actingon the occurred between the 80-81 and 81 -82 project was the financial situation school year: a change in the Mil- of the state of Michigan andthe wood assistant principal; thecom- city of Kalamazoo. Although the mitment of the Milwood principal and recession has not had as much of an new assistant principalto project impact on Kalamazoo as on the rest success;the development ofstrong of the state, budget cuts did result support for the project within the in widespread teacher reductions community; and budget cuts forcing from 1981to 1982. Besides gener- teacher layoffs. ally reducing personnel and resources froni the school (probably Partly due toproject efforts, negatively affecting school climate) the 80-81 assistant principal, who the cuts had a particularly negative was responsible for recommending the effect on the project's Skills Lab high number of out-of-school suspen- class. sions experienced at Milwood during the 80-81 school year, was replaced The teacher who had been instru- byan assistantprincipalwho is mental in designing the class, and more accepting of alternative forms who was to teach it was bumped down of discipline (i.e., referral to to the elementary school level due in-houseratherthan exit suspen- to her lack of seniority. The posi- sion). Also, the building principal tion was covered by several diffe- has become an advocate for the pro- rent teachers throughout the year, ject, both within the school and in and was neverfilled by a person thecommunity at large. Although experienced or trained in teaching a not involved in the early stages of basic skills class. project development, Milwood's principal has become actively involved In short,some major forcefield in project activities, and now changes impinging upon the project serves as an enthusiasticproject werepositive: Extensive support spokesperson before parentgroups, for the project has developed in the central administration staff, and school itself, in the school system, the media. andin thecommunity. Oneforce- field change was negative: teacher Community support for the project layoffs basedon seniorityrather increased sharply during the 81-82 than training or ability. year. For example, The Citizens' Coalition on Discipline (a task

228

. TW1=. .MaIa Milwood Project

Evaluation Results of the differential racial composition of the two schools, an analysis School Climate of covariance was performed. Becauseparental educational level Becausethe Milwood Alternative also tended to differ between Mil- Education Project is primarily tar- wood and South (see Table 1) paren- geted at school-wide climate change, tal education was also introduced as school level comparisons are of a covariate. Table 3 presents the major interest. comparisons that were significant after controlling for race and Milwood and South in 1980-81. parental education. The 1980-81 school year was predominantly a planning year for the Mil- Striking differences remain bet- wood Alternative Project (see the ween the two schools after correct- first interim report). Most project ing fpr the two demographic varia- components were not implemented bles,race and parental education. until the 1982 year. Thus, the 1981 The adjusted means show that Milwood questionnaire results maybe con- students, as compared to South stu- strued as pre-treatment data. The dents, reportedhigher rebellious findings in Table 1 support the autonomy in 1981. More importantly, claims of project staff that Milwood perhaps, Milwood students were sus- suffered from a poorer schoolcli- pended much more often than South mate than South at the initiation of students, reported more delinquency project activities. There were six anddrug use, and skippedschool significant differences on the 1981 more often. student psycho-social scales. Mil- wood students, as compared to South The foregoing analyses indicate students, had lower self-esteem and that the climate at Milwood was less higher rebellious autonomy. Milwood favorable than that at Scuth at the students were alsosuspended more close of the 1981 school year. Mil- often, attended school less, and wood students were misbehaving more, reported more delinquency and drug getting into more trouble at school, use. and cutting school more often. The question becomes what happened as It should be noted that Milwood the project began implementation in had a lower percentage of minority earnest? students than did South in 1981 (31% vs 45%, p = .005), and that race was Milwood and South in 1981-82. correlated with several of the ques- Table 4 documents the results from tionnaire scales. Those that were analyses of variance carried out on significant at the .01 level or bet- the 1981-82 SAES questionnaire sca- ter may be found in Table 2. Being les. Major differences can be noted white is associated withgreater between these results, and those belief in the rules, but less prac- found in Table 1. Table 4 indicates tical knowledge,and less involve- that by the end of 1982 Milwood stu- ment in conventional activities. dents, as compared to South stu- Minorities are suspended more often, dents,reported greater attachment receive lower grades, and their to school, found school less punish- parents have completed fewer years ing, were less alienated,and were of education. Whites report more victimized less. On the other hand, drug usage. Milwood students said they were now lessinvolved inschoolactivities In order to controlthe effects

229

102 Milwood Project than South students,and said that sis of the Metropolitan Achievement they were poorer readers. Milwood Test of basic skills (MAT) indicates students also reported greater prac- that Milwoodstudents wereindeed tical knowledge. ;,s contrasted with poorer readers than South students 1981, there were no differences on in 1981, but this was not the case the SAES questionnaire drug use, in 1982 (see details, below). attendance, orsuspension measures, and the trend was for Milwood stu- The finding oflessinvolvement dents to report less delinquency on the part of Milwood students overall, and less serious delin- implies that theproject has been quency. Student characteristic mea- unsuccessful to date in signifi- sures show that Milwood students cantly increasing Milwood students' rated themselves as having less involvement in school extra-curricu- reading abilitythan students from lar activities. Both project staff South. and the school administration attribute great importance to extra-cur- An analysisof covariance cont- ricular activities in the promotion rolling for race and sex produced of school attachment, thus the find- essentiallythe same analyses, and ing of low involvement in extra-cur- will not be reported here. ricular activities is an important outcome for further program develop- These results indicate thatthe ment efforts. Likewise, project climate at Milwood had improved staff theorize that basic skills relative to South between the spring proficiency is important for life of 1981 and the spring of 1982. Not success(and reducingthe risk of only did Milwood students like delinquency),- so the -decieise in school more than the South students, reported reading ability scores andreport themselves to be less found at Milwood may also suggest alienated, butthey no longer dif- programmatic change. feredfrom, the Southstudents in their self-reported attendance or Climate over time. The next set drug use. Importantly, in 1982, of analyses was conducted to examine Milwood students reported themselves changes in the SAES scores from 1981 to have been victimized less than to 1982. The first set of analyses South students and scored lower on a determined that in 1981, Milwood had measure of alienation. Thereare a. less favorable school climate than also trends toward Milwood students South. This set of analyses are an reporting less delinquencyoverall attempt to ascertainwhether the and less serious delinquency. generally more favorable climate at Milwood in 1982 as compared to South But the positive changes at Mil- was due to Milwood developing a more wood do not tell allof the story. positive school climate, South a Student involvement at Milwood more negative,or some combination remains low (as compared to South), of the two. andin 1982, largedifferencesin self-reported reading ability exist, In the first attempt to investi- which appear to be the result of the gate changes in school climate from joint effects of improvement at the 1980-81 tothe 1981-82school South, and a slide in self-reported year, change scores were calculated reading ability at Milwood (see for thosestudentsfor whom there Table 1). The 1982 differerce in were completedquestionnaires for reported reading abilityshould be both years. These, of course, would interpreted cautiously since analy- be for the most part students that

230

103 Milwood Project

were seventh graders in 1980-81 and the Milwood students included in the eighth graders in 1981-82. change score sample had only experienced one year in which project Table 5 gives theresultsfor interventions were fully opera- this series of analyses of variance tional. Project staff hypothesized on the available SAES scales (Inter- that the eighth grade class of nal Control was not scored for 1981-82was particularly resistant 1980-81). The only change score to influence, largely because of that differed significantly for Mil- their early (seventh grade) negative wood and South was in self-reported experiences in the schoolenviron- reading ability: Milwood showed a ment; as a school Milwood was less decrease of .13 in self-reported harmonious in 1980-81 than in reading ability, while South showed 1981-82. Students exposedto pro- an increase of .34, i.e., Su ,h stu- ject activities in 1981-82 may have dents reported themselves to be already "opted out" of school. somewhat better readers, while Mil- wood students reported themselves to Summary of school climate beslightly poorer readers. This results. The pattern of results result is consistent withearlier from the two sets of data--the bet- analyses. ween school analyses, and the changes from 1981to 1982--suggest There is also a trend toward Mil- three conclusions concerning school wood students becoming more alien- climate at Milwood and South: ated over the 1981-82 year (p=.07). No other comparisons approached sig- 1. Milwood exhibited a slight nificance. improvement in school climate from 1981 to 1982. The school climate at This analysis is difficult to South showed negative changes during interpret due. to the relatively this saws period. small numbers of individuals on which we have data for both years; Comparing Tables 1 and 4it is N's range from 52-84 for the change apparent that from 1981 to 1982 Mil- scores ascompared to191-676 for wood and South reversed their ove- the simple between-groups, 1982 com- rall standing on school climate. parisons. The change scores do not WhereasSouth hada better school includeany student who hadspent climate in 1981 than Milwood, the only one year at either school, and reverse istrue in 1982. The mea- largely exclude seventh graders (and sures of attachment to school, therefore the samples differ in age alienation, rebellious autonomy, also). practical knowledge, self-reported school non-attendance, suspension, delinquency, drug use, and victimi- The effect of the more select zation all support this interpreta- (and older) sample size can be seen tion. inthe alienation measure. While the change scores indicate a big 2. Measures ofInvolvement in jumpat Milwood anda very small conventional activities and self-re- decrease at South, means for the Ported reading ability favor South entire yearly sample at each school over Milwood. These results are (Tables 1 and 4) show that, overall, puzzling. Involvement in school-re- alienation was up somewhat at both lated activities was a major Milwood schools. project focus in 1982. Students at Milwood apparently did not feel its A further consideration isthat

231

104 Milwood Project impact, or not enough students were A third explanation for the involved in project activities late effects might lie in regression in the school year (at the time of artifacts. Milwood was chosen for the SAESquestionnaire administra projectactivitiesbecause it was tion)to differentially affectthe judged to havethepoorest, school Involvement measure. The selfre climate of the three juniorhigh ported readingability measure is schools in Kalamazoo. If Milwood highly suspect in these data was below the mean in school climate because, although Milwood's selfre at the start of the project it would ported reading scores declined bet tend to move toward the average cli ween 1981 and 1982, and South's mate ofthe area schools,and the improved, test scores show precisely results over time would be expected the opposite pattern. to look much as they do. At this juncture, we can only say that Mil 3. It is impossible to unequivo wood is showing promising improve cally attribute the pattern of ments in school climate, but we must school climate results (relative wait for the third year results for improvement at Milwood, and decline more definitive assessment of ove at South) to Milwood project activi rall school climate change. ties. Metropolitan Achievement Test Scores It is reasonable to arguethat the less favorable climate at South A further way of examining organ in 1982 ascompared to1981 is a' izational changes is by analyzing result of restricted resources and the performance ofthe students of cuts in programs resulting from the the two schools on the MetroPOlitan recession. These cuts translated Achievement Test (MAT), a standard mainly into staffing cuts, and cuts ized basic skills test. The MAT in extracurricular programs. The provides scores on reading, use of slight positive climate changes wit language, basic mathematics, and an nessed at Milwood in the past year overall composite (total) score. are impressive because they have come during a time of generally Table 6 presents the spring 1981 declining resources for education in raw and percentile scores forthe the area. In otherwords, this three subtests and the composite argumentassumesthat without the score for Milwood and South. The alternative project efforts, the mean scores presented are adjusted climate at Milwood would have deter for grade,sex, race, and age. On iorated as itdidat South. all three subtests and the composite scores, the South students performed An alternative explanationwas significantly better than the Mil offered forthe decline inschool wood students. The differences are climate at South by its principal. not only statistically significant, She feels that the negative climate but are of a fairly substantial sub effects in 1982 were due to a parti stantive size also. The composite cularly "bad" eighth grade class; percentiles indicate that Milwood that particular cohort at South students (after controlling for caused the problems. This would not background differences) scored at a explain why the effects did not show level that was eight percentile up until these students' eighth points below that of South students. grade year,since they had been at South during the previous school The following table, Table 7, year. gives the results for the analyses of the spring, 1982MAT scores.

232

105 Milwood Project

These results are dramatically poorer performance of Milwood versus differentfrom those of Table 6. Southin 1981 was entirely due to Milwood students no longer differed differences in performance ofstu- from South students in Reading, Lan- dents in the twoseventh grades. guage, or on the overall composite The two groups of 1981 eighth grad- score, and actually showed signifi- ers scored very similarly on the cantly better performance on the tests. mathematicstest. Tables 6 and 7 indicate that the 1982 effects are Table 10 gives the same results the result of virtually no change in for the 1982 scores,and Table 11 performanceat South between 1981 lists the average percentile change and 1982, but of large increases at from 1981 to 1982 for the school by Milwood--eleven percentile points on grade groups. As Table 11 docu- the composite score. ments, both the 1982 Milwood seventh and eighth grade cohorts scored In order to examine the hypothe- higher than their 1981 counterparts, sis that changes between the years with the seventh grade cohort show- aredue to cohort differences, an ing the biggest increase (and analysis of variance was conducted accounting for the disappearance of on both the 1981 and 1982 MAT scores the interaction effect found in the thatincludedschool, grade, sex, 1981 data). The 1982 eighth grade and race as factors, and controlled students(who are essentially the forthe age of the student. This samestudents asthe 1981seventh analysis indicated that, in general, graders, and who do not constitute a whites perform betterthan blacks different cohort) gain eight percen- and other minorities, and girls per- tile points on the composite score. form slightly better than boys. The overall change from 1981 to 1982 These effects were similar for both on the composite index was +12 per- schools, and for both years. Table centile points for Milwood students, 8 gives the adjusted means for the and +1 point for South students. As race and sex main effeCts forthe seen in Table 11, the interaction of overall composite score. The dif- school by year (controlling for sex, ferencebetween theoverallgrade race, and age) on the composite per- levels within aschoolare largely centile score is significant uninterpretable because both the (p<.001). seventh and eighth graders take the seventh grade version ofthe MATs, To summarize the MAT data, Mil- and the eighth graderswould be wood students (particularly seventh expected to score more highly on the graders)performed much better,on tests. the 1982 than the 1981 MATs. South students showed little change bet- More importantly,the only con- ween the two years. As was the case sistent interaction effect was bet- with the school climate findings ween schooland gradein 1981. A derived from the SAES questionnaire, significant interaction effect indi- we cannot unambiguously attribute cates that the results for one vari- thepositive gains at Milwood to able (inthiscase, thestudent's project efforts. This is especially grade in school) depend upon another true given that there wasnot a variable (here, the school in which great deal of attenti.on given to the student was enrolled). Table 9 basic skills education bythe pro- showsthe means forthis interac- jectduringthe 1981-82 year (see tion. Thesemeans, adjusted for the analysis of the Skills Lab, sex, race,and age,show thatthe below). Changes in administration

233

7,-. Milwood Project

(or other, unknown changes) may use, and to more serious delin- account for Milwood's improvement in quency, and they are suspended more. test scores. The third year results There is also a tendency for them to will bemoredefinitive, but, at report greater overall delinquency, thispoint, the basic skillstest and to have lower grades. Finally, outcomes are promising. the NAP students, as compared to the eighth grade at large, have much Specific Project Components lower educational expectations.

The nextsection of the report The difficulty with this analysis willdiscuss data pertinent toan is, of course, the probable gross evaluation of the individual treat- pretreatment differences between the ment components of the Milwood MAP students and . the "control" Alternative Education Project. group: the rest of -the Milwood eighth, grade class. The Milwood Alternative Program. No a priori control group Was Table 12 suggests that pretreat- defined for the MAP. Because of ment differences did exist: The MAP thisthe rest ofthe eighth grade students are significantly older class was used as the nonequivalent than the restof the eighth grade control group for the MAP. This is (probably due to being held back in a conservative comparison. The MAP school), and they indicate chat participants were invited to parti- their parents have much less formal cipate, largely on the basis of hav- educationthanthe parentsof the ingexhibited behavioral andaca- typical Milwood eighth grader. The demic problems, so one would expect MAP studentb Are 'also- SbmeWhat more them to have poorer outcomes. likely to be minority students than are the rest of the eighth grade. The results for comparisons between Milwood Alternative Program A second analysis was undertaken (MAP) students and the rest of the to attempt to control for those pre- Milwood eighth grade (all MAP parti- treatment, differences identified cipants are eighth graders) are above: age,'parental education, and shown in Table 12. Table 12 gives minority status. Table 13 shows the the results for analysis of variance only significant differences that comparisons between MAP participants remain after controlling for the and the rest of the eighth grade on backgroud variables. MAP students the 1982 questionnaire scales- The reportreceivingmore rewards at results are mixed. Three variables school than Milwood eighth graders, central to the efforts ofthe MAP butthey are suspended more often: intervention, School Rewards, Inter- Thus, most ofthe differences that nal Control, and Interpersonal Com- appeared in Table 12 can be attri- petency show significant differences buted to thepre-existingdiffer- that favorthe MAP students. The ences in the groups. Involvement in Conventional Activi- ties scale also shows positive In asense,the lack of differ- effects for the MAP. On the other ences between MAP students and hand, MAP students report lower eighth graders can be construed as a self-esteem and attachment to project success. The MAP students parenti than the rest of the eighth were selected for the program on the grade, and they report less parental basis of having behavioral and aca- emphasison education andgreater demic problems. We would expect negative peerinfluence. TheMAP themto differ negatively from the students alsoadmit to more drug

234 *ttwsod ?r*j.cs

0#*3 p4*4 . 9* ibe bIjb.r rb. 91M0*i4 4f 4 * 4 9*3 3*,. 4 no asessI t&sLf trait resultof 3*o *t 1*4. tLamie sos,. saslysts1.. ibM 4 øft44,* 3 - WOiv 44* $* t*dssts mousI a *$44I3p., 3* W 9** kI* 9*1, Iseresi. Li *$0tte 3sit 1sf b*sc. ük*0 III lbs Otiwsod .Ls*4rb grad. aloes Is psrrM. If this La a pro- 4o 1w3OwL4w0,s3. 4 so ,rnm lusts, Li I. a sopit,. so $I+t ft.. Ws 4ip144 w.(r*oq lust'. *041 *9* 4* 1W $* *P 4.so**o W lb. ibe.. sot. of data reported *a4s4r*4 .4*. ra' 4* ?aM. 13, 13, aid' 14 coso.rg. WM40 0051*0444M'4$ i, ssi*.' loused Lb. (etloutag cs.c*wstos Vs $ØC404 4* soø4t* .41* b. do set ssrrsssly 4*,. s desip ado- 0 o* 4 4W 4$) $P 4ss1e4*os.clsoLv.ly oralv.te Lb. 4* 1*. *00*43*04 I** ,ffsttiwoss.s.41b $P hi aasrtrg 41**P *144*0 -9* *4th *P.* tie pils. V4*ti,ir Li..ft.c*s. *t*** $1*t ***14&W *#*N 9* *P*$ 19*r p.*titw.orseptic.,ibly *44*: 4W *P $ 1 *et dremoUc .aoagb to be pithed 341*. s** is, S *p .u*eIetsitIy wee. cart.,., types *0*0 *l1i0 $* 94W0. 1I* N .4 rnsrly.e.. lbs "global" iupree'- 3*444,4*4,4 9* *Iai Oil $041 4. that lbs pragr so*i * $st 1*4* **# o' r*, sey Is eeesfsl1y cr.aUag a 44 *. ** is,9* sebist esetesmisi Is which atudest. 4* IL 14 that i.sre blYiPig difficulty is 4*. W i41114*.4t is sibisi eeport aee.t,t.g sore raward.e i4M. *.**M*4 *1*44 *11* sri Is 4* ssbe.*.Asthe us. it... those 3 *54*414*4 *44l, JW4t1W' aIuds.ti ottil set 4. trouble sore 1*0*4 t4s K 41*. thu lbs "scans." thUd, aid 5*4*0* 9*4 44**$4 t49* y auffsr us, negative fl* 41441*04*. hush Ii deeIop1og a network of t,Issdobtpe cub other behavior or 4* 9*oe f (4*1IN stUdlits (as sea- *_4 444 I*$44 *I$w* *4oth 1$L er.4 by the Sagest,. Peer Tiflneacs - **)4.to, 4W seal .). 49* 4 lb. Th1iMitt4. The srt progr ** 1s4**4 *dy. Is so be sutned is the eo*4r, lb. ffi. iI 3b4 SkIlls Lab, a basic skills cuss 0** *P4 1W0* rern. As 1. ii.. ,va1atIos of the 4*4*4* ope**s14s 9*3 1*41 $P dasses, there La no equlvalait .o4* $**4i 4* i*4' 4 ossissi groop to serveasa basis of *4 4* 4ititssU t. 40 ...puIass. ThiSkillsLab (EL) 41$44 4*444440 p45144bad etsdesls wore tvdeil. selected Op. 009* It Isisoso they bad relatively sever. 4* lbs *su. Is dtfVls.lti.eIsroediag aid sub, *4*10* lb. t*ot* sri* Nil LU (they bad either failed two or more sol *m * 14*4*$ s*4 lb. slaoa.e she previous year, or ware *4*. 4*, t4* *epittms4*s tattIng dvrisg the current year), or 4+ 4$*M**$ s r*44o*or .41*1 lad Osw. attuduc.o or behavior øm34 *4*4sc*flsssl psOhsa. those t,.dsnU who were

108 u.f1, , s-.w*-r,-.. 't ?Inwood Project

eighth graders had academic skills- This analysis simply verifies that were considered to be attoo what one would expect about a group low a level for them to adequately of students that were selected for function inthe MAP program--where having large deficits in basic theinstructionparallelsthat of skills,and says little about the the regular school curriculum--or success of the intervention in meet- were thought to be sodisruptive ing the needs of those students. that they would be a negative influ- The differences between this group ence on the other MAP students. and the Milwood population as a Seventh gradersin the program had whole were considered too gross to academic problems or disciplinary conduct an analysis controlling for difficulties, anddid not qualify background variables. for special education or Title I programming. Thus, the Skills Lab Table 15 also gives' the results students were a select group: they for signficance tests for differ- had weak academic skills, and were ences betweenthe SL and the MAP usually presenting disciplinary students. Again, the differences problems. between the groups are extreme, and mirror the results from the compari- Table 15 presents the results of sons between the SL and the Milwood ANOVAs comparing the Skills Lab stu- students. (See Table 12 to compare dents first to the rest of the Mil- MAP means with SL means.) Although wood population (Skills Lab students the outcome differences were again could bein either the seventh or large (and presumably, pretreatment eighth grade), andsecond, to the differences were large as well), a MAP students. series of analyiesi of covariance were conducted on the scores of the Looking atthe differences bet- SL and MAP students, controlling for ween the SL students and the general parental education, reading ability, Milwood student population,asone and age. Table16 gives adjusted might expect given that the Skills means and probability levels for Lab students are selected because of those measures that remained signi- having very poor basic skills, they ficant. differ from the other Milwood students on a host of measures. The The psychosocial scales show that Skills Lab students are less at the end of the 1982 school year, attached to school, find school more the SL students were less attached punishing, expend less effort in to school, expended less effort at school, and report greater negative school,and were lessinvolved in peer influence. They are also less school activities than were the MAP attached to their parents and have students (even while controlling for lower self-esteem. Behaviorally, the background variables). They theSkillsLabstudents say that also found schoolmorepunishing. they are suspended more often,are Not surprisingly then, they expected more Jelinquent, use drugs more to go less far in school. The SL often, and receive lower grades. participants also reported more They are also older,expectto go negative peer influence. leas far in school,and(not surprisingly), report being poorer The behavioralself-report mea- readers. Finally, theSkills Lab sures indicate that the SL students students report that their parents engagedin more delinquency, drug are less educated than the typical use, andseriousdelinquency, and Milwood parent. were suspended more in school. They

236

109 111.....0.11 ... Mil wood Project reported receiving lower grades than instructor's salaryoutof project did the MAP students. funds, a search fora new teacher was undertaken. The choices offered Skills Lab students finished the to the project were still limited to 1982 school yearwithmuch less those aaliable for recall based on favorable outcomes than the Milwood seniority requirements. It was not Alternative Program students. until January that project staff Because we do not have the data to identified a teacher that they hoped control adequately for pretreatment would be suitable. The final differences, the outcome differences teacher was a former assistant high between the SL and MAP students are school football coach, who had been not very informative concerning the bumped "down" to the middle school Skills Lab's success. There are level. Hehad no. experience in large outcome differences, but there teaching low-skills students, and were probably large input differ- was not trained in special education ences between the two groups of stu- (the teacher around whom the program dents. had originally been designed held a masters degree in special educa- Itis apparent though,that the tion). Project staff understood, Skills Lab did not have large posi- though, that he could be a patient tive effects on its students; the SL teacher. Staff subsequently found participants finished the year too that he was unexcited about teaching poorly. In addition, project staff the basic 'skills class. He has weredisappointed inthe program. since returned to football. In Partly in response to the required short,the Skills Lab had multiple OJJDP budget cuts, project staff teachersthroughout theyear, and decided to discontinue the interven- never atany time, had a teacher tion for the 1982-83 year. experienced in or enthusiastic about teaching low-skills children. To what do we attribute the undistinguished performanceof the Home-school liaison. The home- Skills Lab? Interviews with project school liaison along with the atten- staff indicate one fundamental prob- dance monitor (a secretarial-type lem: the teacher around whom the position) are responsible for intervention was designed,and who closely monitoring daily attendance, contributed much toward the program andinterveningby .letter, phone, in its planning stages in 1981, did and homevisits--in a structured, notultimately teach in the lab. sequentialprocess--toinsure that Schoolsystem budget cuts resulted students are present at school. The inteacher layoffs and caused the attendancemonitor collects daily originalteacherto be transferred absence reports, and prepares a to the elementary level. Initially, daily list of students absent, which her position was covered by substi- includes the number of consecutive tute teachers, and later by a days that the student has been teacher selected by central adminis- absent. The home-school liaison tration,withoutconsultation with then uses that information in making the project. This person was a home contacts. (The home-school counselor and had nevertaught in liaison is also responsible for the classroom, much less taught escorting students home that are basic skills to academically handi- given exit suspensions andwhose capped students. After project parents cannot pick them up.) staff complained about the appointment, and refused to pay the

237 no Milwood Project

The SAES self-reported attendance have been absent for several conse- data shows no change from 1981 to cutive days. 1982 for either Milwood or South (see Tables 1 and 4). Individual The In-school Suspension Center. level attendance data are not yet As is the casewith attendance, available at the writing of this besides the SAES questionnaire mea- report, but the aggregated school sures, only aggregate counts of sus- attendance data (as reported by the pension are currently available. schools to the school district) can The questionnaire data indicated be examined. From 1981 to 1982 the little change in suspensions at Mil- averagedailyabsenceratenearly wood, and an increase at South (see doubled at South from 11% to 21%. Tables 1 and 4). Theaggregated During this same period, the absence exit suspension information shows rate decreased at Milwood from 13% that asone would expect with the to 11%. This pattern of effects installation of an in-school suspen- resemblesthat found fortheSAES sion center, thenumber of exit school climate measures: little (out-of-school) suspensions declined (though positive) change at Milwood ,,dramatically at Milwood from 1981 to accompanying a a large negative 1982. In 1981 there were 510 exit change at South. suspensions levied, while in 1982, there were only105. This repre- The home-school liaison spends a sents an 80% decrease in exit sus- great deal of his time working with pensions. At South, exit suspen- chronic non-attenders, students that sions showed a moderate decrease are very often absent from school, from 195 to 170. usually for several consecutive days at a time. The records of Milwood It is impossible at this time to and South were searched to identify determine how much of the dramatic students who missed 30% or more of decline in the exit suspension rate the days for which they were at Milwood isdue toa change in enrolled to see if there was a assistant principals or due to the changein the number ofsuch stu- development of the in-house suspen- dents. At South during 1981 74 sion room. Milwood's previous children (11% of the average atten- assistant principal handed out many dance period enrollment) fell in the more exit suspensions than the aver- non-attender category. In 1982 age Kalamazoo disciplinarian;Mil- there were 73(11% ofthe average wood's exit suspension.*rate for the attendance period enrollment) 1980-81 year was over twice the chronicnon-attenders. In short, average rate of the other two Kala- there was no change in the number of mazoo junior high schools. The Mil- truant students at South. At Mil- wood project helped lobby for a new wood there was a drop in the number assistant principal, and the new of chronic non-attenders from 83 person does not resort'as quickly to (12% ofthe average enrollment) to out-of7school suspension. 57 (9% of the enrollment). This 25% drop in the proportion of truants at At the same time,the 105 exit Milwood is statistically significant suspensions at Milwood in 1982 were (z = 2.14, p < .05). considerably lessthan the 170 at both South and Hillside junior high Theseresults suggestthat the schools. Consequently, the avail- home-school liaison is successfully ability of the in-school suspension increasing attendance at school, room appears to supress the number particularly by increasing the of out-of-school suspensions. attendance of studentswho would

238

J Milwood Project

The Student Council. Student fetched, and grants astonishing Council representatives for each power to a rather weak intervention. homeroom class were chosen by a lot- Likewise, it is hard to see why stu- tery from the students in that home- dent council membership would room who nominated themselvesfor adversely affectone'seducational student council membership. This aspirations. procedure produced a natural quasi- experiment. Table 17 gives the This pattern of puzzling results results for the ANOVAs comparing the tends to support the hypothesis student council members to the ran- that, despite an apparent randomiza- dom controls (those students not tion process, thestudentcouncil selected in the lottery). Perhaps students werea somewhat different thethree most relevant SAES mea- population than the random controls. sures in terms of the goals of the Although none of the following dif- programare attachment to school, ferences are statistically signifi- school rewards,and involvement in cant, they consistentlyfavor the school activities. Of these, control group over the student coun- attachment to school shows no effect cil members. Thestudentcouncil of the program, but both school members are more likely to be male rewardsand involvement in school than the controls (55%vs. 45%), activities show significant positive i.e., girls weremore likely to effects for the student council volunteer forthe student council, intervention. but boys were morelikely to be selected for it. The student coun- The pictureis clouded when one cilmembers' parentstendto have examines the rest of the results of had less education(p = .20)than analysis; several negative outcomes the parents of the control students, are apparent. The student council and the student council students students report more negative peer also report themselves to be influence than the random controls. slightly poorer readers. In addi- Theyalsohave lowerself-esteem, tion, there is alsoa weak trend and are less attached to their (p=.13) for the student council mem- parents. They say that their bers to be suspended more often than parents put less emphasis on their the controls. Again, itis diffi- education than the control students. cult to imagine whybeing inthe Finally, the studentcouncil stu- student council would cause one to dents have lower educational expec- get suspended. tations than do the controls. One possible explanation for the These results are puzzling, espe- results is that student council mem- cially those concerned with parental bers were chosen in a random lottery variables. There does not appear to by homeroom, i.e., from each pool of be any theoretical reason why stu- self-nomineesfrom eachhomeroom dent council membership would affect class, a representative and an these variables one way or another, alternate were drawn. Because the much less negatively. Conceivably, MAP and Skills Lab classes consti- since the student council ran some tuted homerooms, students with their activities involving parents, the qualities may have been overrepre- students "learned" that their sented on the student council. An parents didnot careabout their analysis of variance was carried out education, and therefore became less which excluded those student council attached to their parents. This members and controls who were explanation seems a little far- enrolled in either the MAP or Skills

239 Milwood Project

Labprograms. Significantresults the unique contributions or effects from this analysis may be found in of the student project advisory. Table 18. Summary of Second YearEvaluation As can be seen, these results are Results verysimilar to those reported in Table 17. The non-MAP and non- 1. Milwood has shown some slight Skills Lab student council members increase in the quality of its report lower parental emphasis on school climate, while the climate at education, and lessattachment to South (the control school) has their parents. These student coun- declined. cil members have lower self-esteem, and suffer from greater negative 2. We cannot as yet conclusively peer influence. They also work less attribute the differences in the hard in school. The student council school climate changes inthe two members still reportmore school schools tothe Milwood Alternative rewards (the effect is less signifi- Education Project, but it appears cant because of the decrease in n, ,probable that the project is favora- but the absolute size of the differ- bly affecting the climate of Milwood ence is the same). Oddly, the stu- Junior High School under conditions dent council members that were not of declining resources available for in the MAP and Skills Lab did not Kalamazoo schools. feel more involved in school activities. To summarize this analysis, 3. Milwood demonstrated signifi- even removing the effects of a pos- cant and substantial _improvement sible overselection of students with MATpeiformance between 1981 and behavior and academic problems,the 1982, as compared to South where student council apparently had nega- performance remained the same across tive effects on the attitudes of the the two years. Both the seventh and students that served on it. eighth grades showed improvement at Milwood, with the seventh grade mak- There are three possibilities to ing somewhat stronger gains in explain these anomalous results: scores. We cannot judge the impact (a)the randominzation process was of the Milwood Alternative Education not truly random, and some unidenti- Project on these basic skills gains. fied selection bias was introduced (this possibility exists because 4. The MAP program is probably each homeroom teacher did his or her successful in creating a school own random selection from that environment thatis more rewarding room's poolof nominees, and we do for academically marginal students. not know if this procedure was car- It is possible that the grouping of ried out properly); (b) the lottery such children together may lead to was random, and by chance, the some negative peer influence. groups were not equivalent; and (c) the lottery was random,the groups 5. There is no evidence that the were initially equivalent,and the Skills Lab was successful in student council experience produced increasing the positive school some negative attitudinal effects in experiences of its students. the children. Whether this was due to a lack in continuity and expertise in the The Student Proiect Advisory. At teaching staff for the intervention, the writing of this report, no due to some problem inherent to the information was available concerning class, or due to pre-existing char-

240 113 Milwood Project acteristics ofthe students, we do (d) greater and more systematicuse not know. of college student and adult volunteers; (e) elimination of the atten- 6. The home-school liaison is dance clerk and counselor's secre- probably successfully improving tarypositions; and (f) increased attendance at Milwood, particularly emphasis on student involvementin forstudents thatarechronically school activities. absent. This intervention has been viewedso favorably by theschool Changes inthe Milwood Alternative system that home-school liaisons Program have been installed (at school system expense) in all Kalamazoo junior The MAP was originally designed high schools. to include some "positive" influence students: students who were having 7. Most of the large decrease in somedifficulty in theiracademic exit suspensions at Milwood can be work, but who were not considered to attributed to the change in assis- be behavior problems. It was hoped tant principals, but comparison with that these students would serve to the other Kalamazoo junior high '''set a positive example for the less school& suggests that the availabil- well-behaved students. In planning ity of the in-school suspension room for the 1982-83 year, project staff hasalso decreased the number of decided that there had been too few exit suspensions. of the positive influence students (only one or two in a class of 8. The student council increased twenty), and that they had not been school rewards for its members, but "strong" leaders in the class. did not helpthem feel any more Therefore, their positive modeling involved in school. There is evi- was felt to have been ineffective. dence that the student council may For the coming year, a larger number have hadnegative effects on the of positive influence students have children's attitudes, perhaps been included in each class,(alt- through a negative peer influence hough the numbers of positive stu- mechanism. This possibility seems dents are still small) and students unlikely, so we await replication of have been selected who are believed these effects in the coming year's to be stronger peer leaders. It is data. hopedthat thethe greater number and influence ofthe positive students in the class willdecrease Changes Planned for 1982-83 some negative peer influence effects that project staff felt took place Changes in the Milwood Alterna- during the 1982-83 year. tive Education Project that are planned for 1982-83 fall intosix In addition to the inclusion of a general categories: (a) changes in greater number of positive influence the student composition, use of students a different selection cri- instructor time in the Milwood terion was used for the majority of Alternative Program and time spent the MAP students. Students were doing academic work; (b) elimination selected who werejudged to been ofthe Skills Lab class and substi- attending school enough to be able tution of an individualized reading to benefit from the treatment. In assessment and tutoring program; (c) 1981-82, several students withdrew addition of an intensive study pro- orweredropped from the program gram for selected failing students; that never attended school regularly

241 114 Milwood Project

from the beginning of the year; they of the instructors' time involves never received the treatment. some extension of the MAP interven- Chronic non-attenders were not tion intotheafternoon, when the selected for 1983. Also, it is the students attend regular classes. In impression of project staff that 1982, the MAP teachers taught regu- they did notselectstudentsthat lar classes in the afternoon, when had as severe discipline or academic the MAP students were taking class problems as some of the students withthe restof the Milwood stu- included last year. In short, the dents. Forthe upcoming year, one selection criterion were moved "up." MAP instructor's job has been res- This selection change was made tructured so that he is available in because project staff felt that many the afternoons to help MAP students of the students included in the pro- and theirteachersin their main- gram in itsinitial year were net stream classes. This "support" for responsive to treatment because they MAP students in the afternoon, when had,in effect, already dropped out they are not in the alternative of school, or were'exhibiting disCi- environment, was deemednecessary plinaryproblemstoo severe tobe for tworeasons. First, theMAP dealt with in a school-within-a- students still had difficulty in school. Staffbelieve that these someof their non-program classPes, "worst" students were adversely and it was felt that they and their affecting the rest ofthe students teachers could benefit from some aid inthe class. Such students were from a program teacher. Second, not invited to participate this because the MAP students take all of year. their morning classes together, there is a Iendeney-forthem to end Two changes have been made in the up in the same classes in the after- MAP instructors' and counselors' noon due to a limitednumber of time. Formerly during the first electives. In effect,some ofthe period of each day (the "group afternoon class teachers end up with time"), one of the three school a "MAP" class when they do not have counselors assisted each of the the training or desire to work with threeMAPhomeroominstructors in this group of students. It was carrying outthe group activities. thought desirable to have a MAP Building staff felt that this was an instructor available to help teach- inefficient use of counselor time, ers cope with this overabundance of

, and it did not provide any counselor "problem" students. "cover" for the rest of the Milwood population for that first period Another important programmatic each day. Consequently, counselors change involves the addition ofan rotate duties during the first group after school study hall for MAP stu- time period, with one of the counse- dents. At weekly intervals, the lors always remaining in the coun- students' progress on their written selingdepartmentoffice. One of assignments is reviewed, and stu- the MAP classes doesnot have a dents who are performing poorly or counselor, then, during group time, have assignments outstanding are but the MAPstafffeels thatthe required to participate in an after instructors can carry out the activ- school supervised study hall. Stu- ities themselves since they now have dents and their parents are informed a year of experience in facilitating oftardy work on Friday, and ifit the group time. is not completed, they are required to attend the study period on the A more fundamental program change following Tuesday, Wednesday, or

242

a . 115 Milwood Project

Thursday. The MAP teachers rotate week. Students willbe tested to in supervising this special study insure that they do have a reading hall. problem and to diagnose the nature of the reading difficulty. An indi- The last important change in the vidualized plan based on intensive MAP program involves the stucturing tutoring will be drawn up for each of the "group time."Group counsel- child. It will be the reading spe- ing typeactivitiesarenowheld cialist's job to insure thatthe only three daysa week instead of child is receiving the reading five. The other two days are training judged to be appropriate. devoted to supervised study halls. Students work on their assignments Importantly, the reading tutoring or study for exams while receiving will focus on the skills and aca- individual attention from their MAP demic areas that the child is exper- instructor. iencing most trouble with, and tutoringmaterials will be taken To summarize the major changes in from the class from which the child the MAP forthe 1983school year, is drawn for tutoring. Because of they are ofthree kinds: (a) the transportation problems, it was program participants are expected to judgedto be impossible to run an be better behaved, and to.have bet- after-school tutoring program. The ter academic skills; (b) program reading specialist therefore decided support will be provided to the stu- to tutor children during classes dents outside of the regular morning they were already failing (had program, and (c) more time is being received an "F" in the previous spent on academic pursuits, and less report card), reasoning that the on group counseling. child could not do any more poorly, and it was therefore unlikely that Changes in Basic Skills Instruction the once-a-week removal for tutoring would hurtthe child's performance TheSkills Labclass hasbeen in that class. The child's assign- discontinued. It was not viewed by ment for the classhe or she is project and building staff as having missing during the tutoring sessions been successful, and it was elimi- constitutesthe materialfor that nated in reponse to the required 30% session. For example, if a child is third year funding budget cut. tutored during his science class, he is tutored in reading using the sci- Because theupgrading of basic ence material his class will be cov- skills is theoretically important to ering that day. project staff, an alternative--less expensive--attempt to address the Addition of an Intensive Stud/ Pro- problem has been designed. A read- gram ing specialist has been hired who will be responsible for testing Eleven students that are experi- children identified by teachers and encingacademicfailure, whohave project staff as having reading been judged byproject and school problems. Academic success and MAT staff to have the requisite ability scores are the primary referral to perform successfullyin school, sources. The specialist is a gradu- and are not enrolled in the MAP, are ate student at Western Michigan involved in an Intensive Study Pro- University who holds a Masters gram. In this program,they meet degree inreading education. She weekly with a project staffer to go will work at the school 2-3 days per over their assignments, plan their

243

110 Milwood Project

next week's academic work, and The final change in Milwood receive tutoring. The staffer (who Alternative Education Project activ- is in the building full time) checks ities forthe 1983 year isone of ontheiracademicprogressalmost emphasis. Project staff have always daily,and works with the students been concerned with getting Milwood andtheir teachers to insure that students involved in school activi- the students keep upin their day- ties and school decision-making. to-day assignments. They were disappointed in the evaluation results that indicatedthat The program centers around a the students did not perceive them- "time management" approach, in which selvestobe any more involved in the timely scheduling and completion the school or to be influencing of assignments is emphasized. It is school policies. Project staff plan hoped that the students will better to spend more time inthe coming know what is required of them in year working with students in plan- each of their classes, will schedule ningandcarrying out activities. a particular time to do the assign- They hope to increase the number of ment, and will complete it on time. ,activities (through planning and the It is believed that the focused use of volunteers) that students can attention of a project staffer will become involved in. motivate these students to complete their school work and improve their Particular interventions that are grades. planned to involve students in school activities are a pep club, a Additional Project Changes choir, an intramural program, and a spring theatrical produition.- These In 1980-81some limited use of programs will be designed and imple- college student volunteers had been mented with the aid ofthe student made in tutoring, and in supervising council and the studentadvisory the in-school suspension center. board, giving these students more Student volunteers were not used in say in the events at school. 1981-82 because of problems in supervising thevolunteersand in insuring that they would be availa- Recommendations bleon aregular basis. For the 1982-83 year student volunteers will 1. MAP teachers must be aware of again be recruited, and a more thepossibility of peer influence systematic effort willbe made to contributing to a negative classroom supervise and coordinate their atmosphere. Particularly, disrup- efforts. Vol;,:nteers will be primar- tive behavior that is reinforced by ily used in helping with extracurri- the other students in the room can cular activities,in tutoring,and destroy the possibility of learning. in collecting and maintaining atten- This isa drawback to any program dance data. that seeks to put a large proportion of behavior problem students Due to the OJJDP budget cuts, the together in one class. Project project was forced to eliminate two staff have recognized the problem, support staff positions: the atten- and have attempted to rectify it by dance clerk and the secretary in the scheduling more "positive" students. counseling center. The duties of . It is probable though, that the num- these persons are being covered by ber of positivestudents (two or other project staff, and by the use three) in a class of twenty, is of volunteers. insufficientto overcome the nega-

244

11? Milwood Project tive peer influence ofthe rest of study (Powers andWittmer, 1951) theclass. The MAP teachers will inve6tigated the del inquency- preven- haveto be aware ofthe potential tion potential of assigning a case- problem, and should be prepared to worker to monitor the behavior and respond quickly to contagious beha- provide support to lower class boys vioral disruption. identified at risk for delinquency. Long-term follow-ups tothat study 2. The affective education cur- (McCord, 1978) indicate that the riculum for MAP class, organized program actually had severe negative around a three-day-a-week group effects onthe life experience of experience,is not focused, and is the participants. McCord attributes often carried out in an ad-hoc fash- the negative effects toa violated ion. It is apparent that the expectancy that "things are going to counselors and the teachers develop get better": children'in the pro- much of the group materialon the gram felt "special" and were led to spot. It is unlikely that the believe that their lives would now objectives of the program can bemet take a dramatic, turn for the better. unless specific interventions are 1.;These children found that life did designed to address.them. not, in fact, get much better. Apparently, many of the participants 3. Plansunderway to document internalized their failures, decid- the actual curriculum taught in the ing that, "Even with all of this MAP, should be examined. The help, I still can't make it." description of the MAP will only be Accordingly, the Milwood volunteers usefulto future program implemen- must beware of setting up unrealis- ters to the extent that it is clear, tic expectations in their clients, and covers all activities and inter- that,if not met, produce negative ventions undertaken. feelings of self-worth in the children. 4. The teacher of the new remedial reading program shouldattend 6. The introduction ofthe in- closely to possible negative effects school suspension room did not of taking students out of their decrease the number of self-reported regular classes. The program does suspensions (which includes both in- plan to organize the reading tutor- andout-of-schoolsuspensions). A ing around the material the students difficulty with in-school suspension are covering in their cldsswork, but is that it is a less severe sanction it is probable that not as much thanout-of-schoolsuspension, and material will be covered in the thus, tendsto be resorted to for tutoring sessions as in the regular less extreme disdiplinary infrac- classes. In addition, the negative tions. Ironically, this "progres- labeling that could result from sive" procedure can result in more being sent to this class could harm suspensions. Fortunately, in the the overall academic performance of case of the Milwood project, the the students. The reading special- institution ofin-school suspension ist should make every effort to pro- did not increase the number of videtutoring outside of the stu- self-reported suspensions and did in dents' regular class schedule,and fact decrease the number of exit allow the students to remain in suspensions--the program's primary their mainstrean classes as much as objective. Nevertheless, project possible. staff should make clear to building personnel the possibility of too 5. The Cambridge-Sommerville

245 Milwood Project frequent suspension, resulting from theoretically unlikely that the the decreased severityof the pun- negative effects associated with the ishment. Every effort should be student councilin 1982 were actu- made to insure that suspensions, ally program outcomes, the possibil- either in- or out-of-school, are ity exists. Project staff must pay leviedonlyforclearly specified careful attention to any processes extraordinary offenses. that may occur in the student council meetings or in the school at 7. A final caution concerns the large that could produce the nega- the StudentCouncil. While it is tive effects found in the 1982 data.

246

119 Milwood Project

References

Carlton, R, and Cook, M. The Milwood Alternative Education Project: Interim Report. In G. D. Gottfredson (ed.), The School Action Effectiveness Study: First Interim Report (Repoit No. 325). Baltimore: The Johns Hopkins Univ- ersity, Center for Social Organization of Schools, 1982.

McCord, J. A thirty-year follow-up of treatment effects. American Psycholo- gist, 1978, 33, 284-289.

Powers, E, and Wittmer, H. An experiment in the prevention of delinquency: The Cambridge-Somerville youth study. New York: Columbia University Press, 1951.

247 120 Table 1

Comparisons of Milwood and Southon Selected 1981 SAES Measures

Milwood South

ri SD N M SD N p

School variables Attachment to school .63 .27 206 .65 .26 196 .67 School rewards .20 .27 217 .23 .27" 205 .37 School punishment .20 .29 217 .25 .26 206 .06* Parental emphasis on education .68 .26 218 .68 .26 217 .92 School effort .57 .31 223 .61 .29 219 .23 Negative peer influence .21 .2P 219 .22 .23 217 .55 Victimization .14 .17 211 .13 .17 202 .52 Involvement .19 .16 214 .22 .16 210 .08*

Psycho-social variables Attachment to parents .59 .28 222 .60 .27 218 .87 Belief in rules .68 .25 205 .72 .24 187 .14 Positive self-concept .69 .17 194 .73 .16 189 .03** Rebellious autonomy .73 .32 193 .63 .30 173 .003** Interpersonal competency .80 .23 204 .79 .21 181 .69 Internal controla ------Alienation .36 .30 202 .37 .29 193 .83 Practical knowledge 1.37 .44 206 1.39 .45 191 .66 Educational expectation 3.48 1.72 222 3.61 1.73 218 .44

Self-reported behavior variables School non-attendance index .54 .74 224 .37 .68 218 .01** Delinquency .19 .20 216 .14 .18 210 .005** Serious delinquency .12 .19 216 .10 .19 211 .18 Suspensions .25 .43 210 .14 .35 203 .007** Drug use .30 .32 215 .17 .24 210 .0000** Self-reported grades 2.67 .91 220 2.69 .84 219 .80

Background variables Sex (% male) 49% -- 221 51% -- 220 .74 Age 13.10 .87 225 13.15 .95 221 .48 Race (% minority) 31% 220 45% -- 209 .005** Parental education 2.28 1.15 196 2.49 1.17 189 .08 Reading ability 1.82 .87 200 1.85 .86 201 .72

a Not available in 1981.

* Nearly significant. ** Statistically significant.

-248- 121 figNititist CetrolatIoss Botwvim get, (Iiimecity vs. *ROI *04 MI Quoottemostro halt's°

44000140,004,* 111040Vt*

tiolot le 04. W.* .13 .006 380

011141 too .15 .000 409

lovoltromosit .17 .000 409

tfsies4 .18 .000 422

Plitvotal tidoestom .22 .000 370

Ptittital taawledi, -.11 .01 385

footeoltioss -.IS .000 399

mA.,26,4m4varwr1pairstmamm.wcamsmaimmawat

Stolottlp*mu, co6o4 "0" and vbite coded "1."

bigotmire 141amallee Samplo.

..29 122 9.111161121. Z. oiL111.17. . =11.11113111111.1.Y110.61:111111 4.1....,101111111MVIINENIMEMI*11..- 7.11NIMIWOM.. Table 3

Significant Differences between Milwood and South on Selected 1981 SAES Measures Controlling for Race and Fdi-ental Education

Milwood South

Adjusted Adjusted mean N mean N p

Suspensions .25 181 .12 165 .002

Rebellious autonomy .71 167 .62 145 .01

School nonattendance .52 4192 .38 176 .04

Delinquency .19 184 .15 169 .03

Drug Use .30 183 .18 169 .000

250

123 Table 4

Comparisons of Milwood and Southon Selected 1982 SAES Measures

Milwood South

M SD N N SD N1

School variables Attachment to school .64 .28 474 .57 .26 224 .002** School rewards .17 .23 496 .19 .24 221 .30 School punishment .22 .25 497 .29 .30 221 .001** Parental emphasis on education .62 .28 407 .61 .30 210 .68 School effort .58 .32 563 .59 .29 233 .68 Negative peer influence .24 .25 .24 230 .58 Victimization .16 .20.223 45L91.71 .21 .26 220 .005** Involvement .18 .16 550 .21 .17 231 .02**

Psycho-social variables Attachment to parents .55 .30 558 .55 .29 232 .99 Belief in rules .70 .27 332 .68 .26 191 .40 Positive self-concept .74 .18 384 .73 .18 199 .53 Rebellious autonomy .63 .34 302 .65 .32 183 .52 Interpersonal competency .78 .24 328 .75 .25 188 .18 Internal control .40 .26 346 .43 .26 191 .22 Alienation .38 .29 342 .43 .25 193 .04** Practical knowledge 1.35 .46 295 1.25 .49 186 .02** Educational expectation .42 1.79 566 3.42 1.72 236 .72

Self-reported behavior variables School non-attendance index .48 .73 566 .56 .62 234 .14 Delinquency .17 .20 507 .20 .21 215 .07* Serious delinquency .11 .19 509 .14 .20 214 .06* Suspensions .21 .41 459 .18 .38 218 .36 Drug use .28 :34 514 .27 .31 220 .72 Self-reported grades 2.65 1.08 565 2.68 .95 230 .72

Background variables Sex (% male) 48% -- 565 54% -- 235 .001** Age 13.23 .93 573 13.22 .90 235 .89 Race (% minority) 34% -- 676 46% -- 308 .001** Parental education 2.56 1.21 511 2.55 1.13 218 .92 Reading ability 1.75 .88 516 2.04 .84 216 .001**

1These are the actual N's. Means and SD's for the South sample are weighted by the probability of sampling; all 1980-81 respondents were resurveyed. * Strong trend. **Statistically significant.

-251- Table 5

Comparisons of Milwood and South on Changes in SAES Scores from Spring 1981 to Spring 1982

Milwood South

SD N M SD N

Changes in school variables Attachment to school -.09 .33 69 -.05 .37 68 .48 School rewards -.07 .36 75 -.06 .37 68 .85 School punishment .04 .42 .75 -.04 .41 69 .26 Parental emphasis on education -.06 .38 69 -.09 .37 71 .60 School effort -...05 .42 81 -.01 .40 74 .63 Negative peer influence .07 131 75 .01 .32 74 .31 Victimization .05 .29 71 -.06 .25 67 .82 Involvement -.02 .21 76 .00 .25 70 .56

Change in psycho-social variables Attachment to parents -.06 .43 83 -.05 .42 74 .85 Belief in rules -.02 .36 59 -.10 .34 55 .21 Positive self-concept .00 .27 58 .00 -.24 60- .98- Rebellious autonomy -.03 .42 47 .07 .43 52 .25 Interpersonal competency -.03 .36 55 .00 .30 57 .68 Internal controla Alienation .11 .39 60 -.01 .36 59 .07* Practical knowledge -.03 .55 55 -.14 .61 54 .32 Educational expectation -.12 2.45 84 -.33 2.63 72 .60

Changes in self-reported behavior variables School non-attendance index .07 1.16 84 .19 1.14 73 .51 Delinquency .05 .31 70 .05 .30 67 .95 Serious delinquency .03 .27 70 .02 .30 67 .86 Suspensions .01 .63 70 .12 .48 67 .27 Drug use .10 .49 73 .12 .41 68 '.84 Self-reported grades -.15 1.53 80 -.04 1.36 70 .65

Changes in school population (background) variables Parental education .04 1.71 69 .12 1.62 61 .77 Reading ability -.13 1.23 67 .34 1.34 65 .04**

allot available for 1980-81.

*Nearly significant. **Statistically significant.

-252- 12 5 Table 6

South Students Performed Better than Milwood Students on 1981 M.A.T.s

Milwood South

Adjusted Adjusted meana N meana N p

Reading 32.5b 586 36.3 571 .000* (44)c (52)

Language 33.6 586 36.6 571 .000* (44) (51)

Math 28.4 586 31.4 571 .000* (47) (56)

Composite 94.5 586 104.3 571 .000* (45) (53)

Note. Results are the same whether using raw scores, percentiles, scaled scores, or normal curve equivalent scores.

a Means are adjusted for grade, sex, race and age, using SPSS routine ANOVA with classical analysis, controlling for school, grade, sex, race, and age. b Adjusted raw score.

c AdJusted percentile score.

*Highly statistically significant. Table 7

Milwood Students Perform as Well as South Students on 1982 Reading and Language M.A.T.s, and Peform Better than South Students on 1982 Math M.A.T.s

Milwood South

Adjusted Adjusted meana N meana

Reading 38.0b 5z6 38.3 580 .61 (56)c (56)

Language 37.7 576 37.5 580 .58 (53) (53)

Math 32.2 576 31.1 580 .01* (58) (54)

Composite 108.0 576 106.8 580 .42 (56) (54)

Note. Results are the same whether using raw scores, percentiles, scaled scores, or normal curve equivalent scores. a Means are adjusted for grade, sex, race and age, using SPSS routine ANOVA with classical analysis, controlling for grade, sex, race, and age. bAdjustedraw score. cAdjusted percentile score.

*Statistically significant.

254

127 Table 8

M.A.T. Composite Raw and Percentile Scores for Students of Different Ethnicity, Adjusted for Age: Kalamazoo 1981 and 1982a

White Black Other

Adj. Meanb N Adj. Mean N Adj. Mean N

1981 105.56c 770 84.84 338 102.09 49 .000* (55)d (34) (51)

1982 114.31 765 82.69 358 96.05 33 .000* (62) (40) (55)

M.A.T. Composite Raw and Percentile Scores for Students of Different Gender, Adjusted for Age:Kalamazoo 1981 and 1982a

Male Female

Adj. Meanb N Adj. Mean

1981 97.09 559 101.48 598 .008* (47) (51)

1982 105.81 566 108.89 590 .03 (53) (56)

Note. Results are the same whether using raw scores, percentiles, scaled scores, or normal curve equivalent scores. a Race effects are adjusted for grade, sex, and age. Sex effects are adjusted for race, grade, and age. b Adjusted means and 2. levels are from SPSS routine ANOVA, using classical analysis. c Adjusted raw score. dAdjusted percentilescore.

*Statistically signficant.

255 128 Table 9

South Students Perform Better than Milw000d Seventh Graders on the 1981 M.A.T.s

Milwood South

Seventh Eighth Seventh Eighth

Adj. Mean N Adj. Mean N Adj. Mean N Adj. Mean N

Reading 27.4a 312 39.8' 274 31.1 284 40.0 287 .000* (35)b (59) (43) (58)

Language 28.8 312 40.3 274 31.5 284 40.5 287 .000* (35) (59) (40) (59)

Math 23.6 312 35.2 274 26.2 284 35.4 287 .000* (35) (65) (42) (66)

Composite 79.8 312 115.3 274 88.8 284 115.9 287 .000* (34) (61) (40) (62)

Note. Results are the same whether using raw scores, percentiles, scaled scores, or normal curve equivalent scores. Means are computed separately for each school, and adjusted for sex, race, and age. Adjusted means are from SPSS routine ANOVA, using classical analysis.2lev- els are for the interaction of school and grade, controlling for school, grade, sex, age, and all other two-way interactions. aAdjusted raw score. b Adjusted percentile score.

*Highly statistically signficant.

-256- Table 10

Milwood Students Perform as Well as South Students on the 1982 M.A.T.s

Milwood South

Seventh Eighth. Seventh Eighth

Adj. Mean N Adj. Mean N Adj. Mean N Adj. Mean N

Reading 32.7a 292 45.1 284 31.8 291 43.3 289 .39 (47)b (69) " (44) (64)

Language 33.6 292 43.5 284 32.2 291 41.3 289 .31 (46) (65) (43) (61)

Math 28.7 292 37.2 284 26.4 291 34.6 289 .13 (50) (70) (42) (60)

Composite 94.9 292 125.8 284 90.3 291 119.2 289 .21 (56) (69) (42) 62)

Note. Results are the same whether using raw scores, percentiles, scaled scores, or normal curve equivalent scores. Means are computed separately for each school, and adjusted for sex, race, and age. Adjusted means are from SPSS routine ANOVA, using classical analysis. 2 levels are for the interaction of school and grade, controlling for school, grade, sex, age, and all other twoway interactions. aAdjusted raw score. b Adjusted percentile score.

*Highly statistically signficant.

-257- Table 11

Average Change in Adjusted M.A.T. Percentile Scores from 1981 to 1982 for the Milwood and South Seventh and Eighth Grades

Milwood South

Seventh Eighth Overall Seventh Eighth Overall

Reading +12 +10 +12'' +1 +6 +2

Language +11 +6 +9 +3 +2 +2

Math +15 +5 +11 0 +6 +2

Composite +22 +8 +12 +2 0 +1

Note. These percentile change scores are adjusted for sex race, and age. The interaction of school x year, controlling for sex, race, and age, is significant, p<.001, for all four M.A.T. scores.

-258- 13 Table 12

Comparisons of Milwood Alternative Program Students to Milwood Eighth Graders

M.A.P. Eighth Graders

M SD N M SD N p

School variables Attachment to school .62 .25 45 .61 .28 202 .73 School rewards .20 .24 47 .13 .19 213 .02** School punishment .26 .27 .47 .24 .26 213 .53 Parental emphasis on education .55 .26 42 .59 .28 182 .42 School, effort .48 .32 51 .54 .32 230 .23 Negative peer influence .34 ,p.29 49 .23 .22 231 .005** Victimization .19 .23 46 .15 .20 213 .19 Involvement .17 .16 51 .17 .15 229 .77

Psycho-social variables Attachment to parents .48 .24 51 .51 .30 231 .42 Belief in rules .65 .23 35 .68 .28 147 .54 Positive self-concept .68 .15 39 .72 .18 166 .19 Rebellious autonomy .72 .31 36 .68 .32 127 .46 Interpersonal competency .82 .24 37 .75 .24 140 .09* Internal control .51 .28 38 .41 .25 153 .02* Alienation .46 .26 38 .38 .28 150 .10* Practical knowledge 1.34 .37 36 1.40 .44 130 .43 Educational expectation 2.76 1.89 51 3.47 1.75 235 .01**

Self-reported behavior variables School non-attendance index .63 .75 51 .52 .74 234 .36 Delinquency .27 .25 45 .21 .21 219 .10* Serious delinquency .19 .25 42 .12 .20 219 .04** Suspensions .44 .50 43 .20 .40 204 .001** Drug use .41 .40 43 .37 .40 219 .04** Self-reported grades 2.28 .82 47 2.59 1.10 234 .06*

Background variables Sex (% male) 49% 49 44% -- 233 .65 Age 14.00 .81 51 13.70 .70 235 .009** Race (Z minority) 26% 51 20% -- 290 .08* Parental education 1.97 1.12 44 2.61 1.12 217 .001** Reading ability 1.72 .86 46 1.76 .86 223 .77

* Almost significant. **Statistically significant.

-259-

13 1,==.111410 Table 13

Significant Differences between M.A.P. and Milwood Eighth Graders, Controlling for Parental Education, Ethnicity, and Age

M.A.P. Eighth Graders

Adjusted Adjusted Outcome measure mean N mean

School rewards .23 38 .12 186 .002

Suspensions .41 35 .21 177 .01

-260- 133 a " 7 -.,:'..7',;;PW7?"',_,..4:::,;:trInaagrASPialefeERZEZAVEZIMICI:Zel,....2111ZENC.A. Table 14

Significant Changes on SAES Scores from 1981 to 1982 for M.A.P. Students and Milwood Eighth Graders

M.A.P. Other Eighth Graders

Outcome measure Mean Change SD N Mean Change SD N p

Educational expectation -1.67 2.69 15 .22 2.29 68 .006**

Negative peer influence .25 .36 11 .03 .29 63 .03**

-261- 134 Table 15

Comparisons of Skills Lab Students to Milwood Population and to M.A.P.

Skills Lab Milwood M.A.P.a

M SD N M SD N p

School, variables Ati hment to school .43 .23 14 .64 .28 464 .02** .006* School rewards .11 .18 9 .17 .23 487 .44 .15 School punishment .45 .32 9 .21 .25 488 .005** .03** Parental emphasis on education .50 .29 7 .62 .28 400 .25 .77 School effort .28 .32 16 .59 .32 547 .001** .004** Negative peer influence .56 .28 14 .23 .22 533 .001** .001** Victimization .22 .34 9 .15 .19 482 .30 .77 Involvement .12 .14 15 .18 .16 535 .11 .17

Psycho-social variables Attachment to parents .43 .32 16 .56 ".30 542- --.09*- .59- --: Belief in rules .57 .19 5 .71 .19 327 .24 .35 Positive self-concept .55 .21 5 .74 .18 379 .03** .04** Rebellious autonomy .73 .37 5 .63 .34 297 .49 .93 Interpersonal competency .83 .32 6 .78 .23 322 .61 .82 Internal control .52 .30 6 .40 .26 340 .28 .94 Alienation .51 .32 7 .38 .29 335 .21 .58 Practical knowledge 1.19 .28 6 1.36 .47 289 .38 .28 Educational expectation 1.35 1.70 17 3.43 1.76 549 .001** .001**

Self-reported behavior variables School non-attendance index .94 .97 17 .47 .72 549 .009** .02** Delinquency .47 .28 12 .17 .20 495 .001** .001** Serious delinquency .39 .32 12 .10 .18 497 .001** .002 ** Suspensions .88 .35 8 .20 .40 451 .001** .0l ** Drug use .56 .37 14 .27 .34 500 .002** .06** Self-reported grades 1.50 .94 14 2.68 1.07 551 .001** .001**

Background variables SQX (% male) 69% 16 472 -- 549 .15 .23 Age 14.2914. 17 13.20 -- 556 .001** .09* Race (% minority) 13% 24 23% -- 652 .46 - .13 Parental education 1.33 .86 15 2.60 1,20 496 .001** .008** Reading ability 1.18 .81 17 1.77 .88 499 .006** .008**

aComparison of Skills Lab students to M.A.P. students.

* Strong trend. **Statistically significant Tattle 16

Stkeificsot Conparisoos of Vills Lab and N.A.P. Students, Costrolliog (or forestal Education, Readirsg Ability and Age

IWOIWWWWMI.OWWW WWWWWW 0.111.1.11.WWWWwor Skills Lab N.A.P. 1111.1100.11 Adjusted Adjusted mean N mean 7110WW1VMS.WWwpwimmilm

4$ Altocksest to school .40 .68 30 .03

tescatiseal espetcatios 1.14 15 3.13 1 .001

tovelwomost .08 13 .21 31 .03

Ative per infleesce .56 12 .28 30 .03

kiwi effort .36 14 .64 31 .03

Stbool renisboont .45 7 .24 31 .09

eel istoteay .47 10 .21 27 .02

kuj see .62 12 .29 27 .06

Serteus deliequesty .41 10 .15 27 .04

Grades 1.57 12 2.49 29 .004

Sosponsione .96 6 .34 30 .02 Table 17

Comparison of Quasi-Randomly Selected Student Council Students to Students Not Selected for the Student Council

Student Council Control

Personal characteristic M SD N M SD N

School variables Attachment to school .59 .27 25 .64 .29 151 .40 School rewards .30 .31 28 .20 .23 158 .05** School punishment .21 .20 28 .25 .26 159 .51 Parental emphasis on education .53 .32 22 .66 .25 137 .04** School effort .50 .32 28 .58 .37 175 .21 Negative peer influence .32 .26 25 .23 .23 170 .04** Victimization .19 .22 28 .18 .22 159 .75 Involvement .27 .15 28 .20 .16 171 .03**

Psycho-social variables Attachment to parents .43 .33 30 .56 .29 175 .02** Belief in rules .69 .28 18 .70 .27 116 .91 Positive self-concept .66 .23 20 __.75. .18. 133 .05** Rebellious autonomy .63 .37 17 .64 .33 109 .92 Interpersonal competency .76 .27 18 .79 .23 113 .61 Internal control .49 .23 18 .43 .27 121 .39 Alienation .39 .32 17 .38 .29 119 .89 Practical knowledge 1.34 .47 17 1.40 .44 106 .58 Educational expectation 2.60 1.87 30 3.58 1.76 176 .006**

Self-reported behavior variables School non-attendance index .53 .73 30 .47 .73 178 .64 Delinquency .21 .17 24 .19 .21 158 .70 Serious delinquency .10 .12 24 .12 .20 158 .73 Suspensions .35 .48 26 .21 .41 147 .13 Drug use .32 .38 24 .31 .36 159 .97 Self-reported grades 2.69 .97 29 2.69 1.02 175 .98

Background variables Sex (% male) 55% 29 45% -- 173 .42 Age 13.27 .87 30 13.31 .87 178 .78 Race (% minority) 27% -- 34 22% -- 226 .53 Parental education 2.31 1.51 27 2.66 1.23 158 .20 Reading ability 1.70 .91 27 1.82 .87 161 .52

* Almost significant. **Statistically significant.

-264- 137 Table 18

Significant Differences between Student Council Members and Other Students When M.A.P. and Skills Lab Students are Excluded from the Analysis

Student Council Other Students

M SD N M . SD N

Attachment to parents .44 .35 25 .58 .30 151 .04

Involvementa .26 ,.15 23 .21 .17 147 .14

Negative peer influence .35 .24 25 .22 .22 146 .006

Parental emphasis on education .54 .33 17 .66 .25 117 .09

School effort .48 .32 23 .60 .32 151 .10

School rewards .29 .32 23 .19 .25 136 .08

Self-esteem .65 .27 15 .75 .16 114 .05

aIncluded because of its relevance to the project. Not statistically significant.

-265- 138 -266-

139 Project PREP: Second Interim Report

D. K. Ogawa and G. D. Gottfredson

Rationale Although the theories elaborated thus far deal with why the problems ProjectPREP is continuing its exist in District 9, further elabo- effort to reduce high truancy, drop- ration of PREP'saction theories, out, and suspension rates, along i.e.,the project's theories which with disruptive student behaviors in will guide their interventions, is 12 schools in District 9in Bronx, needed for the third year. New York. Ultimately, Project PREP anticipates that teachers and stu- Objectives dents will perceive school as a safe and viable place for learning. Alt- The projectobjectives,gleaned hough the statement of the project's from several sources, are summarized theoretical rationale remains essen- in Table 1. Drawing from the tially the same as last year's, the updated theoretical framework, the following ideas have been added: following four objectives were added to last year's: (a) reducing nega- 1. Lack ofopportunities and tivepeerinfluence amongprogram positive experiences create participants when compared to compa- fear and frustration in stu- rable controls; (b) increasing dents, lessens stakes in con- opportunities for a successful tran- formity, lessens self-esteem sition to high school and/or a and ultimately leads to anger career; (c) increasingthe use of and hositility. Creation of fair and consistent school rules for opportunities, e.g., jobs, and program participants; (d) increasing positive use of leisure time students' beliefs in conventional will decrease delinquency and rules. This year's "objectives" decrease the chances of asso- include processes or critical bench- ciating with those who do marks necessary for implementation. engage in delinquency. Were the project's theoretical framework sharpened, a clearer dis- 2. Students lack caring, posi- tinction could be made between tive,and strong role models objectives and critical benchmarks and a consistent set of firm for program implementation. and fair guidelines to follow. Intended Interventions 3. Students lack commitment to the conventional norms and In addition tothe three major beliefs of society. components in the first year (i.e., the Alternative School, the Citizen- ship Clusters, and the Youth component) a fourth intervention, student council,was added forthe second year of operation. The student This second interim report sup- council is composed of eight elected plements an earlier report(Ogawa, representatives from the Alternative 1982) which should be consulted for School and one representative from a basic overview of this project and each ofthe eight cluster schools. an account of earlier activities.

-267-

140 PREP

The Youth Director is primarily director, a teacher-coordinator and responsible for the student council the paraprofessionals. at the Alternative School andthe two teacher-coordinators are respon- In addition to the data collec- sible for the studentrepresenta- tor, a new Youth Director was hired tives atthe Cluster schools. The for thesecond andthird year to council convenes at least monthly to coordinate the Youth Program and the learn aboutdecision-making proce- student council. dures and developing an agenda for initiating school policy changes. Program Implementation After a meeting at the Alternative School, the Cluster representative Project PREPhas been serving meets with the rest ofthe Cluster students for two years; however, for students at his or her home school. the first half year, the project was This intervention is directed at the run bya director with a different following objectives: (a) reducing philosophy than the present direc- negative peer influence; (b) in- tor. Although the present director creasing the number of active, pro- ,,has been on staff since April 1981, ductive youths, (c) increasing ithas only been recently, during self-concept, and d) in-creas-ing this past year, that the project is attachment to school. moving in the direction that he has conceptualized. An example ofthis Project Environment is the development of curriculum to prepare students for options other

A major change has taken place in than college. Pianos and other _ School District 9, the implementing musical instruments were= acquired, organization. During the first year as well as equipment for a photogra- District 9's DeputySuperintendent pl,v class and an automechanics (DS) acted as project manager. Fol- lowing the DS's resignation, the administrative responsibility for The extent and nature of this Project PREP was shifted to the Dis- year's implementation was ascer- trict's funded programs office, and tained primarily from project the Community School Board has records, the program development become more involved with monitoring worksheets, supplemented with inter- the project. views with the project's staff. No formal comprehensive monitoring dev- Staffing ices were developed to record other important dimensions of implementa- The District 9 Director of Evalu- tion such as the degree and extent ation and Research and his assistant to which project staff have adopted are no longer involved in performing alternative education techniques evaluation activities. TheCommu- (for example, seeHall & Loucks, nity School Boardhas assigned a 1977; Leithwood & Montgomery, 1980). data collector to assist the project director with major evaluation Alternative School activities such as the spring survey administration. Although the data Most of the project development collector supervised the paraprofes- occurred with this intervention. sionals who actually collected the The participants are stillseventh data, he did not adopt the role of and eighth graders referred from an on-site evaluationcoordinator. four local junior high schools for Many of the on-going evaluation behavioral problems. However, the tasks were performed by the project

-268- PREP

emphasis of the intervention now studentsare prepared academically includes providing practical experi- for high school. The project direc- encesas well as academics. Both tor was concerned, however, that the career awareness element and the Project PREP students would become transition to high school components so acclimated to the supportive were specified in the original pro- alternative school environment that posal, but were not fully conceptu- they would beunableto make the alized orimplemented. The career transition to high school once they awareness element originally graduate from the eighth grade. As involved exposing students to guest a consequence, arrangements were speakers of differentprofessions. made with local high school princi- As implemented in year two, this pals to allow eighth grade PREP stu- component allows students to get dents to attend classes in the morn- experience at the alternative school ing. Five eighth graders were in various activities such as pho- placed in high school last February. tography, auto mechanics, carpentry, They took courses during the morning and electronics. Inaddition, by then came back tothe Alternative the end ofthe year,students were School in the afternoons for tutor- exposed to job resumes, participated ," ing. and counseling,in addition to in mock job interviews, and, by the participating in the practical cur- summer, 25 students were reportedly riculum. placed in jobs around the community forminimum pay. In addition to Both the academic and practical providingstudents with vocational components of the Alternative School experiences, the program provides are designed to increaseacademic students with practical experiences achievement, self-esteem, attachment such as exposure to museums, restau- to school, student involvement, and rants, ball-games, and taking a to decrease negative peer influence train trip to other nearby cities. among the targeted students as compared to comparable control stu- English, math, social studies, dents. In addition to the academic and science are taught in the morn- andpractical components, a third ings. Teachers work with eight stu- component, informal counseling, dents per class for a one-hour per- designed to increase students' iod. Individualized educational beliefs in conventional beliefs, is plans, based on testscores,were primarily performed bythe project initiated for students in December, director during field trips or other 1981, and were updated the following activities whenever and wherever a April for the remainder of the year. topic comes up. Students also call Students experiencing academic dif- the youth director and teachers ficulties are also provided with after hours for help. In order to tutoring afterschool. For those provideconsistent availability of experiencing emotional difficulties, staff to students, arrangements are psychological and drug counseling being made to provide a 24-hour arealso available from thestaff "hot-line" to students. The project psychologistandthe localmental director feels that the exposure to health clinic. different experiences and a supportive caring staff are the most Although an alternative setting, important aspects of the alternative the school is structured as a junior school. highor intermediate schoolwhere

-269- 142 PREP

Citizenship Clusters misbehaving. She ascertains why the student misbehavedand allows the While the project director is in student to vent his or her emotions. charge of the Alternative School, She then explains why that behavior the two teacher-coordinators are was inappropriateanddrawsup a primarily responsible for the Citiz- behavioral contract agreed upon by enship Clusters,i.e., the teachers the student, teacher,and parapro- of project students, the paraprofes- fessional. Ifthe student fulfills sionals who operate the Time-Out the contract, he or she receives Room,and the students themselves. points which are later redeemed for The teacher-coordinators arranged prizes. When there are no students weekly meetingswithteachersand in the Time-Out Room,the parapro- paraprofessionals, coordinated stu- fessional goes to a Cluster class to dent council activities, provided get fourstudents for'reading and the prizes for the token economy and math tutoring. Each student is to were in charge of evaluation issues be seen by the paraprofessional such as maintaining the design and twice each week for rap sessions or supervising data collection activi- ,counseling. Whethervisiting the ties. Time-Out Room for tutoring or a behavioral problem, students are in Unlike last year, the token econ- the Time -Cut Room no longer than 45 omy went smoothly due to the ready minutes before returning to class. availability of the prizes. How- ever,the function ofthe Time-Out Although all paraprofessionals in Room changed from the behavioral the eight cluster schools do essen- model of a nonstimulating, "cool tially the same tasks, the intensity down" environment for a misbehaving variesacross schools, as can be student to a microcosm of the "per- seen in Table 4.' Forbehavioral fect classroom." Once behavioral problems, students at one school had contracts were established with pro- as few as five contacts whereas at ject students, tne Time-Out Room was another school students had an aver- utilized less. Some principals felt age of 87 contacts. Theaverage that the paraprofessionals were not number of rap sessions each student using their time \efficiently and had ranged from two at one school to consequently suu,sted that the par- as many as160 at another school. aprofessionals oiler tutoring to Part of this variation may depend on project students. The Time-Out Room the relationship between the para- eventually became E supportive envi- professional and teacher. Some ronment where ,Ludents received teachersinitially feltthreatened indivicivalized ' itorial and counsel- by the presence of a paraprofession- ingservic s .nn the paraprofes- al providing direct services to stu- sionals, as well as a place to "cool dents. This was especially true if down." It appears from subjective students became attached to the accounts that the Time-Out Room met paraprofessionals. Other teachers the needs of studentswho needed excessively use the Time-Out Room to personal attention, both academi- unburden themselves from certain cally and emotionally, but who could students. Meetings with the teach- not get this from a regular class- ers, paraprofessionals, and the room. staff psychologist were held in an attemptto resolve these problems; One paraprofessional from a jun- but, attendance at these meetings ior high school describes her role was minimal. as first talking to a student who is sent to the Time-Out Room' for

-270-

143 PREP

Insummary, the"Time-Out-Room" Student Council is a misnomer for the model actually being implemented in the Cluster This was the first year this Schools. Although it appears that intervention was implemented. Alt- the "Time-Out-Room" has moved away houghcouncilmembers were intro- from the behavioral model, the ori- ducedto parliamentaryprocedures, ginal conceptualization of this took field trips to governing agen- intervention, as outlined in the cies, and addressed school and com- project's proposal, includes indivi- munity problems, the project direc- dualized tutoring and counseling tor felt the councilwas not as services. Unclear role expectations strong as anticipated. This was due between teachers and paraprofession- in part to the fact that the council als and haphazard referralof stu- only met oncea month and council dents at some schools threaten the decisions were often temporary. strength and integrity of this intervention. Overall, the strength Parent Involvement of this intervention,accordingto the project director, depends on the This has been the most difficult relationship between the student and component to implement. Ministers, the paraprofessional. technical assistance, and consultants have been utilized this year Youth Component in order to organize parents. The variousstrategies weresuccessful This intervention was originally in involving some parents. designed to bring Alternative School and Cluster students together to Implementation Overview engage in positive leisure time activities. In year two,however, Project PREP'S interventions have the Alternative students and the been implemented in varying degrees. Cluster students became more inde- Tables 3 through 4 provide an over- pendent of each other. Alternative view of project implementation. students plan their own field trips Table 3 shows the average number of separate from Cluster students and times alternative school students utilize separate night centers. were reported to be involved in var- There are 16 night centers in the ious activities,and Table 4 pre- District sponsored by Continuing sentssimilar information for the Education which areopen from 7-9 cluster schools. Implementation is p.m. Only one of these centers is vastly different in different run by the Alternative School Staff. schools. Alternative students attend this center while Cluster students attend Interim Outcomes other nearby centers staffed by some of the para-professionals who take Methods attendance. However, token points are no longer given to students who Design. With thefirstyear's attend the night center. Although non-equivalent control group design, students are not required to attend little could be concluded about the the night centers,field trips, or effectiveness of Project PREP's other activities,they are strongly interventions. Consequently, an encouraged. Ofallthe activities experimental design was established of the Youth component,the night in both the Alternative School and centers are utilizedmost often. the Cluster schools. A pool of eli- Other components include winter gible referrals from the four feeder camp, summer camp and summer jobs.

-271- PREP juniorhigh schools were randomly the Gates' program which offers assigned to either the Alternative remedial reading. In addition, one school or a control condition in Clusterschoolhas a drug program which the students would remain in forstudents and Special Education his or her original school. A simi- has resource rooms setup atsome lar procedure was conducted for the schools to deal with academic prob- Cluster Schools. Cluster treatment lems. While Project PREP is unique students were designated by the PREP in that it offers both academic and staff to utilize the Time-Out-Room. behavioral services, there are other Cluster controls were to be treated sources from which controls can no differently from non-program par- receive help. ticipants. Subjects. Ofthe treatment and Dueto the mobility of students control students, 70% were black and in the Bronx,it was necessary to 30% hispanic. Females made up 33% maintain a continuous pool of refer- of the sample while 67% of the sam- rals. Because some treatment stu- ple were males. Fifteenstudents dents onthe waiting list subse- returned to I I the Alternative School quently withdrew from school, it from last year, leaving 25 openings became necessary to develop a treat- to befilled bystudents randomly ment waiting list and a control selected from a referral pool. Dur- waiting list. Each time aperson ing the year,the project director from the treatment waiting list was accepted into the Alternative School assigned to treatment, a person from three non-random referrals from thecontrol waiting list was also principals and one from the proba- added to the control group. Post- tion department. Each Cluster randomization checks using the SAES School served 15 students, with the pretestshowed nosizablediffer- exception of School 148, which ences between treatment and control expandedits program later in the students. Larger pools of eligibles school year to 30 students. were developed in some schools than in others, resulting in some non- Measures. Weintended to rely proportionality in the design. primarily upon theSAESquestion- naire for measures of Project PREP's The experimental design was ini- goals and objectives. Official tially met with some resistance by school records and the Project's principals who wanted to place cer- parent questionnaire were also to be tain students in the Alternative used. The project administered the School and other principals who parent questionnaire twice during wanted allstudents served by the the year to parents of PREP partici- Time-Out Rooms. Over time, princi- pants. pals accepted the randomization process; however, there were a few Data quality. The utility of the occasions when the project director data available is severely limited accepted nonrandomized referrals due to extensive missinginforma- into the Alternative School. These tion. Required information was not students are excluded from our ana- provided in sufficiently complete lyses. Although control students form to make most analyses meaning- reportedly do not receive any PREP ful. For completeness, tables show services, itis possible that they the actual number of cases for whom receive other services provided by data are available. In Table 5 we the District. For example, control illustrate the missing data problem. students can attend any of the night For theAlternativeSchool, there centers or they may be involved in

-272-

145 PREP were 21 control and 31 treatment The results shown for class cut students. For the Cluster schools, ting and school cutting in Table 6, there were 96 control and 162 treat although showing significant differ ment students. Complete data on ences,are not very meaningfuldue withdrawal from school were availa to the low response rates for these ble. Data on absences from school questionnairebased measures. are also reasonably complete. Other data areso often missingthat no Project PREP's Objectives. The confidence may be had in analyses of remainder of the results shown in the data that are present./ (Table 5 Table 6 relate to Project PREP is illustrative only. Other questi objectives. Due to extensive miss onnairebased measures resemble the ing data, these results are undepen measures shown in Table 5in their dable. Nevertheless, the results completeness.) suggest that victimization may be slightly higher among Alternative Results school students than among controls (p < .05),and that negative peer Project PREP'sgoals. Table 6 influence may be higher in both showstreatmentandcontrolgroup Alternative School and cluster comparisons on several measures of treatment groups than in the control Project PREP's goals. Four measures groups (p < .10). were intended to assess truancy and dropout. As noted earlier, however, A number of Project PREP objec the only sufficiently complete data tives are specific to the Alterna for meaningful analyses are for tive School. Results relating to absences (based on schoolrecords) these objectives are summarized in and withdrawalfrom school. Stu Table 7(for objectives relating to dents at the Alternative School withdrew from school more often than controls, but this result could have occurred by chance. Treatment stu cluster schools was performed sepa dents in the Cluster schools with rately for each school. Because of drew from school significantly (p < differences in the 'size of the pool .02) moreoftenthandidcontrol of eligibles, different proportions students. This difference is of students were selected for treat unlikely to have occurred by chance. ment in theeight schools. This impliesthat school is 'associated Alternative School treatment stu with probability oftreatment,and dents were absent slightly and non that differences observed between significantly less often than con the pooled treatment and control trol students. Among cluster school groups could be due to school dif students for whom data were ferences aswell as to treatment reported, attendance was better (.10 differences. Data are not analyzed < p< .11) for treatment than con separately by school due to the very trol students. Attendance data were low statistical power that would reported, however, only for students result from the small sample sizes who did not withdraw fromschool. involved. When significant treat Recall that significantly more mentcontrol differences were found, treatment (13%) thancontrol Stu statistical controls for school were dents (4%) withdrew from school.' introduced to determine whether school differences could account for the result. Suchadditional ana lyses were performed for withdrawal 1 The randomization process for from school and days absent. (These 273

146 PREP students). Aswas true formost less stillin 1982. (f) More stu measures summarized in Table 6, dentteacher interaction than in the missing data makes interpretation of typical school was reportedboth these results difficult. No signi years, but students reported less ficant differences were found bet interactionin 1982 thanin 1981. weentreatmentandcontrol groups (g) In 1981 students reported however. receiving about the same levels of punishment in school as did students One way to assess the objectives in the typical school, but slightly for theAlternative School itself more than averagein 1982. (h) In (those relating to fair and consis both years students reported receiv tent rulesand to teacher compe ing many more rewards in school than tency) is to compare the school cli did students in the typical school. mate measures bearing on these objectives to those for other The, results of the parent questi schools and to the Alternative onnaire are shown in Table 8. Alt School's standing on these measures hough more parents who responded to in the previous year. Chapter 5 and ,,the questionnaire reported visiting the Appendix of Part I of this the school and being aware of vari volume show results ofthe climate ous opportunities inthe community surveys conducted in the spring of in the second half of the sch:ol 1981and1982. An examination of year, the sample may be biased due these results suggests the follow to the lowresponserate (32 to ing: (a) Students report thatthe 44%). In addition, factorswhich rules are fairer than do students in are not controlled for otherthan most other schools, and reports of Project PREP's interventions, may fairness increased somewhat in 1982 have produced the observed differ over 1981. (b) In both years stu ences. dents reported that the school rules were clearerthan did students in Discussion other schools. (c) In 1981 students reported fairly typical levels of Limitations victimization andsafety; in 1982 students reported significantly more Problems of missing data ravaged safety thanin1981. (d) In both the utility of the evaluation. years students reported high levels Satisfactorily complete data were of individualized instruction. available only for two outcome mea (e) Students reported experiencing sures: School absences and withdra slightly less disrespect from teach wal from school. Failure to admin ers than did students in the typical ister the questionnaire to treatment school in1981,and they reported and control ,students, and poor com pletion rates among studentswho were surveyed'imade the questionnaire measures only of very limited use analyses were not performed for fulness. The projectdevelopedand questionnaire measures where missing administered parent questionnaire data were severe limitations.) suffered from similar low response Large and significant differences in rates. Finally, information on dis withdrawal across schools are pre ciplinary referrals was collected in sentin the data. Even when con such a way that made it impossible trols forschool are applied, the to distinguish between students with observed difference in withdrawal is no referrals and students for whom present and statistically signifi no data were collected, and were cant (p< .025). therefore unusable. Accordingly, we 274

14.7 PREP regard only the absence and withdra- and management would be useful. wal results as dependable, and focus our discussion on those results. The PDE managment plan will need to be specified more carefully next Withdrawal from School year in order to carry out data collection activities. PREP showed A larger proportion ofstudents minimalimprovement over the first participating in the Alternative year with respect to data collec- School thanof control group stu- tion; however, and serious problems dents withdrew from school (19% vs. persist. Identification of obsta- 5%, respectively). This result, cles inthis area can be fed into although not statistically signifi- renewed planning to strengthen the cant, is worrisome. There is no evaluation. evidence that participation in the Alternative School reduces the risk The process of identifying obsta- of dropout. cles and formulating strategies to overcome the obstacles worked well A larger proportion of treatment with implementing randomization this students in the cluster schools than year. Randomization will again be of control students withdrewfrom implemented next year in the Alter- school (13% vs. 4%) and this differ- native and Cluster schools. Some of ence is statistically significant the Cluster schools will be expand- when statistical controls for school ing the number of students they of attendance are applied (p < serve from 15 to 30, which will .025). Evidence suggests that par- require maintaining a large pool of ticipation in the Citizenship Clus- referrals. Due to the high mobility ters increases the risk of dropout. rate of students, referral pools must be replenished and maintained Absence:; throughout the schoolyear. This effort will be of limited use unless Students attending the Alterna- the completeness of data collection tive School are absent about as improves. often as control students. There is noevidence that participation_ in Problems Project PREP is Encounter- the Alternative School reduces ing truancy. Implementation. The project Treatment students in the cluster director is skeptical about the use- schools who have not withdrawn by fulness ofthe Time-Out-Room as an the end of the school year are intervention. He feels the themes absent lessthan arecontrolstu- and prizes may be useful for elemen- dents, butthisdifferenceis not tary school students but that trips statistically significant. There is and various practical experiences no evidence thatparticipation in may be more valuable with the older the Citizenship Cluster reduces students. As a consequence, the truancy. Alternative School and the Clusters are separate entities, with the pro- Evaluation Strategies for 1982-83 ject director responsible for the Alternative Schooland theteacher The Program Development Evalua- coordinators responsible for the tion model has never been adopted by Clusters. The project director Project PREP. Clearer specification devotes most of his attention to the of the project's action theory and Alternative School. more careful attention to planning

-275-

148 PREP

Perhaps a more difficult task for if the effects of the project are to PREP is getting parents involved in be determined. their children's activities. It may be, however, that the project is Successes Project PREP is Experienc- dealing with an overwhelming obsta- ing cle in their forcefield--the community environment. Parents in the Althoughthe objectiveevidence community are often more concerned does not depict Project PREP as suc- aboutlarger issues suchas unem- cessful, subjective evaluations of ployment, poor public services, and the project are more promising. The high crime, rather than their chil- project is praised in the District; dren's activities. The community other schools in the district want environment may also make it diffi- to be served by PREP. After a visit cult for the project to reduce del- to the Alternative school, one staff inquent behavior in the schools and psychologist wasso impressed with among PREP students if the surround- the students, she sought air time on ing environment supports this behav- a radio broadcast so students could ior. 1,tell about their experiences.

Evaluationissues. Theproject Teachers and principalsin PREP did not devote sufficient personnel schools were initially skeptical, resources and attention tocollect but now they want the program the data completely, despite the expanded. Some students come to hiring of an on-site data coordina- school early, anddo not want to tor. Part ofthe problem alsois Sane 8th leave after_ school_ is out._Some the quality of records kept at some graders do notwant to graduate. of the schools. The way question- Non-PREP students at the Cluster naire administration has been plan- Schools have asked how they can join ned for and executed have been seri- Project PREP and participate in the ous defects which must be remedied activities and in student council.

-276- 1.49 PREP

Aeforepeeg

Moll, tit, 4 Look*, S.?. A developmental model for determining whetherthe tsvommeti it attuelly implvmented. kgagrigAnikiggjaignALiumg), 414404 1. I. :63.274.

Wit,wpo44 g.A. 4 Montgomtry4 D. S. tvaluating program implementation.IYA1- 1044AM617410+ 1980. 2, 193-214.

Nowa, D.K. Project PRSIPt An interim report of its evaluation. In C.D. Cott- emboli (111.)nig > ti v 1122 !0 *etiolate! Johns Sophia§ University, Center for the Study of the Social 6visotsation of Schools, (Report No325), 1982. Table 1

Project PREP's Goals and Associated Objectives

Goal I: Reduce truancy and dropout in the 12 participatingschools.

Objectives:

1. Increaseparental involvement and enhance the extentto which they value education. 2. Increase the number of active and involved students. 3. Increase academic achievement and success. 4. Increase student commitment/attachment to school. 5. Increase chances for a successful transition to high school and career. 6. Increase student self-concept. 7. Reduce negative peer influence, 8. Increase teacher competencies so thy can cope with problem behaviors and meet students' needs. 9. Increase students belief in conventional rules.

Goal II: Reduce the amount of disruptive behavior in the 12 schools.

Objectives:

1. See objectives 1 - 9 for Goal I, above. 2. Increase :.he use of consistent and fair guidelines for behavior.

Goal III: Reduce the number of suspensions in the 12 schools.

Objectives:

1. See objectives for other two goals (objectives are the same).

aThis objective is specificto the Alternative School.

-278-

151 Table 2 Average Number of Times Time-out Room Utilized During the 1981-1982 School Year for Each Cluster School

School Behavioral Rap Sessions, Problems Counseling M SD N M SD

55 21.9 25.6 17 3.2 3.2 17 ,,

63 44.7 7.7 16 89.3 7.7 15

64 86.6 17.8 14 160.5 18.6 13

82 4.9 6.9 22 5.4 14.5 21

117 41.9 14.7 16 56.9 30.0 16

132 13.8 12.1 19 2.3 2.4 19

147 30.1 33.4 18 94.5 68.3 18

148 10.4 9.7 39 8.0 7.8 39

For all schools 26.62 28.4 161 41.5 26.4 158 Table 3

Average Number of Times Alternative School Students Participated in PREP Activities During 1981-82 School Year

Activity N SD

Evening Recreation 28.0 21.7 Tutoring 11.1 8.6 Student Council 2.1 4.0 Field Trips 17.3 11.9 Drug Counseling 8.0 8.1 Volunteer Work 10.3 19.3 Parent Visits 4.1 .6 Practical Curriculum 40.2 26.3 Winter Camp' 0 0

Note. Non-randomized returning students are included; N = 46.

Table 4

Average Number of Times Cluster Students Participated in PREP Activities During the 1981-82 School Year

Evening Recreation Field Trips Student Council Tutoring School M SD N M SD N N SD N M SD /

55 0.0 0.0 17 3.1 1.2 18 0.1 0.2 18 0.0 0.0 : 63 53.9 12.5 15 1.4 2.0 17 0.6 2.4 18 24.3 7.7 : 64 82.1 20.0 14 1.0 0.0 14 0.0 0.0 14 102.1 30.2 :

82 1.4 3.6 22 0.2 0.4 22 0.1 0.6 22 0.3 1.1 :

117 a a a 1.0 0.0 16 0.1 0.2 17 49.9 16.1 : 132 0.5 0.7 2 2.0 0.0 .18 0.2 0.7 19 0.8 0.5 147 46.3 27.5 18 0.9 0.3 18 4.4 2.5 18 57.9 45.2

148 0.6 0.5 40 0.5 0.8 40 0.1 0.3 40 0.7 0.2 A

aInformation not available.

-280- 153 Table 5

Illustrative Percentages of Data Available

Alternative School Cluster Schools Measure Treatment Control Treatment Control

GPA 61 48 76 71

Withdrawal 100 100 100 100

Absences 90 90 84 78

Negative peer influence 54 38 55 53

Attachment to school 48 4 28 41 41

Interpersonal competency 39 19 31 29

Selfconcept 39 19 25 26

Delinquent behavior 45 19 48 44'

281-- 154 Table 6

Treatment-Control Group Comparisons for Project PREP: Overall Project Coals and Objectives

Alternative School Cluster Schools Control Treatment Control Treatment M SD N M SD Measure SD N M SD N

Truancy and dropout .13** .34 162 .04 .20 96 withdravalsa .19 .40 31 .05 .22 21 18.40 26.97 137 25.48 34.50 75 Days absents 9.91 13.15 28 11.53 14.98 19 .81* 1.29 107 .45 1.04 60 Class cutting .72 1.41 18 1.11 1.05 9 .50 .99 105 .38 .94 60 School cutting .44* .92 18 1.33 1.66 9

Disruptive behavior 4 .17 .21 78 .14 .16 42 Delinquent behavior .26 .18 14 .26 .15 7 .22 .24% 64 .21 .25 39 Victimization .25** .27 15 .02 .05'

Suspensions 2.40 5.22 15 3.12 7.36 8 Project records ------.38 , .49 60 .30 .46 37 Self-reported .47 .52 15 .50 .55 6

Other objectives .29 54 .64 .24 32 Parental emph. educ. .54 .30 13 .42 .34 6 .63 .28* .21 89 .22 ,14 51 Negative peer infl. .43* .22 17 .25 .25' 8 .19 87 .24 .21 51 .24 .18 17 .23 .23 9 .25 Involvement 25 .22 4 .66 .16 40 .67 .18 Self-concept .67 .15 12 .69 .76 .20 50 .70 .26 28 Interpersonal competency .63 .30 12 .85 .19 4 .28 67 .62 .26 39 Attachment to school .73 .20 15 .75 .18 6 .55 65.93 14.11 124 72.59 43.95 68 GPAa 66.08 16.08 19 56.58 21.96 10 b 2.39 1.07 102 2.43 .84 58 Grades 2.44 1.15 18 2.00 .92 8 1.41 .78 94 1.50 .91 54 Reading abilityb 1.69 .79 16 1.75 1.04 8

a Basedon school or project records.

bBased on SAES student questionnaire.

* p < .10 ** p < .05 Table 7

Treatment-Control Group Comparisons for Project PREP: Objectives Specific to the Alternative School

Alternative School Control Measure SD N M SD

Increase successful transition to high school or career

Worked for pay last week .59 .51 17 .50 .53 8 Hold full /part -time job .18 .53 17 .25 .46 17 Educational expectation 3.53 1.50 17 3.33 1.87 9 Practical knowledge 1.21 s'.34 11 1.34 .48 5

Increase belief in conventional rules

Belief scale .59 .26 11 .70 .22 5

Note. No significant differences between treatment and control group were found on any measures summarized in this table.

156

-283- .

Table 8 Responses on Project PREP'S Parent

Questionnaire Administered in Winter 1981 &

Spring 1982 for Alternative & Cluster Schools

Alternative School Cluster School

Item

Winter1981 Spring 1982 Winter 1981 Spring 1982

% responding % responding % responding % responding

favorablya favorablyb favorablyc favorablyd

Attended one or more parent association meetings 24% 67% 44% 72%

Aware of reading and math clinic 53 100 59 82

Aware of rec. centers 47 100 29 78

Aware of night centers 29 100 21 58

.L,

Aware of community agencies 18 33; 15 53

Visited school one or more times 65 67 71 88

Helped child with homework once/week or more 47 67 76 62 0 ,

School meets child's needs 76 100 78 82

Positive change in child's behavior 71 100 62 88

Positive change in child's attitude 65 100 53 90

Positive change in child's school atten. 71 100 60 82

Wants child to continue in PREP 76 100 75 93

Response rate: 39% 32% 44% 39%

Note: Respondents were parents who had children participating in Project PREP.

aN=17. bN=12. 157 cN=68. 158 dN=60. The Jazzmobile Alternative Arts Education Project

D. E. Rickert, Jr.

TheJazzmobile Alternative Arts Goals Education Project's aim was to uti lize the arts asa medium through The project's goals were, there whichjuveniles can constructively fore, to reduce truancy and chronic channel their energies. The project absenteeism, reduce youth involve was intended for juveniles in sixth, ment in "street activity," increase seventh, and eighth grades who parental awareness and involvement, showed disruptive behavior, who were and increase positive utilization of chronic absenteesand truants, or students' potential.. Table1 sum who experienced academic failure. marizesthesegoals andindicates how the first goal was measured. Problems, Goals and Theoretical Measures of the other goals were not Rationale identif ied.

4 Problems Theory and Objectives

The project's grant proposal The theory of action in the pro cited community crime and delin ject's Program Development Evalua quency, as well ashigh rates of tion (PDE) worksheets is less far school suspensions, truancy and reaching than that elaborated in the original grant proposal and in early . dropping outas problemson which the project wouldfocus. In its meetings between project and evalua proposed work plansthe following tion staff. The theory appearing in problems were to be addressed: theProgram Development worksheets appearsto have been a compromise 1. Youth hanging out on the resulting from staff discussion sub streets and being involved sequent to the first meeting between with what project staff called project staff and evaluation staff. "street activity." Students Several staff members mentioned that who hang outon thestreets such revision tookplace because rather than go to school were some of the wording of the original seenas morelikely to get theory sounded belligerent. For into trouble with the law. instance, the theory originally con tained a statement to the effect 2. Youth not having the opportu that the school systems were nity to live up to their full designed for middleclass white potential to be successfulin youthsand that the potential of life. minority children is not being real izedbecause parents do not use 3. Lack of parental awareness of theirpower to change the school their children's schoolre system. Key staff made it clear on lated activities and lack of severaloccasionsthatthe "offi involvement in these activi cial" action theory (i.e.,the one ties. elaborated in the PDE plan) was not theirs nor were they free to revise 4. Truancy and chronic absentee or elaborate on it. ism. The grantproposaldescribed a comprehensive ecological theoretical

285 159 Jazzmobile pespective. It stated that economic Interventions Intended impoverishment impinges on the fami lies of many youths in Central Har The art classes. The primary lem. Unemployment, poor housing, intervention planned was the Jazzmo community crime, health problems bile Art classes. Most students in associated with poverty, andpoor the program were to attend daily schoolscontribute toschool fai classes in either drama, music, lure. Dropout and subsequent unem dance or visual arts during the ployment can then introduce a host regular school day. In the of other problems--namely crime, intermediate school,six artistin drug abuse, and alcoholism. structors were to teach four classes daily. An After School/Summer pro Figure 1 illustrates my interpre gram was planned to serve 60 tation of the relation between intermediate school students,who, (a) the theory described in the for one reason or another, could not grantproposal, (b) the theory as attend classesduringthe regular laid out in the project's PDE work school day. The design of the Jazz sheets, and (c) ideas discussed mobileAlternativeEducation Arts informally by project managers and classes was consistent with Jazzmo staff. Elements enclosedin boxes bile's theory of delinquency preven are those included in the PDE work tion: Attachment between students sheet. The central elementis un and artistinstructors would be tapped youth potential.As Figure 1 fostered by small class sizes (e.g. shows, youth potential may notbe less than 10 students per class) and realized for various reasons. But warm, caring artistinstructors who ifit is not, students become bored could relate to the students, espe with traditional schooling. This cially in terms of cultural experi leads to school failure, which leads ences. The artistinstructors were to frustration. Frustrated students to serve as conventional role models are more likelyto bechronically to the students. absent from school and hang out on the streets. This leads to delin The artistinstructorswere to quency. make an effort to provide new exper iences daily for students and Figure 2 is an abbreviation of encourage creative expression to Figure 1 that illustrates the goals keep students excited and stimu-- and objectives of the project as lated. Black cultural history was laid out in the PDE worksheet. The to be incorporated into classes as elements enclosed in boxes were con seemed appropriate to the individual sidered goals by the project. That artistinstructors. Performances is, they were regarded as valuable and shows were seen by Jazzmobile in their own right, and are not sim staff as alternative rewards (Li ply instrumental objectives. grades) to help make 'schoola more desirable place for-students. Other Objectives alternative rewards planned were Jazzmobile teeshirts, certificates Table -2 shows the ojectives of participation, and artrelated (i.e., intermediary outcomes) iden field trips. tified by project personnel and the corresponding measures. As was the Perhaps most importantly, case with goals, many ofthe pro artistinstructors were to increase ject's objectives were not measured the extent to which students value despite some initial plans to pro education by teaching discipline vide these measures. through the arts. Jazzmobile staff 286 160 Jazzmobile maintain that art requires a discip- primarily in the area of art, lined process that is transferable secondarily in other areas, and to to academic matters. reduce youths' sense of hopelessness. According to theoriginal plan artist-instructors were to enhance Parent Component the school staff's approaches to teaching. Conversations with Jazz- The parent component was intended mobile staff members imply that to promote increased attachment bet- intuition was to play a large part ween parents and their children in this enhancement effort and its through their involvement in educa- assessment. tional activities. Home visits by a Field Worker,and parent workshops Curriculum Guides given by the staffs of 'the project, the school, and the police precinct Jazzmobile intended to develop a were to be included in this compo- curriculum guide for each of the nent. fourareasfor whichthere was a Jazzmobile class: drama, music, Project Environment dance, and visual arts. These guides were intended to assist the Implementing Organization Jazzmobile artist-instructors in providing an artistic experience for Jazzmobilebegan in 1965as a students within the traditional aca- non-profit organization giving free demic learning process. Specifi- musical performances, principally of cally, the guides used the specific jazz. art form which each guide deals with asa vehicle for teaching selected Dr. William Taylor, better known content area, skills in language as "Billy" Taylor, the jazz pianist, arts, social studies, science and was a leader in the formation of the math. Harlem Cultural Council, which obtained aninitial $10,000 grant It is not clear what objectives from Ballantine Beer to put on free in the Program Development Plan the concerts in Harlem. Jazzmobile was curriculum guides sought to achieve, incorporated in 1966. Over the but presumablythese guides would, years, Jazzmobile has become through a "trickle- down" effect, increasingly involved in music and raise students' self-esteem and cultural education. The other fine interest in academic subjects. arts (drama, dance, and visual arts) are now included under Jazzmobile's Counseling purview.

A Jazzmobile counselor was The overarching mission of Jazz- assigned to the intermediate school. mobile is the preservation and pro- This counselor's missionwas to pagation of Jazz. This involves counsel students regarding careers presenting free musical performances in the arts andto intervene when to.the community, particularly areas behavior problems arose. The deemed to be culturally deprived. counselorwas also to act as a resource consultantandtake stu- Secondary aims are the following: dentson fieldtrips. The objec- (a) to stimulate residents, espe- tives of these activities were to ciallyyouth, intoa moreactive increaseyouth'scareer awareness, participation in musical perfor-

-287- Jazzmobile mances; (b) to demonstrate to have received ongoing supervision by students the importance of communi- the DO. In practice, there appears cation and cooperation through the to have been some ambiguityabout arts; (c) to bring students into how much authority the Project personal contact with artists they Director and Evaluation Coordinator can relate to and emulate; and hadto make decisions withoutthe (d) to provide for students a ED's approval. This arrangement may "built-in success factor," whereby have caused difficulty in executing through. the arts youngsters can evaluation-related activities experience success, develop their because of ambiguity about appropri- talents, channel their energies, ate channels for organizing the acquire self-discipline, develop work. The official organization good study habits, and develop chart shows no direct link between respect for their school as a vital the PD and the EC. part of their community. Directly under the Project Direc- Internal Organization tor were the project's Guidance Counselor, Curriculum Coordinator, The Jazzmobile Alternative Educa- FieldWorker, and the Artist-In- tion Arts Program was one of several structors. activitieswith which Jazzmobile, Inc. was involved,butduring its The Interorganizational Environment operation was asubstantial source of income for the umbrella agency. Jazzmobile, Inc.,is well known, locally and nationally, and has Figure 3 describes the organiza- informal links with other organizational structure of,Jazzmobile. tions, especially arts organizations Presiding over all Jazzmobile, Inc. in the Harlem community. These affairs is a volunteer Board of linkages include those with the Directors. An Executive Director localpoliceprecinct (i.e., Pre -. (ED) has sole responsibility for the cinct 28), the Upper West Side operation and fiscal management of Health Center, local banks, the Har- all Jazzmobile activities, including lem Urban DevelopmentCorporation, the Alternative Education Arts Pro- the Goddard CommunityCenter, the gram. The ED consultedwith an Dance Theatre of Harlem, Harlem Cul- advisory committee for the Alterna- tural Council, Upton Chamber of Com tive Education Arts Program, which merce, Children's Art Council,and held monthly meetings. Studio Museum of Harlem. Represen- tatives ofsome ofthese organiza- Jazzmobile's Development Officer tions serve on either the Jazzmobile (DO) is directly under the Executive Board of Directors or served on the Director (ED) in the organizational Alternative Education ArtsProgram structure. The DO authored the Advisory Committee. Jazzmobile's alternative education project's President wasa Council Memberof grant proposal, and served under the the National Council for the Arts, a the ED as "Education Specialist" on Presidential appointment. He was theproject. Thelatter involved also President of the Arts and Busi- consulting as needed with the ED on ness Council of New York City. The "educational matters." membership of this organization includes representatives of 60 Also officially directly under national and multi-national corpora- the ED were the ProjectDirector tions (e.g., Exxon, AT&T, Mobil) and (PD) and Evaluation Co ;rdinator 60arts organizations. Jazzmobile (EC). Informally, they appear to

-288- Jazzmobile also has longstanding links with the The Community Community School Board in the target schooldistrict. These links have The community targeted for ser been an asset in Jazzmobile's previ vices by the Jazzmobile Alternative ous arts and cultural projects. Education Arts Program is called Central Harlem. The population of With regard to theAlternative CentralHarlem is almost entirely Education Arts Program, and the black. Reportedly, the unemployment evaluation of the program, two cri rate for black youths in Central tical links were weak: (a) that Harlem is about 60%, while it is 18% with the principal of the project's foradults (over 21). Theincome primary target school, and (b) links levelin Harlem is low: 1979 sta with parents and other residents in tist4cs indicate that 50% of fami the area of the school. lies were earning lessthan $5,000 (grant proposal, p.13) and a large Staffing and Resources percentage of Harlem residents receive some type of public assis The Jazzmobile Alternative Educa tance (e.g., welfare). tion Arts project consisted of the following staff positions: Local health care facilities are Executive Director/Project Man described as inadequate. Infant ager mortality is high. The housing Project Director stock is reportedly declining by 9 ArtistInstructors -- School 3000 units per year due to arson and Day Component abandonment (New York Times, 1978). 4 ArtistInstructors After The specific target area as School/Saturday and Summer Com described by Jazzmobile staff is ponents even worsethan Central Harlem at 1 Evaluation Coordinator large. Crime and delinquency rates 1 Guidance Counselor are amongthe highestin New York 1 Field Worker City. Police data indicate that the 1 Curriculum Coordinator precinct within which the project 1 ClerkTypist operates ranked 11th out of 73 pre 1 Accountant cincts for detention and arrests of 1 Secretary youth between the ages of 7 and 20. 1 Audiovisual Production Special ist The Schools

Job descriptions of the above Three schools in the school dis staff were provided by the project trict were targeted by the Alterna and are included in Appendix A. tive EducationArts Program: An Table 3 summarizes project staffing intermediate school and two elemen by primary work location and educa tary schools. The intermediate tional background. As of June 1979, school wasthe primary target, RS 15.11 fulltime equivalent (FTE) interventions in the elementary workers were employed bythe pro schools were minimal. The elemen ject. Of this, 7.00 FTE workers tary schools are two of four feeder were primarily stationed at the schools for the intermediate school. Jazzmobile office and 8.11 FTE work According to Jazzmobile administra ers were primarily stationed in the tors the intermediate school was schools. Of the 35 people ever chosen by the Community School Board employed by the project, 11 (31%) because of the serious problems with had left the project's staff by June which it is plagued. 1982. 289

163 Jaarambile

Mister, of the Project reported a reduction in drug-related problems and absenteeism, and In 1V7A, Immobile was awarded a parents reportedly became highly grant under the Emergency School Aid involved in the program (Justice Act (MA) Title VII to implement an Assistance News, April 1980). Arts Education Enricbuont Program in smother school district in East Har- Planning for the Alternative Edu- lem. The iciest was to lend support cation Arts Program began in early is the district's desegregation plan March, 1980. A meeting was held on by attempting to ameliorate the March 3, in which Advisory Committee adverse effectsof minority group members fromthe Community School iselatiom. Jazzmobile received District, key personnel from the *mother IESAA grant in 1977 to work School District, police, and commu- is Community School District 3 nity representatives met with Jazz- (Upper Vest Side) and A (East Har- mobilestaff. On March 29th, an lem),Itsobattan. The organization abstract of the proposed program by received still another ESAA grant'in Jazzmobile was submitted for review,

1971 to continue its activities. 111discussion, and approval by members of the Community School Board, Dis- is 1971. Jassmobile become inter- trict administrators, and community feted in delinquency prevention and and business leaders. The proposal diversionfor "first time offen- was approved. Thetarget schools ders." It was awarded a grant from were apparently decided upon at the the Law Enforcement Assistance first planning meeting. Admiaistration MAO to initiate prevention and diversion program in Between the eime- theCoimunity an elomestary school in the district SchoolBoard approved the program wherethisprojectnowoperates. and when the program began opera- Theproject conducted duringthe tion, there was an election and the 1t71'79 school year was called Young Board composition changed. The new Mises* of Modem Art Program, or Board fired the Superintendent and CRAP. There were four basic inter- installed an acting Superintendent. mac* compooentss An in-school The actingsuperintendentdidnot arts education enrichment program, achieve permanent status until an afteriwechool and Saturday program February, 1982. for first-time offenders, workshops for permits, and a summer program. The principals of the target Studeote from grades 3 to 6 partici- schools were apparently toldthat poled. Like the Alternative Educa- the Alternative Education Arts Pro- tion Arts program, major emphasis gram would operate in their schools was placed on helping children to at the beginning of the 1980-81 "enperieece success by channeling school year. The principal ofthe their talents and energies into the intermediate school was reported to acts"' (ANJOU. AlUattAllfit. MAU. have resented the imposition of the April DSO. Police Precinct 28 program. As a result, the principal participated byassisting in con - merely tolerated the Jazzmobile pro- ducting arts workshops and exhibit- ject in her school. The evaluation lift students' art work in the police Of the project fared worse, as the The statics. Students gave performances principal was opposed to it. sod displayed art work at hapsitals basicissue seems to have been a sad churches. complete lack of trust in evaluators who are "outsiders." For a period An evaluation of Project CHAP

-290- 164 1 Jazzmobile of time,this distrust extended to its program in the primary target even the Evaluation Coordinator, a site. Many ofthese problems seem Jazzmobile employee. to have arisen from Jazzmobile's own poor relationship with the princi- The principal's distrust may have pal. These problems were probably beenfostered by accounts of the aggravated byattemptsto evaluate evaluation of the earlier CHAP pro- the program. ject. An article about CHAP appearedin the Justice Assistance Jazzmobile had problems with News (April, 1980). It was entitled scheduling and even finding rooms to "Jazzmobile: A Ray of Hope in the conduct classes. Also the under- Crime-Ridden Streets of Central Har- qualification of some of the Jazzmo- lem." The first sentence of the bilestaff fostered resentment of article read: "The deck is stacked them by regular teaching staff. againt the kids who attend New York's Public School 113." In the The Jazzmobile counselor employed body ofthe article Jazzmobile was during the first year was unable to described as about the only positive establish cooperative arrangements thingthat children inthe school with the regular schOol guidance had going for them. This was seen counselor and was denied access to as a personal affront to the princi- students'school records due to a pal of the school as well as being lack of proper credentials. During insensitive to the positive aspects the second year she was replaced by of Central Harlem. another counselor.

Because the Community School The Jazzmobile field worker had Board in power when Jazzmobile began problems getting the cooperation of operation was not the same one that parents to participate in project helped design the program, there was activities. This lack ofcoopera- little support from Board members in tion may have been related to overcoming the obstaclesthat the attempts to evaluate the project. principal presented. The PTA president opposed an outside evaluation andmayhavegenerated What Was Implemented? some distrust of the project by parents. The First Year Despitethe foregoing problems, Basically, we know little about the project was able to implement what was implemented by Jazzmobile its program in some fashion: during its first year. We do know classes were held, some field trips that classes were held, and that were carried out,and some perfor- artist-instructors were present in mances took place. the school. Access to all data was denied by the school. We were The Second Year advised not to visit the intermediate school during the project's Bythe beginning of thesecond first year of operation, ald little year agreement was reached between monitoring ofthe implementation of the evaluators and key project staff the project's activities was possi- that they would make a serious ble. effort to document plans to overcome obstacles in theirforcefieldand During the first year of opera- document implementation through the tion theJazzmobileprojectfaced Program Development worksheets. numberous obstacles to implementing

-291- 165 Jazzmobile

This' agreementwasmadeduring a this orientation. Areas covered conference in August, 1981. were thefollowing: (a) Personnel matters (sick days, vacation, etc.), In conversations with Jazzmobile (b) system change strategies, staff member:, it becameapparent (c) rationale of data collection, that intuition rather than data was (d) the nationalscope and demons tobe thetechniquefor measuring tration nature ofthe "Delinquency the degree of implementation. We do Prevention Through Alternative Edu not know whether artistinstructors cation" initiative,(e) the Program were "warm and caring," how reward Development Evaluation processand ing the artshows and field trips (f) long and short range goals of were, orto what extent artistin the project weekly unit plans. structors worked with staff teachers to enhance their approaches to For the second year-,a require teaching. Project personnel ment was implemented that artistin resisted the quantification of these structors submit "unit plans" on a implementation standards. regular basis. These plans were supposed to describe what artistin t, Shortly after the August, 1981, structors planned to do andwhy. conference, the project director They were intended for quality con resigned. The Development Officer trol or monitoring purposes. of Jazzmobile stated that this had little impact on the program, as she After the initial orientation, was quickly replaced by a project artistinstructors in the day pro director who was a social worker gram were required to attend weekly rather than an artist. staff meetings with the PrOjEct Director, during which additional Training of ArtistInstructors training took place. Topics covered throughout the year included: A problem during the first year weekly unit plans, guidance con of program operation was that Jazz cerns, special events scheduling and mobile management had little know planning (e.g. art shows, field ledge of what the artistinstructors trips, etc.), curriculum, student were actually doing in their classes council activities, protocol for and why. It became apparent at the interaction with parents, protocol endof the yearthatsome of the for interaction andsharing ideas artist instructors were not doing a with regular staff teachers, and good job. Also, many of the student placement concerns (in sec instructors were apparently not ondary schools). aware that they were employed by a demonstration project whose primary When the after school and Satur purpose was to prevent delinquency. day program started in November, 1981, the participating artistin To ameliorate these problems, structors met twice weekly with the more attention was given to staff Project Director. Training focused training during the secondyear. on adolescent development, behavior Training began with a formal orien modification techniques, special tation week in September, 1981. needs of gifted and talented stu Noel Day of PolarisResearch and dents and referral of students for Development (the technical assis special high school placements. tance contractor) participated in

292 166 Jazzmobile

Diplomacy: Meetings with Key Actors Jazzmobile Evaluation Coor- in the Forcefield dinator (asacting project director), the Jazzmobile During the first year, many prob- Curriculum Coordinator and lems arosethatstrained relations the newly assigned Community between Jazzmobile and the adminis- School District 3 liaison trators of the target schools, par- met with the assistant prin- ticularly the intermediateschool. cipal at the intermediate During thesecond year,Jazzmobile school to schedule Jazzmo- made some effort to anticipate prob- bile classes into the school lemsbefore theyoccurred and to curriculum. approach key actors in order to overcomeobstacles. Many meetings o Early November, 1981: The occurred during the 1981-82 school newly hired Jazzmobile year,but the following appear to Counselor held an initial have been most significant: meeting with the school staff counselor to introduce o Late August, 1981; The Exe- himself and explain his role cutive Director and Evalua- in the Jazzmobileprogram. tion Coordinatormet with Reportedly, the new counse- Dr. Jerry Evans,President lorwasable to strike a of the Community School working relationship with Board - District 3. The the school counselor and meeting concerned matters theyheld regular meetings affecting implementation of thereafter. Unlike the pre- the second year program. As vious Jazzmobile Counselor, aresult of this meeting, the new counselor was theCommunity School Board granted access to student assigned astaff member to files. act as liaison between Jazz- mobile and itself. o L cember 2, 1981: Jazzmo- bile Executive Director, o Sept., 1981: The Jazzmobile Evaluation .Coordinator and Evaluation Coordinatormet new Project Director met with the principal of the with the Community School intermediate school in an Board todiscuss data col- attempt tomakepeace, so lection issues. At that that constructive conversa- meeting, the Board decided tion regarding theevalua- (after 1 1/2 years) that tion could take place. Jazzmobile would need affir- Reports indicate that the mative parental permission meeting was cordial, but it for collection of data from did not lead to data being school records. released for evaluation purposes. A decision was made o December 18,1981: Another by Jazzmobile to wait until Community School Board meet- the principal retired in ing was held so that the November, 1981 beforedata public could respond to the collection would be data collection issues. At attempted. (The principal this time the previously did not actually retire passed resolution that Jazz- until February, 1982). mobile must obtain parental permission was upheld. o September 17, 1981: -293- 167 Jazzmobile

o January 31, 1982: Project November, 1981). The brochure gave director met with school a job description of each staff mem

personnel to coordinate sec ber and listed 11__e,r-1 career accom ond semester of the Jazzmo plishments. The point was to estab bile Alternative Education lishthe staff members as trained Arts Program in order to professional educators. The bro adjust to class shifts as a chure was distributed widely. result of the semester change. Another activity which Jazzmobile engaged in heavily during the second o March 22, 1982: Jazzmobile year was what staff called "creative Executive Director,Project documentation" of program implemen Director and new Evaluation tation. Coordinator met with the Community SchoolBoard and By midOctober,staff had pre school administrators to pared aslide presentation of year discuss objections that the one activities for showing at parent school personnel had to the meetings and workshops with regular SAES survey. The evaluation school staff. By the end of Novem- staff's principal investiga ber, 1981, a presentation was ready. tor and another staff member A video tape of classroom activities f also attended that meeting. was ready by January, 1982. These TheCommunitySchoolBoard slide and video presentations were decided that the selfreport used for training purposesduring delinquency section of the weekly staff meetings_ and were shown student survey must be at parent andteacb-erWErksh4s: deleted in order for it to be administered. Jazzmobile also organized an alternative education resource book Public Relations case, published one issue of a news letter, and helped the intermediate The Jazzmobile Alternative Educa school develop a publicrelations tion Arts Program suffered from a booklet and a yearbook. lack of credibility during its first year. Schoolstaff reportedly saw The Art Classes the arts as an unnecessary "frill." The artist instructors did not Most sixth graders at the two receive the respect from other elementaryschools participated in teachers they felt they deserved, arts instruction provided by Jazzmo and this made it harder to do their bile staff. In the intermediate jobs. school, most (except those retained) who participated as 7th graders dur During the second year Jazzmobile ing the 1980-81 schoolyear were devotee manyresources to showing continued in the program as 8th that Jazzmobile did provide a worth graders. New 7th graders were while service and that it knew what selected in cooperation with it was doing. In the case of intermediate school staff according Project Director and Counselor, this t-D scheduling convenience and per involved hiring peoplewithmore haps other unknown criteria. Sche impressive credentials. dules for participating students were completed by midSeptember, During the second year, a Jazzmo 1981. Classes began September 24, bile Alternative Education Arts Pro 1981, at the two elementary schoOls gram brochure was issued (late 294 168 Jazzmobile and September 28, 1981, at the classes except to say that the intermediate school. While the sem- artist-instructors taught arts. It ester was inprogress 33 students is apparent that much more actually were lost fromthe program inthe took place but this was of an ad hoc intermediate school, owing to a Pro- nature and depended greatly on the motional Gates Program whereby stu- styleof the individual artist-in- dents more than two years behind in structor. Several artistic accounts readingor math wererequired to of instructors' style have been pro- receive remediation. vided by the project and are included in Appendix B. Six artist-instructors worked in the intermediate school-day program Other Activities and three artist-instructors in the elementary schools. Students in the Besides time in Jazzmobile elementary schools attendedtwo to classes, target students partici- three classes weekly. Participating pated ina number of out-of-class students at the intermediate school activities such as field trips, art attended Jazzmobile classes daily. shows and performances.

During October, 1981, four Field trips. Attempts to conduct artist-instructors were hired to field trips duringthe first year teach after-school and Saturday were ad hoc. Lessons were learned classes. Many of the after-school concerning the steps necessary in students were referred by Police the planning of field trips: precinct 28 or the intermediate Licensed school staff must be pre- school as chronic absentees and sent as chaperones' (this requires "first time offenders." Classes finding cooperative staff), and started on November 27, 1981 with 45 parental consentforms are neces- students. Classes met two school sary. days a week from 3:15 - 5:00 p.m. and on Saturdays from 9:00 - noon. A calendar for fieldtrips was put together by the Jazzmobile By the third quarter of the Counselor for the second year. The school year 60 students were in the counselor also compiled a list of afternoon and Saturday program. students who would be participating. Students in the afternoon and Satur- With this information in hand, the day program were given the opportu- counselor and project director were nity to continue during the summer able to make arrangements with months (i.e. July 1, 1982-August 27, parents and school staff. 1982). Summer classesmet daily from 9:00 a.m.- 3:00 p.m. Seven Althoughanexplanation of how Artist/Instructors served 84 stu- each field trip fit into individual dents. Classes were held at one of students'treatment plans wasnot the primary schools. Eight interns available to the evaluators, an from the New York City Summer Youth effort was reportedly made to Employment division ofthe Depart- arrange field trips that were a ment of Employment helped out. meaningfulsupplement to each stu- Arrangements were made for students' dent's program. For instance, stu- lunches through the Free Lunch Pro- dents who were interested in attend- gram. ing city-wide arts high schools made field trips to such places. In all, Itis difficult to capture what 31 field trips were conducted, with went on in the Jazzmobile Art students going to the museum,

-295- 169 Jazzmobile library,movies, plays,monuments, Arts. Three graduating students and parks. from the Intermediate School received the Chancellor Memorial Films were used during the second Award, a citywide recognition for year for the purpose of reinforcing musical proficiency. concepts covered in Jazzmobile classes. Someof the filmsdealt Jazzmobile participated in the with the arts and some with AfroAm graduations held at the target erican history. Eighteen different intermediate schooland two feeder films about such topics as Paul elementary schools. Jazzmobile Spe Robison, the Harlem Globetrotters, cial Art Awards were given to a num Adam Clayton Powell, Roy Wilkens, ber of graduating students. In and sex education) were shown. addition all students received cer tificates of merit. Productions and art shows. Stu dents in the Jazzmobile classes were Parent Component/Fieldwork "rewarded" by having the opportunity to display theirtalents. Toward The Jazzmobile Alternative Educa thisend, 10 productions and arts tion Arts Program emphasized parent shows were held. Failure to obtain involvement in their children's pro proper permits halted plans for a gram activities. Many parent work street festival at which Jazzmobile shops were held throughout the sec students were to perform. ond year. In these workshops, parents were told about the artre Special events. Several special, lated activities that their children events were carried out as part of learned during the day. Videotape the Jazzmobile program. These and --';de presentations of perfor included performances by the Nanette manc's lown et these meetings. Beardon Dance Company and a lecture The Jazzmobile FieldWorker was by Jazz musician Jimmy Owens. active at soliciting parental parti c'Lation. Flyerswere sent home Rewards wi_h students, phone calls and home visits were made, and appeals were All Jazzmobile target students mace to the respective Parent Asso received a Jazzmobile teeshirt and ciations. one occasion a health a certificate of participation. fair was 'eld to attract the atten Students involvedin final produc ticn of,arents. Also, in October, tions had cast parties. 19E Jazzmobile staff held evening hours during "OpenSchool Week" at A Jazzmobile Youth Councilwas allthree schools so that parent established at the intermediate who could not attend workshops could school. Meetings ofelected offi at least meetthe artistinstruc cers began inDecember, 1981,and tors. Narratives about the progress took place weekly thereafter. One of the program were mailer' 'n issue of a Newsletter was published parents in December, 1981. by the council in January, 1982. The newsletter coveredinformation A total of 74 parent workshops aboutthe activities ofJazzmobile were held between September,19L1, participants, as well as articles of and June,1982. Each workshop had interest to the general student between 30 and 35 participants. All body. Seven Jazzmobile participants told, over 105 parents weta consid received scholarships to dance ered regular participants. Some classes at Harlem School of the parents were enlisted to help carry

296 170 Jazzmobile out some ofthe productions put on Evaluation of the Project by Jazzmobile. No evaluation design was Counseling Component describe in the project's grant proposal. Subsequent interviews The Jazzmobile management hired with project staff members indicated an experienced and properly credent that they expected a simple prepost ialedcounselor in October, 1981. summative evaluation involving read The new counselor was better able to ily available aggregate data with no implement the program in the school control group. Jazzmobile had been because he held a degree (i.e. MSW) exposed tothis type ofevaluation and credentials that were respected in its previous youth programs by regular school personnel. He was (e.g., ESAA and Project CHAP). Sev allowed access to confidential eral Jazzmobile staff members sug records and was effective in soli gestedthat they would not have citing the cooperationof regular accepted the grant had they known of teachers for such 'things as chaper the evaluation requirements. oning field trips. The rigorous evaluation--not Besides counseling studentsfor anticipated by Jazzmobile or the "behavioral problems" on an "as school system-- created problems for needed" basis, he engaged some stu Jazzmobile. The building adminis dents in career counseling. He trator of the primary target school orchestrated the previously men was vehementlyopposedtothe pro tioned field trips to artshigh posed evaluation activities, and schools and prepared an arts career this may have created difficulties directory. The new Jazzmobile in implementing other aspects ofthe Counselor established a referral project. A fair summary of evalua procedure for onetoone counseling tionrelated activities is that they and organized a case record system. were unproductive. The Jazzmobile He designed special referralforms project is a good example of a pro and followed a set appointment sche ject thatis not amenable to rigo dule. rousevaluation. Jazzmobilestaff did notsee the value ofrigorous Curriculum Guides evaluation. Evaluators were told time and timeagain by keystaff During the life of the Jazzmobile members that the Jazzmobile project project, two curriculum guides were needed a more "creative" evaluation. completed and a third started. The The factis, however,that Jazzmo first(drama) was completed atthe bile was funded as a delinquency end ofthe first year. Little is prevention demonstration project and known about how this guide was actu needed to be evaluated as such, in a ally used except that there was a rigorous manner. This.end was never formal presentation of the guide to accomplished. Whatfollows is an schooladministrators inSeptember account ofefforts to evaluate the and October, 1981. A second (dance) Jazzmobile project. curriculum guide was developed dur ing the second school year in coop The- First Year eration with the Jazzmobiledance instructors and some regular school Because the Jazzmobile project teachers. Training workshops for had already begun operation when regular schoolteachers in the use evaluation efforts began, the evalu of the dance guide were held in late ation design was a nonequivalent May, 1982. 297 171 Jazzmobile

control group comparison (i.e. Year Two Jazzmobile target students vs. the rest of the students in the After attempts to create a stron- intermediate school). Evaluations ger design for the second year of the after-school component and of failed,the design again defaulted the program in the elementary to a non-equivalentcontrol group schools were never accomplished comparison. The original assignment because problems with the evaluation to treatment and control conditions of the major component--the was based on scheduling convenience intermediate school operation--de- for the school. Entire classes were voured the entire evaluation effort. designated as either Jazzmobile or non-Jazzmobile classes. Then after Although assignment to Jazzmobile theschool year began, the school classes for the 1980-81 school year received the students' reading (CAT) was not random,the assignment was scores from the previous school fairly haphazard, based on schedul- year. Those who did not "pass" were ing convenience. On all variables reassigned to remediation programs, for which data were provided, the and they were not allowed to parti- treatment and non-equivalent control cipate in the Jazzmobile classes. group look similar. Differences between groups on age, sex and race Very late in the school year, the are non-significant. Nevertheless, CommunitySchool Boarddecided to any attempt to interpret first year release data only for students for results must bedone with caution whom affirmative parental permission due to potential bias of an unknown had beenobtained. As expected, nature owing to the following: affirmative permission response rate (a) only students for wham parental was low, 64%(116)for Jazzmobile permission was obtained were allowed participants, and 39% (67) for non- to besurveyed, (b) he difference participants (i.e. non-equivalent in non-response rate to items on the controls). The extent ofthe bias survey between treatment and non-e- introduced into the evaluation quivalent controls was highly signi- design by the shuffling of students ficant (p<.001). Non-response to basedon CATtestscores and the questionnaire items was far greater decision regarding parental permis- for Jazzmobile treatment. students sion remains unknown. than for non-equivalent control students. The SAES Student Survey, with delinquency items deleted, was The only data available for the administered in May, 1982 to treat- first year cohort aresurvey mea- ment and non-equivalent controls for sures--no data from recordswere whom affirmative parental permission ever released to the evaluators. had been obtained in the intermediate school. 74% (135) of this Treatment-control differences on select group responded to the sur- all measured outcome variables were vey. The 135 students for whom Stu- non-significant, suggesting that the dent Questionnaire data were availa- program made little difference for ble represent 38% of the total these outcomes. See Table 4. sample in the intermediate school (N=351). Noattemptwas made to surveythe 214 seventh graders in the two feeder elementary schools.

-298- 172 Jazzmobile

After the end of the school year, control groups appear small relative California Achievement Test scores, to the standard deviations. No attendance and promotion data from pre-treatment measures usable as records were provided for the treat- covariates for attachment to school ment and control students for whom were available, so it was impossible affirmative parental permission had to attempt to adjust for potential been obtained. Data on withdrawals pre-existing group differencesfor were provided forall studentsin this outcome. For student employ- the intermediateschool. Whatever ment, grade level was correlated bias was introduced, thetreatment both with treatment status and and control students allowed to par- employment status. Using grade ticipatelooked remarkablysimilar level as a covariate does not alter on background variables. There were the statistical conclusion. no statistically discernable differences between treatment and controls Measures from school records. on participation in specialeduca- The project provided data on with- tion (t=1.02, p<.31), sex (chi drawals from school (for reasons squared =2.66, dF=1, p<.10), self-re- other than transfer) for allstu- ported parental emphasis on dents in the sample. Other data education (t=-.13, p<.90), self-re- (standardized achievement test ported parentaleducation (t=-.53, scores, attendance, and promotion) p<.59), and pre-treatment standard- were provided only for students ized achievement test (CAT) reading whose, parentsconsented, andonly (t=-.68, p<.50) andmath (t=.91, for students still in school at p<.37) scores. There was no signi- year's end._ ficant difference in racial composition between the treatment and non- Nosignificant differences were equivalentcontrolgroups (t=1.42, found between treatment and control p<.12). The treatment group did groups on math or reading achieve- differfrom the control groupon ment, or on school attendance. grade level. The non-equivalent Furthermore, the differences control group was split about observed appear small relative to equally between 7th and 8th graders. the standard deviations. (See Table The treatment group was weighted 6.) heavily with8th graders (t=3.44, p<.01). A larger proportion of treatment than non-equivalent control group Questionnaire measures. The only students were promoted to the next significantdifferences found bet- grade (93% vs. 81%), and this dif- ween treatment and non-equivalent ference was significant at the p<.05 control groups were on the measures level. But the treatment and non-e- of Attachment to School and Student quivalent control groups also dif- Employment. Table 5 shows that the fered (p<.01) on previous promotion treatment group was more attached to history and the control group con- theschool than thecontrolgroup tained more special education stu- (<.05), and members ofthe control dents. When previous retentions and group more often reported holding a special education status were used a regular part-time or full-time job. covariates(to statistically "con- If the 24 measures reported in Table trol" for these pre-existing group 5 were independent, one would expect differences) no significanteffect to find one difference atthe .05 of treatment or promotions was found level by chance alone. Among the (p=.78). non-significant comparisons, the differences between treatment and

-299- 173 Jazzmobile

Jazzmobile participants were less all students). These limitations likelyto withdraw from school for made statisticalcontrols forpre- reasons other than transfer than existing differences difficult or were non-equivalent control group impossible to apply. Biases of an students (5% vs 13%). No pre-treat- unknown naturemay influence the mentor other data were available pattern of results. Finally, no for students who withdrew, making an direct measures of delinquent behav- attempt to applystatistical con- ior (the primary outcome of interest trols impossible. The observed sig- to the project's sponsors) was avai- nificant difference therefore may be lable in the second year of project a treatment effect, or it may operation. reflect pre-existing differences between the two groups. We have no Despite these limitations, itis wayto disentangle these competing possible to say with -considerable possibilities. assurancethat if theproject had any sizable effects on any of the Discussion outcomes studied, such affects would be detectable with the design used. 11 Limitations of the Evaluation This is so primarily because little difference was found between treat- This evaluation, which was con- mentandcontrol students onthe ducted under difficult circums- background characteristics that were tances,has some important limita- available for study, and because the tions. In the interest of clarity, sample sizes are sufficiently large. the limitations themselves are described here. Later, some of the Limitations of the Project factors contributing to these limitations will be discussed. A number of influences in the project's environment created limi- This has been a quasi-experimen- tations forthe implementation and tal evaluation of an intervention evaluation of the project. On sev- which, given the range of interven- eralpointsthe project's analysis tions possible in schools, was not of its forcefield appears tohave particularly strong on an a priori been incomplete,leading to subse- basis. The non-equivalentcontrol quent problems. For example,that group design used is imperfect Jazzmobile staff working in the because, despite some evidence that school might be considered unquali- treatment and control students were fied due to a lack of conventional not grossly different on available educational credentials was not measures of student background char- anticipated. The resulting non-ac- acteristics, the treatment and con- ceptance of staff in the school, and trol groups may havediffered in the influence that the perception of unknown ways. Furthermore, data non-acceptance had fortheday-to- were generallyavailableonly for day work ofthe artist-instructors students whose parents gave affirma- and the counselor, was not therefore tive consent for the release of anticipated. data, and response rates to the surveys were not perfect. And, most A second example involves diffi- information aboutstudentoutcomes culties in implementing the evalua- was available only for students who tion component. The project tended did not withdraw during the school to focus on the immediate obstacle year (the exceptions is student of the intermediate 'schoolprinci- withdrawal, which was available for pal's reluctance to participatein

-300- 174 Jazzmobile the evaluation, and planned to move the project appears to have hada ahead with data collection after the reasonably strong program of parent principal had retired (an eventual- workshops and a full time community itythat was considered imminent). field worker. No plans were made foran alternative strategy to obtain data should Principal Support and the Level of the principal not retire, nor were Delinquent Behavior strategies developedto move .head with other aspects of evaluation As noted earlier,one potential activities when the initial obstacle explanation of the intermediate was overcome. school principal's resistance to the evaluation activities was the por- The Puzzle of Community Support trait of East Harlem contained in an evaluationreport for an earlier Project managers viewed a lack of Jazzmobile project. That report has parental concern with and commitment did theJazzmobilegrantproposal to education as a problem, as'is for the present project) character- illustratedby their inclusion of ized youth in the community as high these parental characteristics in in delinquency and from disorganized their theory statement. Indeed, one family and community situations. We component of their intervention was are somewhat puzzled by the paradox- the series of parent workshops ical lack of agreement between some described earlier. Yetthe commu- common stereotypes of EastHarlem nity was not successfully mobilized and the Jazzmobile grant proposal's to support the evaluation of the description on the one hand, and the project. This lack of community level of self-reported delinquency support for evaluation is illus- reflected in the 1981 student survey trated by an important meeting results on the other. The level of called by the chairman of the school self-reported delinquent behavior in board to discussevaluationplans our survey was not especially high. with the Jazzmobile and national Although survey non-response in 1981 evaluation staff. This meeting, was high, implying that substantial which was toseek community input biases may exist in the results, the for a schoolboard decision about results do not support an impression the spring 1982 student survey, was of an extremely delinquent popula- attended by relatively few parents tion in the intermediate school. We of children in the intermediate see at least two potential explana- school wherethe survey was tobe tions for this paradox. First, performed, and was dominated by either response bias or invalid vocal individuals opposed to the reporting may have produced mislead- survey. An analysis ofthe force- inglylowestimatesof delinquent field would have suggested the behavior among members of this popu- advisability ofensurir.gthat sup- lation. Second, the degree of indi- porters of the Jazzmobileproject vidual delinquent behavior among the attendthis meeting in large num- youths of this community may be bers. That meeting led to the exaggeratedin popularstereotypes school board's censorL;.,!, self-re- of the community. Whichever of ported delinquency measurer.' from the these possibilities is nearer the survey administration. We are puzz- truth, clearly much remainsto be led by the failure of the implement- learned about the youth of this coming organization to mobilize suffi- munity. The difficultiesof con- cient community support for the ducting evaluation research in the project and its evaluation to over- community pose a challenge for come this obstacle, especially when

-301- 175 Jazzmobile responsible service providers and project, project managers discovered researchers of the future. a need for a structure for holding artist-instructors accountable for Protect Staffing, Personnel Plan- their activities in the schools. At ning, and Management thatpoint they providedtraining for staff on the goals of the proj- Severaldifficulties inproject ect, and with the ,change in project implementation appear to have stem- directors they began providing more med from problems of initial project supervision. This increase in staffing, personnel planning, and structured expectations, training, personnel management. and supervision may havecometoo late in the life of the project. Staffing and personnel planning. The success of most human service Effects of the Intervention endeavors depends heavily on the staffinvolved inthe delivery of Despitethe limitations in the services. One difficulty in the evaluation described earlier, some staffing for this project has ,reasonably confident conclusions are already been described: The lack of possible. The following paragraphs educational credentials necessary to describe our best summative judgment create an impression oflegitimacy about the effects of the Jazzmobile in the school. An additional diffi- Alternative Arts project. culty may have stemmed from a mis- match between the job demandsand Empirical evidence. On the basis personal preferences ortalents of of the empirical evidence, the proj- some staff. The artist-instructors ect had little effecton the stu- were, we assume, for the most part dents exposed to its interventions. individuals with interests and com- Statistically signficant effects petencies in various artistic favoring treatment students were endeavors. Their experience and found on attachment to school (lik- interest in the more conventional ing for school) and on dropout. The aspects of their jobs--developing size of these effects were moderate, curricula, keeping records, adhering ard confidence in their interpreta- to regular schedules, and so tion as actual effects of the alter- forth--may have been less than was native education project is weakened required bythe jobs. Our impres- by the quasi-experimental nature of sion is that a more careful analysis the designand the unavailability of the tasks required by the jobs of sufficient pre-intervention measures project director, instructor, and for thorough application ofappro- evaluation coordinator may have sug- priatestatisticalcontrols. Fur- gested the recruitment of a somewhat thermore, because Co many different different mix of staff. This outcomes were examined, one or more impression receives some support significant results were to be from the unsystematic reports we expected by chance alone. The proj- have received of job dissatisfaction ect doesnotappear, however, to on the part of some staff and from have done harm. The inly statisti- the relatively high turnover among cally significant result favoring project personnel. Both the project the control group was on a measure director and evaluation coordinator of student employment: More control resigned, and turnover among staff than treatment students reported overall exceeded 30%. full- or part-time employment.

Management. Midwaythrough the

-302- 17G Jazzmobile

The evaluation design was, powerful intervention to produce despite its limitations, suffi- much in the way of measurable ciently strong that had the project changes in studentbehaviors and produced large positive effects they attitudes, or (b) these philosophies should have been detected. We find and approaches were not fully imple- no evidenceof such effects in the mented on a day-to-day basis. The empirical evidence. empirical evidence does not allow us to chose among these two alternative Artistic evaluation. The imple- interpretations. menting organization did not anticipate an empirical evaluation of the Conclusion. The alternative arts kind we had hoped would be imple- education intervention as implemented (and of the kind that eventu- mented and studied here did not have ally was implemented in substantial a major influence on the psychoso- degree). Indeed, theorganization cial developmentof' youths in 'the indicated that ithad hoped for a community in which it operated. It more artisticevaluation andthat may have increased liking for had it known about the kind ..of school, prevented some dropout, and empirical evaluation expected it decreasedstudent employment, alt- would not have undertaken the pro- hough the evidenceis not ironclad ject in the first place.1 that these effects were produced by the intervention. No evidence sug- The descriptions of artist-in- gests that this intervention pre- structor approaches totheir work, vented delinquent behavior. included in an appendix to this report, are therefore of considera- Recommendations for Future Projects ble interest. These descriptions ------suggest that artist-instructors were The conduct and evaluation of sensitive, caring,individuals. We this project have been instructive, haveno way of knowing howaccu- and they suggest the following rately these descriptions character- recommendations for future delin- ize the workaday activities of these quency prevention projects: instructors, but we have no reason to doubt that they convey an accu- 1. Grant awards for demonstra- rate impression. An iAteresting tion projects should be made only to implication of thesedescriptions, organizations that understandand combined with the empirical evidence agree to fully implement a rigorous about project effects, is either evaluation, and where concrete evi- that (a) artist-instructors imple- dence of a commitment of relevant menting the kinds of personal philo- actors in the organization's force- sophies and approaches described in field to fully cooperate with the theappendix is an insufficiently evaluation is available.

2. Expectations forevaluation should be made concrete and explicit 1 Aletter from S. David Bailey in advance of requestingapplica- (dated 30 April 1983) commenting on tions for grant competitions. a draft of this report is available from Jazzmobile, 159 West 127th 3. Demonstration projects Street, New York, NY 10027, or from involving only arts education show Lois Hybl, Center for Social Organi- limited promise for delinquency pre- zation of Schools, The Johns Hopkins vention, and demonstration projects University, 3505 N. Charles St., proposing only to implement arts Baltimore, MD 21218.

-303- hi nob it educattoo *hook, be ovoided. This about what seems appealing. dee* ma moan that arts educaxion as part of a IWO temples and conceptu- 5. Prevention project implemen- ally otroager *et of interventions ters should carefully consider (and to to he Avoided, only that other should be provided with assistance throbs *tea equala more complex in considering) the potential obsta- and detenotble set ofinterventions cles to project implementation in are to be preferred to a replication the project's environment. of project* reoembling the one derottbed hero. 6. Prevention project implementers should carefully consider (and 4. improokoionislic evidence is should be provided with assistance no tutetitute to hard-nosed (spirt- in determining) the personnel Cal evaluation. The public deserves requirements oftheir interventions propose to prevent dolinquency and inselecting and training the which are demonstrably effective, requisite personnel. and projectsponooro should therefore a** decielons about project 7. Delinquencyprevention pro- tendtma on (a) assessments of evalu. jects should initially and continu- stability. and (b) the empirical and ally demonstrate A clear and direct Weeretical literature on delin- commitment to the prevention of del- 'wary, rather thaw upon Impressions inquency as a primary goal. Jazzmobile

References

In Last Decade, Leaders Say, Harlem's DrTams Have Died. New York Times, March, 1978, pp. 1,13.

Jazzmobile: A Ray of Hope in the Crime-ridden Streets of Central Harlem. Justice Assistance News, 1980, p. 6-7.

-305- 179 Figure 1

The Fully Elaborated Theory, Including Causal FactorsNot . Idressed by Program

Parents are Low parental Low parental

financially awareness about concern and

poor. school support for Survival is disciplinary school

main procedures activities. priority. and practices.

Schools do not Low student Low aspirations: Truancy,

provide youth career sense of chronic

with career awareness, hopelessness. absenteeism.

information. o Educational Diverse talents Failure Frustration 0 system denies unexercised in Black and unrewarded school.

accomplishments. in school.

Male students Education School not involvement

value athletics not valued stimulating. in street

but not by youths. activities.

academics,

Low student - Non - school

teacher environment

attachment, relatively more stimulating than school. V

Involvement Delinquvol with -7hvbvior. negative role models.

181 1 30 ..

Figure 2

Abbreviated Theory Including

Causal Factors the :!roject Hoped to Addms

Increase parental Increase parental 2Increase awareness about concern and attendance school disci- support for Increase - plinary procedures school activities, success 11

and practices. in school. \J Increase , / extent to Decrease

which involvement Increase students' Increase aspirations, diverse in street career awareness.----)decrease ----7talents activities. hopelessness. are used

and

rewarded Increase student- Increase extent 7 in school, Increase teacher to which sense that attachment. youths value school is education. stimulating. A 1

183 1S2 Figure 3, Organizational Arrangements

Alternate :e Arts Education Project

Jazzmobile

Board of

Directors Community and School President Board

\., wr Jazzmobile Executive Advice and consultation Acting Director

and Superindendent

Project Manager 0 op

a Development Officer

Evaluation Project Principal rimm owe W... Coordinator Director Intermediate c -7 PTA School '

Curriculum Arts Field

Coordinator Counselor Woikers 41.=1.1-0M.M

Artist Instructors 185 Table 1

Jazzmobile Alternative Arts Projects: Goals and Corresponding Measures for Which Data Were Actually Provided

Goal I'leasure ......

1. Reduce truancy and chronic a. Data on absences provided absenteeism by the school b. SAES Student Questionnaire Non-attendance Index

2. Reduce youth involvement None in "street" activity which promotes delinquent behavior

3. Youth potential will be None used in a positive way.

4. Increased parent awareness None and involvement

1 S 6

-309- Table 2

Jazzmobile Alternative Arts Project: Objuaives (i.e. Intermediary Outcorsqs) and Corresponding Measures

.. Objective Measure ..

1. Increased student-teacher Student-teachur attachments Interactiona

2. Student:: will become Attachment to excited and stimulated by in- Scboola school experiences

3. Increase youths' career None awareness, primarily in the area of art, Seco ndarily in other areas.

4. Increase students' success None in non-academic areas (i.e. the arts)

5. Increase parental awareness None concerning anhool policies, procedures and practices with regard to disciplinary actions and school referrals to the juvenile justice system

6. Increase Parents' concern Parental Empbaeis on and support for youths' school Education scaled activities

7. Youths previously unused None potentials will be tapped

8. Increased Youth value of Educational aepirationsa education

9. Reduce youths' sense of Alienation sOlea Alienationa

...... aThe project Personnel did not identify these tpeAsures; they seem to us to be relevant and were availaVik through the SAES questionnaire.

,310- 1 8 7 Table 3

Educational Background of Jazzmobile Alternative Arts Education Project as of June, 1982

Work location and education FTE Number

Jazzmobile officea 7.00 9 M.A 2 4 1 A.A 1 High school 2 Schools 8.11 15 M.A 2 B.A /11.S. 4 A.A 1 'Ugh school 8 Total 15.11 24

Note. Of the 35 people ever employed by the-project, 11 (31%) had left the project's staff by June, 1982 (Source: SAES quar terly staffing report).

aIncludz project manager, project director, curriculum spe cialist, accountant, audiovisual specialist, field worker, and secretaties.

311 188 Table 4

Jazzmobile Alternative Education Arts Project: Means and Standard Deviations for Spring 1981 Student Survey Outcomes for Students in Treatment and Non-Equivalent Control Groups

Group

Non-equivalent Treatment Control

Variable ti SD N M -SD N

Attachment to Parents __a __a Negative Peer Influence -.34 4.40 118 -.35 4.92 77 Attachment to School -.50 5.88 31 -.47 6.66 39 Negative Belief 2.68'1.32 20 2.05 1.13 34 Interpersonal Competency 4.00 1.29 19 3.99 .90 35

Positive Self-Concept -.60 4.39 18 .85 5.28 32 Punishment Index -.75 2.02 43 -.08 2.54 52 Rewards Index .14 3.69 41 .37 3.07 48 Victimization .95 1.48 37 .53 .82 50 School Effort 2.05 .76 162 2.00 .73 95 Practical Knowledge 9.17 5.78 20 9.68 5.47 40 Internal-External Control 2.80 .95 20 3.06 .76 35 Alienation 1.24 1.24 22 1.22 1.43 34 Self-Reported Grades 3.01 .83 162 3.11 .79 94 Reading Ability Self- 1.39 .88 145 1.68 .80 78 Reported Satisfaction with School 2.24 .78 167 2.16 .72 95 Progress School Nonattendance 1.04 1.65 166 1.11 1.72 88 Index

Rebellious Autonomy 2.05 1.10 20 2.29 .92 36 Involvement .09 6.58 102 -1.06 5.69 69 Total Delinquency 1.88 2.55 78 1.75 2.54 57 Serious Delinquency .74 1.49 81 .44 1.33 62 Drug Use .63 .98 81 .76 1.17 60

Note. None of the outcomes presented above show statistically significant differences between treatment and control students.

aThe questions comprising this scale did not appear on the Jazzmobile version of the 1981 Student Survey due to objections by the Community School Board.

-312- 189 Table 5

Jazzmobile Alternative Arts Project: Means and Standard Deviations for Spring, 1982 Student Survey Outcomes for Students in Treatment and Non-Equivalent Control Groups

Group

Non-equivalent Treatment Controls

Variable SD N M SD N

Attachment to Parents . .63 .26 81 .60 .25 42 Negative Peer Influence .16 .19 75 .16 .15 39

Attachment to School .81* 1, .18 72 .72 .23 39 Belief Scale Scored Positively .71 .22 42 .75 .24 33 Interpersonal Competency .85 .18 45 .76 .26 32 Positive Self-Concept .80 .14 46 .80 .18 31 Punishment Index .20 .22 77 .18 .26 38 Rewards Index .37 .32 77 .36 .31 38 Victimization .09 .15 77 .13 .20 38 School Effort .67 .26 _80._ .59__.26__38__ Practical Knowledge 1.31 .34 38 1.39 .29 30 Internal-External Control .44 .21 48 .41 .24 31 Alienation .27 .22 48 .30 .30 33 Self-Reported Grades 3.18 .67 88 3.19 .92 42 Reading Ability, Self Rating 1.56 .65 79 1.74 .92 38 Days of School Cut in 4 Weeks Preceding Survey .10 .30 89 .21 .68 42 How Often Cut One or More Classes .39 .73 89 .25 .62 44 Educational Expectation 3.99 1.35 87 3.53 1.80 45 Did You Work for Pay Last Week? .46 .50 86 .65 .48 40 Regular Part-Time or Full-Time Job? .11* .31 84 .30 .56 43 Suspended from School During Term .16 .37 69 .18 .39 38 School Nonattendance Index .38 .58 87 .31 .64 42 Rebellious Autonrimy .60 .34 37 .52 .36 30 Involvement .24 .18 77 .24 .20 37

*Treatment group significantly different from non-equivalent controls (p<.05).

-313- 190 Table 6 Jazzmobile Alternative Arts Project: Means and Standard Deviations for 1981-82 Outcomes Measured Using School Records

Group

Non-equivalent Treatment controls Outcome variable M SD N M SD

April, 1982, California Achievement Test(CAT) Grade Equivalent Scores: Reading 7.69 1.99 105 7.53 2.93 51 Math 7.32 1.55 102 6.88 1.99 48

Number of days absent 15.11 16.80 106 13.82 18.46 56 during the school yeara Promoted to next grade .93 .26 111 .81 .40 62 (1=promoted, 0=not)b Withdrawal for reasons .05* .21 195 .13 .34 156 other than transferc

Note. N's vary depending on the availability of data. Data on withdrawals were provided for the entire sample. Parental permission was required for the release of other information to the evaluation. Per- mission was obtained for a total of 183 students. Attendance, promotion, and test score data were missing for a few of these students. a No data were available for students who withdrew during the school year. bThe unadjusted proportion of students promoted was significant at the p<.05 level. The treatment and control group also differed at the. .01 level on prior promotions. An analysis of covariance, treating prior promotions and special education status as covariates, implies no significant difference (p=.78) between treatment and control groups. Because prior promotion data were unavailable for many students, the N's for this analysis were 76 and 18. c No pre-treatment datawere available for students who withdrew. Therefore we were unable to perform analyses that control for pretreatment differences.

-314- 19i Appendix A Job Descriptions Provided by Jazzmobile

Executive Director/Program Mana-er

Manages program to insure that implementation and prescribed activities are carried outin accordance with specified objectives and in accordance with O.J.J.D.P. guidelines: Plans and develops methods and procedures for implementing program, directs and coordinates program activities,and exercises control over personnel responsible for specific functions or phases of program. Selects personnel according to knowledge and experience in area with which program isconcerned,arts education. Confers with staffto explain program and individual responsibilities for functions and phases of program. Directs and coordinates through subordinate managerial personnel, activities concerned with implementation and carrying out objectives of program. Reviews reports and records' ofactivities to insure progress is being accomplished toward specified program objectives and modifies or changes methodology as required to redirectactivitiesand attainobjectives. Preparesprogram reports for the Office of Juvenile Justice and Delinquency Prevention. Con- trols expenditures in accordance with budget allocations. Meets withthe Advisory-Committee and District Representatives. Reports to the President and Board of Directors of Jazzmobile, Inc.

Educational Specialist

No description available.

Project Director

Under the direct supervision of the Program Manager, plans, directs, and coordinates activities of designated project to insure that aims, goals,or objectives specified forproject are accomplished inaccordance with prescribed priorities, time limitation, and funding conditions: Reviews project proposal or plan to ascertain time frame and funding limitations, and to determine methods and procedures foraccomplishmentof project and staffing requirements. Establishes work plan and schedules for each phase of project in accordance with time limitations and funding. Confers with staff to outline project plans. Modifies schedules, as required. Prepares project status reports for management. Confers with project personnelto provide technical advice and to assist in solving problems. Oversees daily project operations in all three school sites.

Artist/Instructors

Art. Instructs pupils in art, such as painting, sketching, designing, and sculpturing: Demonstrates method and procedure to pupils. Observes pupils' work to make criticisms and corrections. Directs planning and supervising of student contestsand arranging of art exhibits. Conductsworkshops for parents regarding the art discipline the student is learning.

Drama. Teaches acting principles and techniques to groups: Conducts read- ings to evaluate student's talent. Adapts course of study and training meth-

-315-

4 Jazzmobile ods to meet student's need and ability. Teaches enunciation, diction, voice development, and dialects, using voice exercises, speech drills, explanation, lectures, and improvisation. Discusses and demonstrates vocal and body expression to teach acting styles, character development, and personality pro- jection. Directs plays for schooland public performances. Auditions students to selectcastand assignparts. Rehearsesand drills students to insure they master parts. Assigns nonperforming students to backstage production tasks, such as constructing,painting, moving scenery, operating stage light, and sound equipment. Directs activities of students involved in constructing,painting, and lighting scenery. Teaches elements ofstagecraft, stage makeup, costume craft, play writing, and play direction. Conducts workshops for parents of participating students regarding the art discipline the student is learning.

Dance. Instructs pupilsin ballet, modern and ethnic forms of dancing: Observes students to determine physical and artistic qualifications and limitations and plans programs to meet students' needs and aspirations. Explains and demonstrates techniques and methods of regulating movements of body to musical or rhythmic accompaniment. Drills 'Pupils in execution of dance steps. Teaches historyof dance. Choreographs and directs dance perfumance. As instructor, conducts workshops for parents of participating students regarding the art discipline the student is learning.

Guitar. Teaches groups guitarin public school: Plans daily classroom work based on teaching outline he/she has prepared for this special program. Evaluates student's interests, aptitudes, temperament, and individual characteristics to determine abilities. Instructs students in music theory, harmony, score and sight reading, composition, music appreciation, andprovides group lessons using technical knowledge, aesthetic appreciation, andappropri- ate teaching techniques. Conducts workshops for parents of participating students regarding the art discipline the student is learning.

Percussion. Teachesgroupspercussion in public school: Plans daily classroom work based on teaching outline he/she has prepared for this special program. Evaluates student's interests, aptitudes,temperament, and individual characteristicsto determine abilities. Instructs studentsin music theory, harmony, score and sight reading, composition, music appreciation, and provides group lessons using technical knowledge, aesthetic appreciation, and appropriate teaching techniques. Conducts workshops for parents of participating students regarding the art discipline the student is learning.

Evaluation Coordinator

Directs research acitivities concerned with the program: Initiates procedures to determine if program objectives are being met. Provides interpretation of research data gathered. Works as the liaison between the program and the national evaluators concerning the management information system. Pro- vides evaluators with the data as outlined in the evaluation design. Coordi- nates data collection methods with the project director and school staff.

-316- 193 Jazzmobile

Counselor

Counsels individuals and provides grOup educational and vocational guidance services relating to the arts: Collects, organizes, and analyzes information about individuals through records, tests, interviews, and professional sources, to appraise their interests, aptitudes, abilities,and personality characteristics, for vocational and educational planning. Compiles and stud ies occupational, educational, and economic information to aid counselees in making and carrying out vocational and educational objectives. Refers stu dents to placement service. Assists individuals to understand and overcome social and emotionalproblems. Assists gifted and talented students with placement into special arts high schools. Provides arts activities and educa tional opportunities information for parents of the participating students.

Field Worker

Wor1s s the direct liaison between the program and the community. Under the direction ofthe Project Director organizes and conducts workshops for parents of the participating students regarding the program goals,objectives and their involvement. Also conducts parent workshops in conjunction with the guidancecounselor and artist/instructorsconcerning their subjectareas. Assists the Guidance Counselor with disciplinary problems requiring parental involvement.

Curriculum Coordinator

Plans and develops alternative education curriculum materials integrating the arts into basic subject areas for grades 6, 7, and 8: Develops materials for inservice education ofteaching personnel. Confers with localschool officials to develop curriculum materials. Confers with lay and professional groups to disseminate and receive inputon curriculum materials. Conducts conferences designed to promote arts education activities related to core cur riculum areas. Conducts research into areas,such as teaching methods and techniques.

Accountant

Manages all fiscal activities for the O.J.J.D.P. program. Applies princi ples of accounting to deviseand implement accountingsystem. Maintains accounts and records. Performs such bookkeeping activities as recording dis bursements,expenses, and tax payments. Prepares quarterly fiscal reports. Preparescashadvancement requests. Prepares weeklypayroll. Represents Jazzmobile before government agency upon certification by agency involved.

Audiovisual Production Specialist

Plans and produces audio, visual, and audiovisual material for communica tion, information, training, and learning purposes: Develops production ideas based on assignment and generates own ideas based on objectives andpersonal interest. Conducts research or utilizes knowledge and training to determine format, approach, content, level, and media which will be -losteffective, meet objectives, and remain within budget restrictions. Plans and develops prepro duction ideas into outlines and scripts. Locates and secures settings, prop

317 194 Jazzmobile erties, effects,and other production necessities. Sets up, adjustsand operates equipment, such as cameras, sound-mixers, andrecorders during pro duction. Conducts training sessions on selection, use, and design of audiovi sual materials and operation of presentation equipment.

Secretary

Schedules appointments, gives information to callers, takes dictation,and otherwise relieves officials ofclerical work and minor administrative and business detail: Reads and routes incoming mail.Locates and attaches appro priate file to correspondence to be answered by employer. Types routine cor respondence. Files correspondence and other records. Answers telephone and gives information to callers or routes call to appropriate official and places out going calls. Schedulesappointments for employer. Greets- visitors, ascertains nature of business, and conducts visitors to employer orappropri ate person. Arranges travel schedule and reservations. Compiles and types statistical reports. Makes copies of correspondence or other printed matter, using copying machine. Mayprepare outgping mail,using postagemetering machine. Types all curriculum materials.

Clerk/Typist

Compiles data and operates typewriter in performance of routineclerical duties to maintain evaluation data: Types reports, correspondence, applica tion forms, and data for evaluation. Files records and reports, posts infor mation to records, sorts and distributes mail, _answers telephone,and performs similar duties. Appendix B

The following promotional material, prepared by Jazzmobile stafferWilliam

Harris, presents their artist instructors in a positive light.These "artistic evaluations" are interest_ng and serve to give some of the flavorof the project's activities. We present them without fUrther comment.

Wendell Williams

Wendell Williams, a guitar instructor, relates his firstexperience of personal counseling of one of his Jazzmobile students. A 7th grade student ten- tatively indicated that he smoked marijuana. He later confessed that all of the money from his after-school, part-time job wasgoing toward purchasing the drug which he realized had become "a problem."He did not feel that he could talk to his parents about it, and fortunately Mr.Williams was an understanding, but nonjudgmental adult with whom he could discuss his-dilemma.

Williams emphasized that although he was sympathetic,concerned and supportive, the solution had to come from the student himself. The student was able to overcome his problem and thanked his instructor forhis help and concern. This incident taught Williams the importance of opencommunication with the members of his class. He instituted loosely scheduled class periods for general discussions of their problems and goals. Sometimes one or more of the students would feel the need for thrashing out problems andwould initiate the talk sessions.

"The whole thing is getting them on your side," Williams says. During these sessions, in addition to their problems, he discussesthe positives and negatives of the life of an aspiring artist, and also explores careeralterna- tives which most students had little knowledge of.

Earl Stewart

Percussion instructor, Earl Stewart, had his first contactwith a neighborhood family when one of their six sons entered his class. Although he was one of Stewart's best students, he was eventually lost to the streets.

The three younger brothers, 12, 10, and 8, all goodstudents, are now taking percussion. Stewart realizes that the street life and the"quick and easy" money are as great a lure to them as it was to theirbrother. He also realizes that without the positive attraction of Jazzmobile,it is a real possibility that they would already have fallen into "thelife."

-319- His method of combating the reality of that negative attraction is to constantly remind them of the alternative. Jazzmobile, he points out, is more than a day camp. To prove his point, he now has them involved in the project to raise money through performances.

Self-discipline is constantly stressed as a necessary ingredient for remaining in his class, and for making positive progress in their own lives.

Stewart has appointed the oldest of the brothers as his drum captain. He has taken the students to the rehearsals of his professional group. Although the final verdict is far from being in, Stewart believes the brothers are beginning to realize that they do not have to resort to the life style of their older sibling.

Elba Muley

Ms. Muley had an art student 12 or 13 years old. She found that he had a history of physical run-ins with his instructors and had spent time in a disciplinary school situation as a result. t,*

His abilities with the visual arts, expressed primarily through wall graffiti,were obvious, but sports and physical aggressiveness held his interest more than art.

Muley developed a project for a few of her students in various classes to create comic books which spoke to the problem of violence,stealing and drugs as they affected the school environment. The student, working independently by his own choice, developed the storyline, characters, dialogue and art work for the book. He worked formany weeks on the project. Though his interest was evident, it was a constant struggle to motivate him on adaily basis.

When the work was nearly completed he found out that others were also involved in similar pursuits. This angered him to the point that he tore up the materials that he had worked so long and hard on. Ms. Muley asked him to leave the class. He did, but returned a few days later. They discussed their misunderstanding both as it related to the nature of the project, and their personal conflict which resulted from that misunderstanding..

With this new understanding the student set about recreating his manu- script. He used his lunch periods and any free time he had to meet the dead- line for completing his submission. The student's aunt contacted his instructor to commend her for the marked change evident in his behavior.

The book was finished on time and was one of only three to receive an award in an art competition. He later completed a second book about graffiti.

Miriam Bacote

A girl, who Ms. Bacote soon realized lacked self confidence, entered her dance class. She came as a member of a clique of girls who thought the dance class was to simply involve learning the latest popular steps. When they found what was actually expected of them they withdrew from the class. The girl stayed, but was torn by an on-going conflict between her desire to follow her friends, and her interest in learning to dance.

197 Ms. Bacote contacted the student's mother in the hope that encouragement from home would help to relieve the girl's anxiety about her choice. Unfortu nately, the mother was more concerned with her own street activities and was no help. The student obviously benefited from Bacote's concern, and came to her own understanding of the value of the discipline involved in an endeavor that she loved. She developed to the point that she was featured as the lead dancer in one of the class's performances.

Roberto Reyes

Guitar instructor Roberto Reyes is well aware of the problem of providing technical instruction to students with widely varied levels of aptitude for the instrument. Using a method of initially having them copy the things that he plays, he begins to get a feel for their coordination and ability to memor ize, as well as getting them all moving at the same time at their various lev els. Step by step encouragement is most important at this stage, giving credit for even the smallest acccmiplishment. Actual musical notation comes only after everyone has a newly acquired sense of the confidence that they have some facility on the instrument.

Reyes' method of teaching his students to respect their instrument paral lels his philosophy of dealing with them as human beings. The period leading to the actual handson experience with the guitar is highly structured. The question of who paid for the guitars is discussed. Once it is understood that their parents are, in essence, owners of the instruments, a sense or responsi bility and privilege is connected with their use.- The students know that any abuse of the instrument means revocation of the privilegeof using it. He emphasizes that he has never lost a guitar through abuse or theft.

Felice Batiste

The image this art instructor tries to project is that he is his students' friend as well as their instructor. In addition to the specific techniques of artistic expression, Mr. Batiste strives to instill in them an appreciation of art and its varied experiences.

He maintains an "open door policy," always providing students with a friendly, impartial ear. He attempts to explain the seemingly unfair contrad ictions they are faced with. Batiste, who does not insist that his students call him by his surname, says that he does not ever remember having a free lunch period; there always seem to be a number of students who want to talk.

Batiste initially concentrates on classroom assignments which can be com pleted in one session; both as an alternative to the ongoing pressures of the rest of their school experience, and as an attempt to instill a steadily grow ing sense of accomplishment and selfconfidence.

An example of his oneonone contact with his students is a situation that occurred in a previous summer's program. Batiste had an introverted, but talented student who should have been graduating but had been left back. She wanted to go to an art school but was intimidated by the art test which was a prerequisite to entrance in the school of her choice. She was discouraged to the point of wanting to quit school. Batiste helped her with the basics of

-321- 198 the school test and convinced her of the importance of staying in school. :Ale eventually passed the exam, and is now doing well.

322 19I Otro Camino: Second Interim Report

J. St. John

Otro Camino is a community-based program that functions as an exten- tion ofthe school day. Project staff work cooperatively with local school administrators and faculty to assess student academic needs and schedule program activities. Besides the academic component, the program includes activities aimed at student social development and community participation in services offered.

Interim results suggest that the program may be increasing Otro Cam- ino participants' school grades, performance on standardized tests, and educational expectations. Students' reports of influence in the school and involvement in extracurricular activities may also be increased by the program. Negative findings for Otro Camino's academic service component on arrest and reports of punishing experiences in school are most likely due to selection bias rather than to the program.

Major Program Design Changes members. Community members were to be representative of many aspects of Changes to Community Development community life,including business, Components social services, police, and parents from several barrios in the la Playa Advisory council. Otro Camino community, with the majority on this had planned to institute an advisory 11-person committee beingparents. council during the first program The student committee was to include year. Although the council was a representative of each grade level formed, the mission of the organiza- served (/ altogether), the president tion was not clear, nor was the com- of the Presidents' Club at Otro Cam- position of membership specified. ino, student council representatives The council met only once that year. from the school, and other students In the second year, project managers who were interested in being on the decided the council should have four council (up to 11 members total). committees: (a) educators, (b) community members, (c) students, The council was charged bypro- and (d) an executive committee com- ject managers to (a) assess whether posed of elected officers from each the practicesat Otro Camino were committee and project management. educationally sound and work to The educators' committee was to improve the local schools (educa- includethe three schooldirectors tors), (b) assist Otro Camino in (principals), the district superin- meeting the needs ofthe community tendent, each school's faculty liai- (community members), and (c) assess son to Otro Camino, regional univer- whether or not Otro Camino student sity faculty, and others, upto 11 activities were adequate and sufficient (students). Their charge went beyond making recommendations and included assistancein implewenting This report supplements an earlier recommendations. report onthisproject (St. John, 1982) which should be consulted for The committees met separately, an account of the project'sfirst usually on the sameevening, and year of operation. after the separate meetings, the

-323--

200 Otro Camino total group met to hear the recom- attached to school. But they mendations ofeach committee. The regarded current teaching practices council did notbeginfunctioning atthe schools as an obstacle to fullyuntil January and reportedly their effectiveness. They hypothes- met two more times that school year. ized that attachment to school would increase only if teaching practices Project staff reported on the in the regular school improved. recommendations made by the parents' Furthermore, they thought that per- committee during the year. Commit- ceptions of the regular school might tee members were discouraged by the even worsen for students who parti- low turnoutfor their meetings and cipatedin Otro Camino activities. suggestedthat Otro Camino sponsor The program managers decided to special community events to gain institute teachertraining courses more visibility in the grass-rodts for the target schools' faculties in community. They suggested that stu- hopes that some teaching methods dents nominate theirmothers for would 'change. receipt of.the "Mother ofla Playa Award" that would be given by Otro School administrators agreed to Camino. All nominees would be noti- turn over three half-day in-service fied and invited to the awards cere- training sessions each semester to mony at Otro Camino and thereby gain Otro Camino personnel. Otro Camino more information aboutthe program offered workshops at two in-service and their children's participation. sessions in the fall semester, but Otro Camino staff and students later abandoned the practice. They implementedthis recommendation in feltthat thecompetition between the spring. The community committee theschool - organizations- and -Otro - - recommendation resulted in the staff Camino was being heightened by their and students sponsoring several efforts, and that the competition otherspecial community events as was hindering program operations. well. Otro Camino sponsored observations of International Women's In the second semester, managers Week, Holy Week,and Library Week, attempted to implement regular Otro and hosted numerous celebrations of Camino staff follow-up of partici- student participant achievements. pants by contacting school faculty. This was to serve thepurpose of School faculty support. SAES letting teachers know that goals Spring 1981 student questionnaire regarding students were shared,and responses indicated that students at that Otro Camino's purpose was to all three schools were reporting low support the efforts ofthe schools self-concepts and that their ability despite their different methods. to interact with others was low (two However, very little contact between schools were below the fifteenth Otro Camino staff and school faculty percentile on both measures and the actually occurred. Otro Camino man- other school was below the thirtieth agers reported that theirefforts percentile on the SAES Self-Concept were often thwarted because school and Interpersonal Competency mea- faculty schedules did not allow time sures). Otro Camino managers had to meet. designed their program to help students with their self-esteem and Parent involvement. Project interactive skills. They predicted staff planned to involve many that as students improved in these parents in the services offeredto areas they would feel more comforta- students. Early attempts to solicit blein theschooland become more parent volunteers to assist with the

-324-

201 Otro Camino student service component met with staff and managers believed this only limited success. Other strate- model was effective and implemented gies were developed to foster, parent small group instruction for the support ofstudents. A staff com- 81-82 school year on a semester mittee went to the advisory council basis. Projectmanagers believed and asked for suggestions on work- that the small group instruction shop topics that staff could take to approach would makeit possible to parents at community centers, serve larger numbers of students. recreation fields, government housing project community halls, and the Three distinctly different kinds like. A schedule set the beginning of academic assistance were offered of the workshops in the second sem- using the small group approach. ester of the 82-83 school year. During the firstsemester, groups Staff planned for the public rela- were involved in creative learning tions club to help with publicity by activities to reinforce specific broadcasting workshop times and academicskills. These activities places by loud speakers from cars. were intended to be fun. For ins- The staff audiovisual specialist was tance, some studentswho reported to arrange for electrical power at they needed help with Spanish read- outdoor sites .so that slides, video ing skills selected placement in a tapes, and movies could be part of group that readand discussed the allworkshops. At eachworkshop, daily newspaper. During the second staff planned to survey parents to semester, groups were conducted find out what kind of education top- using more traditional materials and icsthey would liketo have other traditional tutoring methods. That workshops on, and these topics would is, students brought in their school be added to the list for other books and Otro Camino staff helped times. This strategy was thought to students complete assignments. Stu- be responsive to the perceived needs dents were also tested for areas of of parents. Implementation data for weakness and given supplemental the 1982-83 year will show whether materials and instruction in these or not the strategy was successful areas. at increasing parental participation. School administrators requested that Otro Camino provideintensive Changes to Student Services Compo- tutoring to a group of students nents identified as potential dropouts. The schools sent 35 students in the Academic assistance at Otro Cam- seventhand eighth grades to the ino. Academic assistance has been a program site beginning in early major component of Otro Camino since March and ending in mid-May (the itsinception. Inthefirst year close of the school year). Otro the component took the form of indi- Camino staff divided the group into vidual tutoring in subject areas in five small groups. An elaborate which students reported having dif- schedule was established that pro- ficulty. This approach was used vided two 50-minute periods of because the program wasa drop-in tutoring each school morning. The centerand studentsattended when daily schedule included rotation they had time. In summer, 1981, through two subjects and allocated students participated forthe full four 50-minute period4to six sub- day. To accommodate the schedule jects in a ten-day period. change, the project adopted a small group instruction model. Project

-325-

202 Otro Camino

Library science. Otro Camino investigative science, and drama and managers arranged to train five high performing arts. Two clubs were school students in libraryproce- organized to address educational dures and techniques for high school issues in the community, andstu- credit. These students worked in dents organized a public relations Otro Camino's library learning book clubto help promote Otro Camino. cataloging methods, operating a book Executives from each club composed a lending system,and assisting with club of presidents who coordinated locating references for students. activities and kept all participants Four hours a week were spent working and staff up to date on each club. in the library, and students received one school credit for the One club had a special purpose. course. Several youths with discipline problems were referred to. the project Otro Camino mini-courses atthe from the high school. Repeated schools. Each Otro Camino staff counseling efforts had proven unsuc- member developed a mini-course dur- cessful except in eliciting some ing the first semester for possible personal goals. Several students in implementation at the schools in the this group indicated they wanted to second semester. Staff contacted become martial arts experts. A school teachers to ask permission to relative of an OtroCamino staff incorporate the mini-course into the personowned a karate school and regular classroom courses. Mini volunteered to donate one class per courses were designed to lastten week day for these students. In the days, had a limited number of learn- class, students learned the discip- ing objectives, included daily les- line necessary to - become experts, - - son plans, and were to be given by and school personnel reported the Otro Camino staff with the teacher students' self-discipline improved present in the room. The courses markedly at school. Once the stu- covered specialtopicsin Spanish, dents had mastered basic karate English, math, science, social sci- maneuvers, they performed at a ence, and folk crafts. Each course school assembly, and appeared to was developed fora particular age have become respected members of the group. Not allteachers agreed to school community. I observed a participate. Project managers well-received presentation of skills reported that three of these courses given by karate club members during were given duringthe 81-82 school Otro Camino's week celebrating stu- year, and that their course on the dent achievements in April. la Playa community was being given in each tenth grade social science Learning resource center. Origi- class beginning in August, 1982. nally Otro Camino planned for the learning resource center to be used Clubs. Otro Camino attempted to by drop-instudents(studentswho promotestudent social development were not registered in formal activ- through clubs. Otro Camino devel- ities) as an additional resource for oped and implemented 19 clubs during their school assignments. The cen- the 81-82 school year. Project ter included a library and audiovi- plans had called for the implementa- sual center. In the second year of tion of five clubs beginning in the operation, very few non-registered firstprogram year, butnone were students used the center, but large begun until the second year. The numbers of registered students used implementedclubs coveredpersonal the center facilities at times they growth(career andpsychological), were not scheduled for activities. recreation, arts and crafts,

-326--

203 Ogre Cadge

44004.*** OOP fteopera- 144401.4, el 4 ttoo *dew toStrop Contoes 044*.o* OWNOS 1404tow jib poprottos aid tests- so. wsod tag for otodonts. Jeb-sookies kelp woe *flood to smallWINO*. SCW 6444 *Simko. dooto rootood iestrictios is boo to woes* sop* tend aid al $y ter jobs, istorilaw- 444* Ao to ten *tilts, istorporossal asoessica- 04040 *Wise* Its* Atli% sad 'Moist 's* rossire- 6...4.*aid sos4* for wises oroors at ooroato owl4 to sumo embers. Ibey .** aala istat***360$* it** .*coiood assistingo is locaties 44$4 444 040444, oit* 1tt 4 4*400 it4440so *MOO 400004 404,404 T6* mit ***pored*. odocotion *SO* ,44r okommow444t 404140.44$410 speristios woo osepostil is sem- 44.04041* 401040410444.0 440 *WSW iss 44 aides* pfsionost positiose. 04* 0044040 4440,410i04O. sod bet two 14ass000ts sir. acteally St Ow. 400* off. pro** 4ea444 m. staff diocevatol lbat lb* WOO 6606004/04 onseits swoon mild fakow limo wore veryWittis 040 *000444* 464000 04041404044 tort* *tanslied promoted placing 0440 4040 44444**4 014 04 46. rosiest .tom. to Jobe tor pay. 4tosta44 .441.41* Aid, oho 00166 oppioyers *wood to 4f *4411-ostaitoora sot t. *stools Woe* wood to Om, or*osicredit for 410 10141 oolowomor ptessoosts, stodosto fre- eway **stied est to take the 4 4 4400100411 444* 1140440 440P, ovotiobto pottties. Appsrestly 4404400 414404w *tidos.* distded tboy mold be bet- 4404 #4440444414 4444444+ 0404 44414, ter aft pro/par ts* ter trusts*atter **MONO 0#40041 ott, Pro)00, ttsb stbsol by stayias is **beet. otooma4amo 01*** *04440 o****** ot otidonts did work as indical 1000s4o maboo0* oo4a0mta 4oto aoltooio*** arottoittoo oboe aokototo cob olatotasto sad secretaries dar- Mows ate- atitt44414 *so 44opowoctofie t** the Mot sm000tor. ilioSo were isperttsod by potpie at *moo40440044*, 40,oto*o41 at tbs., OW et** a0 the 44re Cods* sivestlin 4001 *4**,14 bmw bowl oo*b eisposetico mood NG lbw* tkop baltpod iodter rossiord sibool cultt. 400mal otomoots**4 otoo WO to wood imm00 ***oily* atalmOvo aobottmlow. to addition to tMoo positions to 400 pia*****0 to ottioslet aotote** stinanity orssolsatties. the coops/r- tamotoimgc, etie* ciwesttos specialist tsusht a **arse to folk *rafts. Sbe took the 41,04. 44 *4 *404.*vs portition4a Is Sae lass ober* stn - mere*M insIba SNOOM444 iit000lo ictiOmio ta s towed poop,* 4644014 44140410044 Usdo of Was. they *ors nekins. 11.010.04* modest* 014 44601044wv olotodoro. lb** poporso41, Wiwi ors omboolito. ts3w omen J. WSW tho booboos. **testis) of *ow *04 gloom**,44wootor. hooir effort,. ii41041 ato**Istowo tow instare 4444 tot vtatioreas roloosowtotitiatif too 400010 wolloqw **Wmtato, oa 44,64* 0444 ott$F4o1 tooth* Otro Camino

Major Force Field Changes local artist offered a class in folk art, 15 community businesses opened A new school opened in August of 48 job training slots for partici- 1981 and caused unanticipated delays pants, thelocal school ofkarate in Otro Camino activities. Two tar- donated one class each day to a get schools changed from double-ses- small group of participants, the sion to single-session days and re- Ponce Regional Office of the Puerto distributed grade levels. Resulting Rico Department of Education sent school enrollment and scheduling personnel to teach community inter- problems lasted for about a month. view and questionnairedesign and Otro Camino managers had to wait statistics to participants, and the until school schedule problems were police department sent a representa- resolved to establish theirsche- tive to the advisory council. The dule. Puerto Rico Department.of Education Research and Evaluation office coop- When Otro Camino finally began to erated in sending standardized test schedule students into activities, scoresto thenational evaluation project managers discovered that project. their plan to serve students during the school day could not be imple- Implementation Study mented. The schools had added electives to the school curriculum and Project staff kept careful students no longer had free periods records on student attendance at during which they could participate. Otro Caminoby activityattended. Project managers quickly assessed At the end of each semester the the feasibility of after-school and dailyhours weresummed for -each week end activities and received a activity, then added across activi- positive response. The ensuing ties for each student. This was the reorganization further delayed pro- primary means of reporting activi- ject start-up. tiesoffered, numbers served, and total Otro Caminostudent service Changesin schedules meant that hours. large numbers of students were served in a fewhourseach day. These data are tabulated in This necessitated offeringcourses Tables 1 and 2. The tablesshow that met two or three days a week at thatclubsarethe mostintensive alternate timesratherthanevery service provided by Otro Camino. day because of space shortages. Students who participated in clubs Project managers also gained school spent 25-30 hours per semester, or administrators' permission to close to twohoursperweek, in arrange with school teachers for these activities. The intensity of inclusion of ten-daymini-courses the vocational and academic services given by Otro Camino staff in regu- increased from semester one to sem- lar school classrooms. ester two,and the counseling services remained relatively low. The Some changes were salutary. Otro main counselingservice wasgroup Camino's good reputation spread counseling for 81 students who appa- through the community and resulted rently met approximately once a in the enlistment of several commu- month. nity volunteers. A parentvolun- teered to help with tutoring, area Tables1 and 2 show that,alt- university professors assisted in hough theabsolute number of stu- diagnosing students' academic needs dents served declined from the first and designing treatment programs, a

-328-

205 Otro Camino to the second semester,the inten- The SAES measures are described sity of serviceincreased. During in Gottfredson, Ogawa, Rickert, & the first semester 74.8% of the stu- Gottfredsn (1982). Two distinct dents participated in only one act- scoring procedures were used. The ivity--clubs, most likely. But dur- 1981 questionnaire scales mostly ingthesecond semester morethan involved sums of standard scores for two-thirds of the students were items in a scale. For the 1982 ver- active in two or more different sion, a new scoring procedure was activities. Theaveragehours of implementedto make interpretation participation per student also easier: Most scale scores reflect increased somewhat during the second the proportion of items answered in semester. the desirable direction. Because of this scoring difference and because Outcomes the 1981 scale scores were used only as statistical'controls, the 1981 Evaluation Design scores are not reported here.

Otro Camino has an open door Analysis Procedures admission policy, meaning anyone who comes for services is served. Ran- Measures ofintensity of treat- dom selection of programpartici- ment were computed as follows: pants would have drastically altered Records of treatment hours were col- this policy. A nonequivalent com- lapsed into major categories: Aca- parison group design was estab- demic assistance activities,voca- lished. This design enabled the tional development activities, club comparison of participating and activities, counseling activities, non-participating students from the and participation in activities dur- targeted grades at each of the three ing 1980-81 (the first year of pro- program schools. Measures from gram operation). Spring, 1981 SAES surveys and school record data were used to statisti- The number of participation hours cally control for the nonequi'.alence in each category was summed for both of the groups. semesters and correlated with each outcome measure. The correlation Data matrix was examined for correlati- ions significant at the p<.U5 level The project was specifically (two-tailed). designed to help students in the major transition grades 7, 9, and Then, regression equations were 12, so data were collected on all used to estimate project effect students in the 1981 pre-target parameters--that is the part of the grade. Pre-target year and target correlation that can be attributed year data include the SAES student to participation in the program questionnaire(administered in the rather than selection effects. spring of 1981 and spring of 1982), These equations included as statis- school records data on school atten- tical controls any pre-target year dance and grades, Puerto Rico Stand- measures that correlated with both ardized Test Scores, and records of the activity category and the out- police arrests for both years. come.1 Since patterns ofcorrela- These data were also reported for tions were different for various samples of all other grades between 6 and 12.

-329-

206 Otro Camino activities, different control varia- more alienation, lower self-con- bles wereused invarious tests. cepts, less interpersonal compe- Notes on tables of results indicate tency,and spending less effort on the control variables for activity school work than nonparticipants categories. (Table 5); and participants reported more delinquent activity(self-re- Results port and record of arrests), more victimization, more suspensions, Tables 3 through 6 present the less attachment to parents, more correlation and regressioncoeffi- negative peer influence, lower cients forall outcomes. Outcomes belief in the validityof conven- are organized accordingto project tionalrules, less practical know- goals (Tables 3 and 4), project ledge, less internal control, and objectives (Table 5), and other more school punishment and fewer theoretically relevant objectives school rewards than nonparticipants (Table 6). Appendix tables show (Table 6). Most of the correlations means andstandard deviations for were small and nonsignificant, but all outcomes by activity category eleven were statisticallysignifi- for persons participating in each cant. kind of activity. These uncontrolled associations Academic Activities are not unexpected: Persons seeking academic assistancein the program Twenty-eight ofthirty-nine cor- may reasonably be expected to be in relations (72%) indicated the asso- need of it. These resultsdo not ciation between participation in imply that the participation/in Otro academic activities and academic Camino's academic activities brought outcomes was in the unfavorable about these outcomes. direction. (Read across the rows of Tables3-6.) Participantachieve- To examine whether these associa- ment was lower than nonparticipant tions remainsignificant when pre- achievement on 12 of 13 measures existing differences among students (Table 3); participants reported are statistically controlled, the skipping more classes (Table 4), betas in Tables 3 through 6 must be inspected. When controls are applied, number of hours of participation in academic activities is 1 In technical terms, a model of the associated only with arrest and form y = n(-C4T.A,te was estimated school punishment. In the context for each outcome variable (y), of the model for the data examined where 7is the standardized regres- here, this association could be sion coefficent for hours of parti- interpreted as an effect of partici- cipation in activity category (x), pation in Otro Cmmino's academic the/:: are the standardized regres- activities. Two of these betas are sion coefficients for the exogenous significant.

control variables f , and e is random error. The specification of Vocational Activities each such model (i.e, the choice of

the ) ) wasmade to include as Thirty-seven of forty-one corre- exogenous control variables any 1981 lations (90%) between participation measure correlated both with extent in vocational activities and out- of participation in activity x and comes were in thefavorable direc- with the particular y under consid- tion. Only on reports of having a eration.

-330-

207 Otro Camino job (Table 3), amount of school Counseling Activities involvement, internal control, and school rewards(Table 6)were the Thirty offorty-one correlations nonparticipants favored. Twenty-one (73%) between counseling activities of theseuncontrolledcorrelations and outcomes were in the favorable are significant, but when statisti- direction. Nonparticipants were calcontrols are applied,only two favored on math grades, school grad- remain significant. In the context uation, having a job (Table 3), of the models ofthe data examined self-reports of school non-atten- here the results (shown in Table 3) dance(Table 4),alienation (Table could be interpreted as implying 5), reports of victimization,sus- that participation in Otro Camino's pensions, belief in the validity of vocational activities increased conventional rules, and practical scores on a standardized test of knowledge(Table 6). .The applica- Spanish language and increased tion of statistical controls reduced grades (according to self-report).2 the number ofsignificant correlations from six to two. In the con- Club Activities text of the models examined here, ',the results imply that participation Thirty-four of forty-one correla- in Otro Camino counseling activities tions (83%) between participation in increases educational expectations clubs and outcomes were in the favo- (Table 3) and involvement in school rable direction. Only for self-re- activities (Table 6). portof reading ability (Table 3), reports of victimization, suspen- A re-examination of academic sions, belief in the validity of activities. Project managers were conventional rules, rebellious asked about the academic activities autonomy, and internal control results. They hypothesized that the (Table 6) werethe nonparticipants students referred for intensive favored. Fifteen of these correla- tutoring may have been markedly tions weresignificant. Whensta- different from students voluntarily tistical controls were applied, participating in Otro Camino's other three remained significant. In the academic activities. Furthermore context of the models examined, par- the tutoring came late in the year ticipation in clubs increased social and the nature of this intervention science grades (Table 3), students' was different fromother academic reports aboutstudent influence in activities. They requested that the school3 (Table 5), and reports academic activities be re-examined of involvement in school activities excluding the tutoring activity, so (Table 6). the results could be used in helping decide which way to approach academic activities in the caning year. When tutoring is excluded and only 2Although not significant when sta- those academic activities indicated tisticalcontrols are applied, the by the absence of an in Table 1 pattern of results for actual school are examined, there is no dependable grades accords with self-report indication ofany effect(positive results. or negative) of participation in the 3This measure is composed of items remaining academic components. ordinarily scored at the school level (Gottfredson, Ogawa, Rickert, & Gottfredson, 1982) but combined here into an individual-level "scale" the psychometric properties of which have not been examined.

-331- Otro Camino

Discussion This is precisely the number of significant differences observed. Except for results associated with participation in academic In support ofthe argument that activities, the results are encour- the results indicate, the effects of aging. Most of the correlations are participation, many 'of the results in the right direction and there is are sensible relative to project some evidence that participation had activities. Project intervention positive effects. However, the with students referred for intensive actualdifference between partici- tutoring did not begin untilnear pants and nonparticipants is small. the end of the school year, allowing Forinstance, less than 1% ofthe plenty of time for academic and variance in Spanish language stand- behavior problems tooccur before ardized test scores is accounted for participation and after pre-level by participation in vocational measures used as statistical con- activities. Most of the variance is trols were collected. The reasons accounted for by grades and ability for their school referralincluded measured by standardized test scores recent disciplinary problems, in the previous year. "declining achievement, and. attendance problems. The findings cannotnecessarily beattributed toparticipation in Several of the vocational devel- the specific activity category opment activities emphasized lan- listed because many students (about guage skills and exam preparation. 78%) participated in more than one It is not surprising that standard- activity. The analytic procedures ized test scores-improved-for -this used included only hours of partici- group. Students in this group were pation in one category at a time as also reported to have renewed their a means of helping project managers commitment to education in prepara- learn what happened for the various tion for careers. categories of activities implemented. Many clubs emphasized the community cultureand history. Others Project participation was not were aimed specifically at develop- under experimental control; thus it ing leadership skills. It is sensi- is possible that the results reflect ble that students in these activi- self-selection or differential matu- ties had higher social science ration. Participants were plausibly gradesand perceived more student more highly motivated to better influence at the school thanthe themselves and measures applied as nonparticipants. On the other hand, statistical controls may have been people who are "joiners" in one set- insufficient to control for this ting are likely to be "joiners" in possibility. anothersetting. Thus it is not surprising to find Otro Camino par- A final word ofcaution inthe ticipants also participated in a interpretation of the significant variety of school activities, and resultsfrom the regression equa- thisresult may thereforereflect tions: Some significant differences primarily a selection artifact. can be expected to arise by chance. In these analyses, about two of the The counseling component included 39 or 41 equations examined for each helping students decide about future categoryof participation would be plans and determinetheeducation expectedto reachsignificance at needed to successfully reach their thep<.05level simply by chance.

-332-

209 Otro Camino personal goals. Counselors tothe goals, theory, and objec emphasized the need to stay in tives, and implement planned compo school--both staying in school now nents with integrity. They have and pursuing education in the encountered numerous obstacles, and future. Much of the counseling implementation of some components occurredat thehighschoolwhere has, been seriously delayed. But many of the students who were going eachdelay andeach obstacle has toleaveschool had already with resulted in more planning, and these drawnandat a time instudents' efforts appear systematically to lives when decisions about the strengthen the component while future must be made. It is sensible remainingtrue to thetheory that thatthis component is associated led totheir development. Perhaps with higher educational expecta the most serious implementation tions. problem the project would ever face is delay while new strategies are Finally, I have observed that the develdped to insure implementation entire project staff are committed of components anchored in theory.

333 Otro Camino

References

Gottfredson, G. D., Ogawa, D. K. Rickert, D. E., & Gottfredson, D. C. Mea sures used in the School Action Effectiveness Study. In G. D. Gottfredson (ed.), The School Action Effectiveness Study: First Interim Report. Bal timore: Johns Hopkins University, Center for Social Organization of Schools, 1982.

St. John. Otro Camino. In G. D. Gottfredson (ed.), The Schoo_ Action Effec tiveness Study: First Interim Report (Report No. 325), Baltimore: The Johns Hopkins University, Center for Social Organization of Schools, 1982. Table 1

Number of Students Served and Intensity of Treatment--by Activity and Semester During the 1981-82 School Year

Semester 1 Semester 2

Number of Avg. hrs./ % Total Number of Avg. hrs./ % Total Activityb -students student effort students student effort

Academic 157 11.2 13.3 115 22.7 27.0

Oral Communication-- 37 5.5 1.5 8 6.4 .5 English Reading--English 16 5.9 .7 7 7.1 .5 Speak, Read and 13 5.8 .6 0 0.0 0.0 Write--English Keeping with the 1 2.0 t, 0.0 1 1.0 0.0 News--Spanish Learning to Read 1 24.0 .2 2 8.5 .2 and Write--Spanish Growing with Music 16 8.8 1.1 0 0.0 0.0 --Spanish The Why of Math- -Math 6 3.7 .2 0 0.0 0.0 Multiplying to 2 2.0 0.0_ 0_ _0.0_ 0.0 Divide--Math Everyday Problems 21 3.4 .5 6 2.8 .2 --Math Photography--Science 30 28.9 6.6 20 29.8 6.3 Puerto Rico in the 2 30.0 .5 0 0.0 0.0 World--Social Science Learning More 14 13.0 1.4 0 0.0 0.0 About Puerto Rico- - Social Science Civicsa 0 0.0 0.0 26 8.5 2.3 Natural Sciencea 2 2.0 0.0 37 8.5 2.3 Englisha 0 0.0 0.0 38 5.7 2.3 Spanisha 0 0.0 0.0 39 7.0 2.9 Matha 0 0.0 0.0 43 6.2 2.8 Social Sciencea 0 0.0 0.0 38 7.3 2.9 Investigation Club 0 0.0 0.0 5 71.0 3.8

Note. "Number of students" column reports the number of students who had at least one hour of participation in a given activity. The "average hours per student" column reports the average number of hours for students who participated for at least one hour in the activity. The "Z total effort" column reports the total number of hours spent by all students as a ratio of the total number ofparticipation hours. aSecond semester participation hours in these activities represent timein Otro Cam- ino intensive tutoring; all other courses in the academic group representcreative gcademic activities. All 1981-82 activities except the Summer, 1982 "Mini-University" are included on this table.

(continued on next page)

-335-

21_2 Table 1 (cont.)

Semester 1 Semester 2

Number of Avg. hrs./ % Total Number of Avg. hrs./% Total Activityb students student effort students student effort

Vocational 216 17.9 29.4 43 26.0 11.9 Librarian Aides 10 48.8 3.7 7 36.4 2.7 Audiovisual 75 4.0 2.3 4 13.0 .6 Techniques My Career for the 4 14.0 .4 0 0.0 0.0 Future

Getting Prepared 20 29.1 4.4 32 - 25.3 8.6 for Work Where Will I Study 8 23.3 1.4 0 0.0 0.0 College Entrance 103 21.1 16.5 0 0.0 0.0 Exam Preparation Short Term Careers 8 10.1 .6 0 0.0 0.0 Club

Clubs 269 24.7 51.3 156 30.0 49.7 Public Relations 10 2.0 ,2 12 39.2 5.0 Education 0 0.0 0.0 0 0.0 0.0 Sports 0 0.0 0.0 0 0.0 0.0 Model Building 13 4.9 .5 3 2.0 .1 Dramatic Arts Club 10 94.0 7.1 13 7.5 1.0 26.5 5.6 Aesthetic and 48 16.8 . 7.1 20 Modeling Club Astronomy Club 23 55.2 9.7 14 20.9 3.1 14.9 1.9 12 11.9 1.5 . Gardening Club 17 Handicrafts for 57 20.6 8.9 28 16.3 4.8 Girls Handicrafts Club 46 40.8 14.3 17 53.2 9.6 Folk Dances of 27 4.5 .9 8 60.1 5.1 La Playa Folklore of La Playa 57 2.1 .9 8 28.0 2.4 Presidents Club 0 0.0 0.0 0 0.0 0.0 Karate Club 0 0.0 0.0 15 16.4 2.6 Vocal Group 0 0.0 0.0 10 82.9 8.8

Counseling 117 7.6 6.8 140 7.2 10.7 Self Development 14 16.0 1.7 0 0.0 0.0 Club Individual 22 12.5 2.1 33 2.0 .7 Counseling Group Counseling 81 4.8 3.0 111 8.5 10.0

Total 653 20.1 100.0 397 23.7 100.0 Table 2

Number of Students Participating in Otro Camino Activities--by Semester of the 1981-82 School Year

Number of Concurrent Semester 1 Semester 2 Activities

1 493 74.8 136 31.9

2 142 21.5 122 28.6

3 21 3.2 56 13.1

More than 3 3 , 0.5 112 26.3

Total 659 100.0 426 99.9

Note. Numbers do not include students who participated only in 1980-81 school year activities.

The unduplicated number of students servedfor the 1981782_ school year is 797.

-337-

214 Table 3

Correlation Coefficients and Standardized Regression Coefficients for1981-82 Academic Achievement Outcomes with Number of Hours in Each Type ofOtro Camino Activity

Project Goal I

Self Report School Grades Standardized Tests SelfReport

Sch. Read. Educ. Have Paid Activity goc. Span. Span. Eng. Grade Abil. Expect. Grad. Job Work Type Span. Eng. Math Sci. Sci. Read. Lang. Read. Math

Academic -.063* -.024 -.060* .016 -.048 -.013 .005 .020 r -.041 -.027 -.053 -.005 .017 -.027 -.Ohl -.020 -- -.020-- -- -- ---- beta ------ -- -- -- --

Vocational .201* .180* .044 .163* .442* -.045 .039 r .164* .145* .043 .039 .100* .095* .200* .071* .084*-- .096 .050 ---- beta .055 .023 -- -- .009 .032 .083* -.025 .077

Clubs .042 .030 .087*-.014 .009 -.032 .020-.009 r .059* .072* .073* .140* .094* .029 .056* .025 ---- -- ---- beta -.014 .003 .017 .069**.023 -- -.014 -- --

Counseling .072* .022 .118* -.038 -.034-.033 .046 .056*-.018 .088* .049 .008 .054 .054 .054 -- ---- 1 -- -- -- -- -- .013 -- .090** to beta -- -.011 -- .031 co Participation .7.' 1981 .045 .091* .036 .006 .043 .064* .028 .036 r .010 .019 -.014 -.101*-.031 .092* .038 .100* -- -- -- -.017 ---- beta -- -----.034 -- .145** -- --

Participation 1981 is a dummy variable measuring whether or not studentsparticipated in 1981-.82 rather than number of Note. Analyses for eac N's range from 447 to 1006, depending on the patterns of missing datafor variables in each equation. hours. statistical outcome variable uses 1981 variables that aresignificantly correlated with that outcome and with participation as reading score; English grade; and SAES friend control variables. Academic activity controls include 1981 standard test English Spanish reading scores; school days absent; influence scale. Vocational activity controls include 1981 standard test' math and school grades for Spanish, English, social science, and science;overall grade point average; and SAES belief, interpersonal co delinquency petency, school rewards index, school punishmentindex, self-concept, rebellious autonomy, school attachment, total Club activity controls include serious delinquency, and victimization scales; self-report of gender;and 1981-82 school grade. interper- 1981 standard test Spanish language score; school days absent, andSAES self-report of grades, attachment to parents, school, victimization, and self-report of socio-economic levelscales. sonal competency, school punishment index, attachment to SA Counseling activity controls include 1981 standard test Englishreading score; school grade for math and social science; and Participation 1981 controls include 1981 school grade for English, andSAES internal control and rebellious autonomy scales. interpersonal competency, rebellious autonomony, attachment to school,and self-report of socio-economiclevel scales.

See Table 1 for listing of specific Otro Caminoactivies for each category. Reported for 1981-82 twelfth-grade students only. Regression coefficient not reported becauseactivity type is not related to outcome. Beta unstable; interpret with caution.

* Coefficient is significant at the P<.05 level. **Coefficient is significant at the P<.01 level.

215; Table 4

Correlation Coefficients and Standardized Regression Coefficients for 1981-82 School Withdrawal and Attendance Outcomes with Number of Hours in Each Type of Otro Camino Activity

Project Goal II and III

School Report Self Report

Percent School Skipping Skipping Activity Type Withdrawal Days AbsentAttendance School Classes

Academic r -.048 -- -.010 -.021 .022 beta -- -- -- -- -- Vocational r -.057* -.094*,, -.000 -.033 -.022 beta -- .024 -- -- -- Clubs r -.063* -.090* -.072* -.052 -.076* beta -.036 -.036 -.048 -- -.046 Counseling r -.049 -.040 .002 .010 -.035 beta -- -- -- -- -- Participation1981 r .045 .021 -.035 -.031 -.054 beta -- -- -- -- --

Note. Participation 1981 is adummy variable measuring whether or not students participated in 1981-82 rather than number of hours. N's range from 447 to 1006, depending on the patterns of missing data for variables in each equation.Each outcome is controlled by 1981 variables that are significantly related to the type of activity. Academic activity controls include 1981 standard test English reading score; English grade; and SAES friend's influence scale. Vocational activity controls include 1981 standard test math and Spanish reading scores; school days absent; school grades for Spanish, English, social science, and science; overall grade point average; and SAES belief, interpersonal competency, school rewards index, school punishment index, self-concept, rebellious autonomy, school attachment, total delinquency, serious delinquency, and victimization scales;self-report of gender; and 1981-82 school grade. Club activity controls include 1981 standard test Spanish language score; school days absent, and SAES self-report of grades, attachment to parents, interpersonal competency, school punishment index, attachment to school, victimization, and self-report of socio-economic level scales.Counsel- ing activity controls include 1981 standard test English reading score; school grade for math and social science; and SAES internal control and rebellious autonomy scales. Participation 1981 controls include 1981 school grade for English, and SAES interpersonal competency, rebelliOus autonomony, attachment to school, and self-report of socio-economic level scales.

See Table 1 for listing of specific Otro Camino activies for each category. Reasons for withdrawal include voluntary and involuntary withdrawal, transfer to another school, and reason unknown. Percent based on number of days active. Overall school and class attendance. Regression coefficient not reported because activity type is not related to outcome. Not analyzed.

* Coefficient is significant at the P<.05 level. **Coefficient is significant at the P<.01 level.

-339- 216 Table 5 .

Correlation Coefficients and Standardardized Regression Coefficients for 1981-82 SAES Student Questionnaire Scales with Number of Hours in Each Type of Otro Camino Activity

Project Objectives

School Self Interpersonal School Student Activity type AlienationAttachment Concept Competency Effort Influence

Academic r .031 .007 -.072* -.035 -.066* -- beta -- --(b.> -.044 -- -.039 --

Vocational r -.053 .054 .106* .059* .124* .092* beta -- -- .060 .038 .067 .028

Clubs r -.031 .076* .059* .026 .077* .100* beta -- .021 .013 -- .038 .104 **

Counseling r .009 .021 .055 .036 .072* .044* beta. -- -- -- ---- .044 --

Participation81 r -.005. -.027 .051 .016 .012 .:06* beta -- -- -- -- -- .086**

Note. Participation 1981 is a dummy variable measuring whether or not students participated in 1981-82 rather than number of hours. N's range from 447 to 1006, depending on the patterns of missing data for variables in each equation.Each outcome is controlled by 1981 variables that are significantly related to the type of activity.Aca- demic activity controls include 1981 standard test English reading score; English grade; and SAES friend's influence scale. Vocational activity controls include 1981 standard test math and Spanish reading scores; school days absent; school grades for Spanish, English, social science, and science; overall grade point average; and SAES belief, interpersonal competency, school rewards index, school punishment index, self- concept, rebellious autonomy, school attachment, total delinquency, serious delinquency, and victimization scales; self-report of gender; and 1981-82 school grade. Club activity controls include 1981 standard test Spanish language score; school days absent, and SAES self-report of grades, attachment to parents, interpersonal competency, school punishment index, attachment to school, victimization, and self-report of socio - economic level scales. Counseling activity controls include 1981 standard test English reading score; school grade for math and social science; and SAES internal control and rebellious autonomy scales. Participation 1981 controls include 1981 school grade for English, and SAES interpersonal competency, rebellious autonomony, attachment to school, and self-report of socio-economic level scales. See Table 1for listing of which Otro Camino activities are included in each category. Regression coefficient is not reported because activity type is not related to outcome. Not examined. * Coefficient is significant at the p<.05 level. **Coefficient is significant at the p<.01 level.

-340- 217 4461* t

4'74ri4v0ifcgo rt*10,#*/,40414 #04666460416 4160464464.16 446,444P-4W* I** 410100,94P'; 04.444401610*-446i4,444 4:0-01,404 %Awl+mi4 40104** tio tro v# 46***46440h4o4014

*04004 +1014$4 o.kitki 44 4,4144w,$ *Ffii

t:. Z.. t- r,t. U, SS It 9. scxn r. rleits 4, 4 09494-94*9099044t 4ioo04ti 44194,0 271,14,M234,U.42-tiMU.S.111iiil

4i4e.00t 4#.6 4444441; 4%444%; 0444*444 6444,40,61 to4614*4 411,,^4 46114* 40+ 4909, 400**090! 404401. 499 No I *014 0-14 0+04 he. .044,40.4 -tootortowo,4*499. rop.44* trot/96i to OM 1 orsw* - , '':1VZ.T.I.VISIlt111,,M7.1111TIT24,U.D8 014444+ 4 r 1 91,1 4 ,104=' ;$941t4 ;004 Alik ill zirnOtb /s1/10 AS6

40404 I-4 ooploon.

14444 VP6t ..01010

.40.0141- 44 op+t 400-$: AO* :4004 =)1066 =Alt lo44 a 24. wooiii

094140910.144411- 1-)4016; A6100 4046 z4,*4 is.0,4 0 4099/0 az** 0

0444 i.,1r14644+4**1 0444 A0,01 al04t4 0041 AIN .01$ !,:t001+ a,tIke L,a,44

40ow *0.4,4400,4,*, 44*0 44, * 41***6 Av44,04o 4401%4416* 4464404 44 4094 *446644* 14t64414441444,4 44 1,0141 1444rt 1664 4,0043,4 -4****. +4466 0110,44. loth fis 404,44099000409, *0 04449009. b 4444094 WO 404 49411019144 10 44414

0.446er deig -OW ...*11* 4400,600414,0 .1%) 'Pe*4t00+040491 +40919/41940 +4440.444044911* 000404 40 409/0tillwr 64 WI 644. *410'. Vit'Ve *10,"4104 #01*040 4400/ .i$'440111*. +WO 11%** +OM 00401.111es**', 0404* +41.46.4 6441 1404* 144(.64 * 1414010.44114+,000Vm 4401.0000.4 !010014114it 4.0041010 0111010 01.10400411 fokst *WA104 11/064909019 94911130* 10/0004 40444 SOTO 111101490t 49440kw itoi,*w.4,4* 44040q4+,.,tow4,004,4 Now+p4 0041o1ww) $99 popottis low4*0 44000 10404494 0044004/1 led IOU 191,4 4910/1694ois4400i roimplo:9**9+ *01,00Op/0* 190410+;. 9040004 900044010404 4#4 P 44,4010P044, ~WA* 00400P407, bc09001 4:4041/400MW; 4/00114 41000ro09999wO, ow4g411940 4044.11000,1049;604 4.14444,64666 64-64**6 **4444441 44 footer4 ii 1,61-41 104140, 60404.. 0**1094490#0* 0400990+ 46f449 9$04 4400M09,4 4444 44004400 10006240winch 606.4 66pa *Moat. 6641 004 06144A0m06*40 110t04104-10144004mon0 4* 4.000kt0-4 -04MAMP*0**Ii4MNI405444*4. pom4ster6411019004 .414091400494 10 0004900f".. -109941049,44 OOP ***44:4,w9p 1T or4,40o.mo..p009.0 owwwi*4.11.4.*4 4***4036409144049, t40044014 404$I.,. 44,0446006 4*-***441. 99.09Loo+4 4.0 OW; +44 090949 1,01 toreiho4 09604994/s *090 60* 440444041 **NWdot eirbei. 44666049940044 *p/trOwt *WW4*44.494,90p 4444 9.10044490/466,6 4440 *****4#4011101 11.9 MO4940k. 4046160$ tatetperweeal ,01409000099494), 490910eti01/P0 1.p1rp09000904.1, 109/99,199019PPO 1** .90490044 914 .499/40'44#49999i 490+140,0144 $4,0000 Mane

/NO, -Ow .9t00,9 0 00* 40490444* ON 004+ 009**'4044100 119444404/099 #14«.4.114 09904 *11404010Pyo Itub- #04.1 Oilaitioro lodpi04404'4+- 4+1104,14 104t0449104000490 AI Ito** 40409101449:. Art: Olopmetowoos moww9,944,9wr 1* -if*, lo*u$-01 *movie*. ir!,-.44-01440*. I. eiftsfi444004 4* o4P44g**, 0 lutoW1m40* W 1019009040090 r 4104, Table 6 (continued)

Correlation Coefficients and Standardised Regression Coefficients for 1981-82 Student Questionnaire Scales with Number of Hours in Each type of Otro Camino Activity

Other Theoretically Relevant Outcomes

Attitutes and Social Development School Experiences

Belief in Practical Rebellioui Internal Activity type Rules Involvement Knowledge Autonomy Control Punishment Rewards

Academic r -.062* .027* -.055* w -.045 -.014* .111* -.006 beta -.047 -- -.043 -- -- .099** --

Vocational r .096* -.013 .086* -.044 -.008 -.107* -.009 beta -.031 -- .063 -- -- -.058 --

Clubs r -.012 .067* .007 .007 -.009 -.024 .051 beta -- .085* -- -- -- -- --

Counseling r -.006 .081* -.024 -.009 .025 -.021 .029 beta -- .083* -- -- -- -- --

Participation 81 r .039 .083* .033 -.023 -.041 .043 .052 beta -- .085 -- -- -- -- -- -- -- Note. Participation 1981 is a dummy variable measuring whether or not students participated in 1981 -b2 rather than number of hours. N's range from 447 to 1006, depending on the patterns of missing data for variables in each equation. Each outcome is controlled by 1981 variables that are significantly related to the type of activity. Academic activity controls include 1981 standard test English reading score; English grade; and SAES friend's influence scale. Vocational activity controls include 1981 standard test math and Spanish reading scores; school days absent; school grades for Spanish, English, social science, and science; overall grade point average; and SAES belief, interpersonal competency, school rewards index, school punishment index, self-concept, rebellious autonomy, school attachment, total delinquency, serious delinquency, and victimization scales; self-report of gender; and 1981-82 school grade. Club activity controls include 1981 standard test Spanish language score; school days absent, and SAES self-report of grades, attachment to parents, interpersonal competency, school punishment index, attachment to school, victimization, and self-report of socio-economic level scales. Counseling activity controls include 1981 standard test English reading score; school grade for math and social science; and SAES internal control and rebellious autonomy scales.Participation 1981 controls include 1981 school grade for English, and SAES interpersonal competency, rebellious autonomony, attachment to school, and self-report of socio-economic level scales.

* Coefficient is significant at the p<.05 level. **Coefficient is significant at the p<.01 level.

2 1 3 -342- r Table A

Means and Standard Deviations for 1981-82 Academic Achievement Outcomes for Each Type of Otro Camino Activity

Outcomes for Project Goal I

Vocation School Grades Standardized Tests Self-Reported

Schl. Read. H.S. Have Pa Activity Soc. Span. Span. Engl. Grade Abil. Crud)) Job Wo Types Span. Eng. Math Sci. Sci. Read. Lang. Read. Math

Academic 20.32 2.11 1.52 .12 .48 .2 Mean 1.66 1.69 2.04 1.73 1.74 39.74 34.17 30.16 7.62 .93 .87 .33 .73 .4 SD 1.01 1.05 1.06 1.06 1.02 12.86 10.80 12.38 203 201 200 236 199 19 N 231 230 213 202 195 197 195 196

Vocational 23.38 2.43 1.56 .52 .20 .2 Mean 2.10 2.10 2.24 1.80 1.89 41.32 37.12 30.11 8.21 .98 .83 .50 .52 .4 SD 1.06 1.10 1.07 1.06 1.03 13.48 12.71 10.41 203 209 194 '193 231 192 19 N 229 226 182 124 129 203 202

Clubs 2.25 1.47 .08 .32 .2 Mean 1.70 1.86 2.17 1.86 1.83 39.73 34.07 29.59 18.76 6.87 .85 .89 .28 .62 .4 SD .95 1.00 .96 .92 .93 13.25 11.37 11.58 302 304 308. 302 300 362 302 3( N 354 355 337 326 309 313

Counseling 18.32 2.14 1.47 .03 .32 .: Mean 1.64 1.73 2.06 1.71 1.75 38.34. 33.59 29.82 12.23 6.78 .86 .92 .17 .62 .4 SD .95 1.01 .97 .95 1.00 13.74 11.30 214 210 207 246 208 2( N 244 240 234 234 221 208 209 1208

Participation 1981 21.50 2.22 1.54 .21 .35 . Mean 1.80 1.84 2.16 1.41 1.62 42.03 34.96 29:67 8.04 .86 .91 .41 .66 .1 SD 1.08 1.05 .98 1.03 1.00 14.23 11.91 11.82 116 1116 120 110 112 157 111 1 N 149 148 131 110 115 117

aSee Table 1 for listing of specific Otro Caminoactivities for each category.

bReported for 1981-82 twelfth-grade students only. Table B

Means and Standard Deviations for 1981-82 School Withdrawal and Attendance for Each Type of Otro Camino Activity

School Report Self Report

Percent School With-days nonatten- Skipping Skipping b Activity typea drawal absentc dance school" classese

Academic Mean .042 .024 1.116 .658 1.213 SD .202 .056 .740 1.084 1.012 N 236 235 199 199 202

Vocational Mean .056 .033 1.163 .801 1.162 SD .231 .135 .767 1.140 .960 N 231 230 196 196 197

Clubs Mean .044 .025 1.107 .610 1.107 SD .206 .080 .704 1.017 .883 N 362 362 307 308 308

Counseling Mean .037 .023 1.165 .722 1.122 SD .188 .077 .745 1.090 .903 N 246 244 212 212 213

Participation81 Mean .108 .055 1.133 .743 1.044 SD .312 .159 .773 1.132 .876 N 157 157 113 113 114

aSee Table 1 for listing of which Otro Camino activitiesare included in Each category. Reasons for withdrawal include voluntary and involuntarywithdrawal, transfer to another school, and reason unknown. Percent based on number of days active. Individual item ranges from 0 to 4 with 0 = no days cutin the last four weeks and 4 = more than 10 days cut in the last 4 weeks. eIndividual item ranges from 0 to 5, with 0 = never cutsclasses and 5 = cuts classes almost every day.

-344- Table C

Means and Standard Deviations for 1981-82 SAES Student Questionnaire Scales for Each Type of Otro Camino Activity

Project Objectives

School Self Interpersonal School AlienationAttachment Concept Competency Effort

Academic Mean .346 .762 .645 .637 .583 SD .227 .212 .173 .268 .263 N 177 186 161 175 181

Vocational Mean .299 .809 .700 .735 .623 SD .222 .198 .180 .234 .280 N 179 189 159 180 188

Clubs Mean .314 .775 .671 .683 .615 SD .224 .209 .179 .278 .267 N 272 288 237 276 289

Counseling Mean .316 .770 .684 .700 .584 SD .215 .214 .173 .247 .279 N 185 199 158 183 196

Participation81 Mean .329 .762 .678 .700 .573 SD .216 .223 .168 .242 .282 N 93 101 84 102 106

Note. See Table 1 for listing of which Otro Camino activities are included in each category.

-345- Table D

Means and Standard Deviations for 1981-82 Student Questionnaire Scales with Each Type of Activity

Other Theoretically Relevant Outcomes

Self Report of Delinquencyb Social Relations Attitudes and Soc. Dvlpmnt. Schl. Expe

Attch. Neg. Belief Rebel. In- Activity Total Ser. Drug School Police to Peer in Pract. Auto- ternal Typea Delin. Delin. Usage Victm. Suspn. Cntct. Prnts. Influ. Rules Invol. Knowl. nomy Cntrl. Punish Rew

Academic Mean .069 .046 .119 .086 .158 .021 .642 .141 .778 .217 1.386 .345 .426 .101 .32 SD .117 .118 .192 .169 .366 .144 .262 .157 .209 .167 .479 .322 .209 .193 .28 N 182 181 179 182 183 236 197 177 175 179 179 171 176 183 185

Vocational Mean .056 .034 .106 .060 .086 .004 .661 .103 .841 .185 1.520 .369 .398 .048 .29 SD .116 .109 .195 .143 .281 .066 .256 .141 .178 .139 .449 .319 .204 .145 .29 182 183 178 176 189 190 N 191 191 189 188 186 . 231 193 187 183

Clubs Mean .053 .036 .082 .082 .156 .008 .669 .141 .779 .230 1.436 .388 .423 .092 .31 .32 SD .110 .102 .173 .172 .363 .091 .266 .172 .203 .181 .456 .328 .219 .196 286 287 N 293 293 268 287 276 362 303 290 277 284 281 273 273

Counseling Mean .045 .031 .073 .072 .157 .008 .691 .145 .776 .226 1.465 .404 .429 .085 .29 SD .097 .090 .152 .156 .365 .090 .249 .171 .202 .192 .450 .325 .225 .176 .29 197 N 192 191 191 196 197 246 209 193 185 192 190 182 184 196

Participation 1981 Mean .065 .043 .116 .078 .173 .019 .651 .144 .782 .234 1.507 .367 .395 .124 .3 SD .123 .109 .205 .180 .381 .137 .273 .184 .231 .166 .414 .360 .251 .233 .3C 101 107 107 105 100 98 157 112 107 101 105 101 98 99 102

a See Table 1 for listing of specific Otro Camino activities for each category.

bSee SAES Student Questionnaire Manual for explanation of these variables.

223 Project PATHE: Second Interim Report

D. C. Gottfredson

Abstract

Interim results suggest that PATHE, a school-based program that combines organizationalchange, directservice to high risk youths, and strong management, significantly increased academic performance for targeted individuals. In one ofthe four middle schools, target students reported significantly less delinquent behavior than did the control group.

The program appears to have had a positive school-level effecton student alienation,self-concept, and beliefin rules as well as on school safety, victimization, and teacher morale. Delinquent behavior was lower in all three PATHE high schools in 1982, althoughdelinquent behavior was higher in the control high school. A nonsignificant trend in favor of the program is evident on several additional PATHE goals and objectives.

Implementation data show much variation across program components and across schools in the fidelity ofthe implementation ofthe program. The expected association between level of implementation and success of the program was not,however, observed. Plans forstrengthening the program and the monitoring procedures for the 1982-83 school year are discussed.

PATHE--Positive Action Through cations of the data for program Holistic Education--effectively com- operations. bines school-wide organizational change, direct service to high-risk Major Program Changes youths and rigorous management and evaluation. The program was .devel- Staffing oped by the Charleston County School District, Charleston, S.C., and has The PATHEstaffremainedrela- been operating in seven middle and tively stable during the 1981-82 senior high schools for two years. schoolyear. Bothspecialists at This report updates the previous St. John's High school were replaced interim evaluationreport for the inAugust, 1981, and the Student PATHE program (D. Gottfredson, Concerns Specialist at Rhett Middle 1982). The backgroundinformation school was replaced mid-year... The included in the first report, replacement specialists resembled including descriptions of the their predecessors in qualifications Charleston community andthe PATHE and backgrounds and the staff program, will not be repeated here. changes seem not to have been dis- This report summarizes major program ruptive to the program. A cut-back changes that occurred between Sep- in funding resulted in the lay-off tember, 1981 and August, 1982, of five specialists at the end of reports results of analyses of the 1981-82 schoolyear, and the implementation and outcome data for evaluator position also turned over the same year, and discusses impli- at that point. The effectof the

-347- 4.44On PATHE reduction in staffat the schools needs, researching problems, for the 1982-83 year will be defining objectives, developing and assessed in the final report for implementingplans, assessingpro- this project. The turnover inthe gress and redefining strategies. evaluatorposition wassmooth and The sub-components of the interven- caused no disruption to the program. tion follow. The new evaluatoris knowledgeable about research and evaluation tech- Curriculum Review and Revision niques, and she is skilled in pro- provides a structure for ongoing gram management. review and revision of the curriculum policies. It identifies train- Program Changes ing and material needs, provides for faculty inservice training, and The goals, theory, and objectives operates a "resource room." of the program remainedunchanged duringthe 1981-82 year. One new Didcipline Policy Review and program component-7Student Team Revision provides a structure for Learning--was added and three--the ongoing review and revision ofthe Speakers Bureau,Summer Program to ¶'discipline policies. This includes Access Creative Experiences, and the the establishment and maintenance of M.U.S.C. Career Awareness and Train- a discipline referral system and ing Programs--were dropped. strategies to generate and publicize school and classroom rules. This In Spring, 1982, the program man- activity also identifiestraining agers reorganized the 28 PATHE andmaterial needs, provides for interventions into five major cate- faculty- inservice training, - and gories of interventions. This identifies materials to bestocked reconceptualization ofthe program, in the "resource room." which amounted to combining interventions that had previouslybeen Leadership /Support Teams receive regarded as separate program compo- training in leadership skillsand nent, helped to organize and focus are given the opportunity to parti- programoperations. A description cipate in the planning and implemen- of the PATHE interventions as reor- tation ofschoolimprovement proj- ganized follows. Descriptions of ects. Each team member gains the components are provided only for management experiencein assessing those interventionswhich changed needs, stating objectives, planning significantly or were addedsince and implementing strategies, and the first interim report,or whose assessing progress toward goals. original descriptions required ela- The fourteamsare the Curriculum bor ation. Support Team, Student Concerns Sup- port Team, Student Leadership Team, PATHE Organizational Structure and the Parent Leadership Team.

This school level intervention is Business/Education Partnerships intended to establish and maintain (high schools only) are formed an organizational structurewhich between the school administration facilitatesshared decision making and local businesses. Businesses among community agencies, students, offer management assistance and teachers, school administrators and other resources to the schools. parents inthe management planning for the school. Theintervention Faculty-Administration Team provides training in assessing Building includes strategies aimed

-348- PATHE at involving administrators and school year. Additional teacher teachers in school improvement training and careful monitoring of activities, and providing training the use of the techniques in classfor school staff in areas identified roomsare plannedforthe 1982-83 as weak. school year.

School-Wide Academic Innovations School-Wide Affective Innovations

Some academic services that had These interventions are targeted previously been defined asschool- at generalschool climate improve- wideservices, such as specialist ment. Staff counseling, previously tutoring, "outside" and peer tutor- considered a school-wide intervening,and diagnostic testing/indivi- tion,is now being directed primar- dualized instructionwere focused ily toward target students. The primarily on target students during subcomponents of the school-wide 1981-82. Two otherservices, the affective intervention are the middle school Exploratory program School Pride Campaign, which and the Curriculum Guide development includes activities aimed at improv- and distribution component were ,ing the overall image of the school, taken over bythe school district. and the Extra-Curricular Activities PATHE continued to assist with these program and the Peer Counseling or programs in several schools, but "Rap Sessions" program. they were not major foci of the program. The remaining school-wide "Rapsessions" are sessions in academic components are Study Skills which adults present information on Training, the Reading Experience topics of concern (such as drug use Program, the Test-taking Skills Pro- or teenagepregnancy) to students gram, the Field Trip Program, and and studentsreact to and discuss Student Team Learning. the topic. These sessions were found by the specialists and program Student Team Learning is a set of managers to be an acceptable substi- instructional techniques that trans- tute for peer counseling sessions, latesinto classroom practice sev- which had proven difficult to imple- eral well-known principles of learn- ment. ing. Team members in STL classrooms study and drill together and prepare Career-Oriented Innovations for quizzes orcross-team competitions. Rewards for academic Programs toincrease transitions improvement are given to the hetero- to college and work (highschools geneous student teams based upon the only) include the Career Exploration progress of the team. The tech- Programs - -FACET and the Trident Tech niques have received positive evalu- Fall and Spring Programs, opened to ations for enhancing learning, self- women during the 1981-82 school concept, liking of school and year--end the Job-seeking Skills increasing cross-race and cross-sex Program. friendships (Slavin, in press). During the 1981-82 school year spe- Services to Target Students cialistsin all PATHE schools were trained in STL methods, as were Thiscomponent wasstrengthened teachers in four of the seven in Spring, 1982. The following schools. Teachers in three schools paragraphsdescribe the redesigned tried out the methods in their program. The actual services pro- classrooms before the close ofthe vided to target students during the 0

-349-

20'1.0 PATHE

1981-82 school were much less pendedduring the 1980-81 school intensive and less programmatic than year, were referred to thePATHE is implied by the following discus program by a teacher, or whose sion. The implementation section of achievement test score (the CTBS this report will discuss in detail total test score) fell in the bottom actual services to target students twentyfifth percentof scores for during the 1981-82 year. that school. Teacher referrals were obtained bysending the form shown The target student service compo in Table 1 to every homeroom teacher nent is intended to provide affec in grades five through eleven of the tive and academic services to a sub PATHE and feeder schools. The cri population of "target" students teria listed atthe bottom of the identified as inneed of special form are the same selection criteria services. The idea behind this that were established forthe pro strategy is that each schoolhas a gram at the time of the initial group ofstudents that will not be grant 'proposal. reached by the organizationallevel PATHE components. These students This procedure resulted in pools need highly individualized treatment "of different sizesfor the seven in orderto get themto the point PATHE schools. Thepoolsfor the that they will benefit from the high schools, but not the middle classroom and school level PATHE schools, were of sufficient size to innovations. The direct service constitute target and control groups component is intended to identify of equal size. In the worst middle appropriatetarget students, diag school case, 71% of the pool had to nose their needs, define behavioral be randomly- assigned-to- the -treat treatment objectives, prescribe ment condition in order to fill the appropriate intensive services, 100 treatment slotsrequired as a monitor progresstoward objectives condition of the grant. and frequently reassess the appro priateness of the treatment for Tables 2 and 3 show the results addressing the problem. of a postrandomization check. Only one of the comparisons shows a sig Program Management and Evaluation nificant pretreatment difference between target and controlgroups. The program managers implemented At least one significant difference a true experiment, developed a data at the p<.05 level would be expected base for management information, and tooccurby chance if thegroups established implementation standards were, in fact, equivalent. for eachprogram componentduring the 1981-82year. Each of these Although implementation data to activities enhanced the manageabil be presented latershowapartial ity andthe evaluatability of the breakdown of the experiment in some program. schools, the design was implemented with enough integrity to enable the Evaluation design. StudenEs eli detection of program effects. The gible for PATHE direct service were program managers have taken steps to randomly assigned to target and con monitor the design of the experiment trolconditionsin September,1982 during the 82-83 school yearand from a pool of eligibles made up of have establishedclearer expecta- all students ineach PATHE school tions for level and quality of ser and the sixth grades of each feeder vices to target students. These elementary school who had been sus- program changes, coupled with the

-350 PATHE decision to retain the 81-82 target scores, attendance, suspension, and control groups, should result in expulsion, and disciplinary inci more dramatic treatmentcontrol dif dents. The item content and psycho ferences atthe end ofthe 1982-83 metric properties of all survey school year. measures are described in Chapter 3 of this volume. Management information system. The evaluator established a manage Change from 1981 to 1982 was ment information system which organ assessed forall outcomes measured ized data on level of services to bythe SAES with the exception of all students in theschool. Data delinquencyforthe middle schools collected in this way were used to (the Charleston project had the report on implementation levels dur selfreport delinquency items ing the Fall'82 semester and again deleted from the Spring, 1981 survey for the entireyear (included in fortheir middle school students). this report). The midyear report Bar charts showing' theamount of based onthesedata exposed unex change on each outcome for each pected low levels of direct service school are available (Gavurin, Hybl to target students. It precipitated and Gottfredson, 1982) and have been a major restructuring of the direct used as a formative evaluation tool service component of the program to bythe PATHE project implementers. allow for more intensiveand pro This report discusses only outcomes grammatic target student services. directly tied to project PATHE's goals and objectives. Implementation standards. The third major improvement in the man Tables 4a through 4c show differ agement of the project was the ences between the 1982 and the 1981 establishment of implementation school scores on survey measures standards for each program compo relevant to the project's goals and nent. The program implementers objectives. Tables 5 and 6 show defined the standards and used them change from baseline to 1982 on to monitor the level of implementa measures of goals and objectives for tion of each program component. The which wehavedatafromofficial ratings derived from these standards school records. Tables 7 through 10 will be discussed in the implementa show means for target and randome tion section of this report. quivalentcontrol studentson all goals and objectives. The discus Effectiveness sion whichfollows willsummarize the information in these tables. The effectiveness of the PATHE program is assessed attwo levels: Differences from 1981 to 1982 and Comparisons of 1980-81 to 1981-82 between the target and control stu school year school averages and com dents are generally not large. parisons of randomly assigned treat Although few reach statistical sig ment and control students on meas nificance according to conventional ures of PATHE goals and objectives. standards, some consistent trends warrant discussion. Throughout the Measures are taken from the SAES following pages I will consider the student and teacher surveys (Gott direction of the observed difference fredson, Ogawa, Rickert, & Gottfred as well as the probability of son, 1982; Gottfredson, Gottfredson, observing such a difference if proj & Cook, 1983) and Charleston County ect PATHE is not effective. SchoolDistrict records ofgrades, California Test of Basic Skills 351 PATHE

tstatistics forcomparisons of cally significant. The high school target and control groups are based control school experienced an on the observed means and standard increase in suspensions and in deviations for each group on Spring, selfreported delinquent behavior. 1982 measures of goals and objec tives. tstatistics for comparisons Progress in the middle schools on of 1981and 1982 student composi this outcomeis more difficult to tional measures (i.e., school aver assess: The selfreport delinquency ages on characteristics of the stu items were deleted from the Spring, dents) arebasedon the observed 1981 middle school survey, and school means and standard deviations aggregate level suspension data from for each year. For comparisons of officialrecords(Table 5)donot 1981and 1982 school climate meas accordwith selfreported suspen ures (i.e., student and teacher sions fromthe survey. (Table 4a). reports ofthe school environment), Official records show an increase in the "tstatistic" is the ratio of suspensionsfor threeof the four the difference between 1982 and 1981 PATHEmiddle schools as well as scores to the standard error of their comparison school. Selfre measurement ofthe 1982 scores for " ports suggest that suspensions were all but St. Paul schools in the down in two of fourPATHE middle OJJDP initiative. Climate measure schools as well as in the comparison differences that exceed twice their middle school. standard error of measurement may be regarded as dependable. Alternativemeasures of school disruption shown in Table 4c show an The following paragraphs use increase instudents'- and teachers' information aboutthe direction of perception of safety and a decrease the difference score to assess in student reports of victimization school level effects of the PATHE in six of the seven PATHE schools. program. The reader should be aware Several ofthese changes were sta that some of the differences are tistically significant. Both com close to zero, but are still inter parison schools experienced preted as improvements or regres increased student victimization but sions, depending on the sign. students in one and teachers in both comparison schools reported more Delinquency school safety.

School level. Measures of delin Individuallevel. Treatmentcon quent or disruptive behavior--sus trol student comparisons on measures pensions and selfreported delin of delinquency (Table 7) show little quency--show the same pattern of evidence of a program effect on this results for high schools (Tables 4c outcome. Although one of the seven and 5). Each of the threePATHE schools showed a sig, ificant reduc high schools experienceda decline tion in delinquency among target from 1980-81 to 1981-82 in both the students, the direction of the number of suspensions and selfre treatmentcontrol difference favored ported delinquent behavior. For St. control students about as often as John's high school,the reductions target students in the other in Drug Useand TotalDelinquency schools. Comparisons of suspensions were statistically significant. For anddisciplinary incidents in the the other two high schools, the school show similar results. decline in suspensions was statisti-

-352

229 PATHE

Attendance sistent statistically significant increasein test scores for Burke. School-level. Attendance rates Evidence from many sources--ratings in the Charleston County schools are of the strength and fidelity of typically high (in 1980 the average implementation and observations and daily attendance rates for the PATHE interviews with the Burke and control schools ranged from 89 specialists--show that Burke imple- to94percent). Ceiling effects, mented the school-wide academic com- the initiationduring the 1981-82 ponents of the program with much school year of a strict district- more fidelity to the program plan wide attendance policy, and the fact than did the other PATHE high that most school personnel (includ- schools. ing PATHE specialists) do not perceive nonattendance to be a problem, Individual-level. Other evidence all reduce the likelihood of prograM favoringapositive effect of the effects on attendance rates. Tables PATHE'program on achievement comes 4c and 5 show that attendance did in from target-control comparisons fact increase inall three PATHE (Table 9). Comparisons of the per- high schools and one or two of the ".,cent scoring in the bottom national PATHE middle schools (depending on quartile of the 1982 CTBS show that the source). in only one school did the control students fare better than the target Individual level. The targetstudents. The overall treatmentcontrol comparisons are promising control difference reaches statisti- for both high and middle PATHE cal significance in Language, an schools (Table 8): Although the area that several PATHE specialists differences never reach statistical focused on during the year, as well significance, target students in as for the total battery. The five of the sevenschools have a direction ofthe difference favors lower proportion of unexcused days the target students for all subar- absent to totaldays enrolled than eas. Target-control comparisons of do control students. The other two schoolgrades alsofavor PATHE in schools show no difference between six ofthe seven schools,and the target and control students. difference for percent receiving failing grades averaged across all Academic Achievement subject areas reaches statistical significance. School-level. All Charleston schools, PATHE and control, improved Transition toEmployment and Post- from 1981 to 1982 on California Test secondary Education of Basic Skills scores, andthis improvement was more pronounced in This outcome will be assessed in the middle than in the high schools the final report, but see the "Edu- (see Table 6). The extent to which cational Expectations" objective project PATHE contributed to the below. improvement is unclear, but the evidence favors PAM: Byand large PATHE Objectives the improvementin Charleston high schools on CTBS test scores was min- The PATHE managers identified a imaland did not reach statistical number of intermediary outcomes significance. One PATHE high toward which the program isaimed. school--Burke--was a clear exception These intermediary outcomes, or to this rule. Table 6 shows a con- "objectives," are primarily student

-353-

230 PATHE and teacher attitudes, but also Student Reports of Rewards and Pun- includecertain teacherbehaviors. ishments in School Relevant comparisons for each of the PATHE objectives are shown in Tables Studentsin fivePATHE schools 4a, 4b, 4c, and 10. Because many of reportedan increase in rewarding the objective measures are somewhat and a decrease in punishing experi- redundant, I will summarize only the ences in school from 1981to 1982. major dimensions here. Both comparison schools experienced a decrease in these intermediary Teacher Morale (Teacher Reports) outcomes. Of the two PATHE middle schools with increases in students' Every PATHE school and the middle reports of school experiences as school comparison school experienced punishing, one was already extremely an increase in teacher morale. The low on this measure. high school comparison school declined in teacher morale. Tre'atment/controlcomparisons of rewarding and punishing experiences Reports of Planning & Action and in the middle schools showed differ- Smooth Aministration t'ences favoring the treatment students in three of the four schools. Both teachers and students Onecomparison reached statistical reported on the extent to which significance. In only one of the their school is characterized by three high schools did the direction planfulness and innovative atti- ofthe treatment/control difference tudes, and teachers reported on favor the treatment students. their perceptions ofthe extent to - - which the school is run smoothly and Students Reports of Fairness and effectively. According to students Clarity of Rules in four of the seven PATHE schools, the degree of planning decreased Three of the PATHE schools from1981 to 1982. Three of the increased on both measures, and fourschools that did notimprove another twoincreased on students' were already above the 85th percen- perceptions of the clarity of the tile for schools in the study, rules, though not the fairness. Two though. Students in both comparison of the seven PATHE schools declined schools reported a decline in plan- on both measures, and both compari- ning. son schools declined on both measures. The magnitude of the changes Teachers reported an increase in from year to year is noteworthy. planning andin smooth administra- The two PAM schools that declined tion in five ofthe PATHE schools. from 1981 to 1982 on the Clarity of Both comparisons schools declined on Rules scale declined only slightly both of these teacher measures. (in fact, the difference score equals zero when rounded off). The Teacher Expectations for Student decline experienced in the compari- Achievement (Teacher Reports) son schools was greater. The major PATHE activities aimed at increasing The direction of change was posi- rule fairness and clarity occurred tive in four of the sevenPATHE during the 1980-81 school year. The schools--two middle and two PATHE discipline referral system was high--and in one of the two control instituted during that year, and schools. several PATHE schools had school- wide campaigns to generate and pub-

-354- PAMIE

oho* 4.4aligoornms rules. ILItti.20111 *Lift op 410144 TOO. the 1114%a )01: :ileouarti samowdolaw, £11butnee of as Charleston 1114414 4,10.4 Olkil 434)4'44Oat school sad controlscored 144 tow iottliiapoolas*efte1PIlea above the Si* mai five scored above polmcmosi pests ersi54tion. tiff;;." the 94th percentile on involvement 0000, Ots.* sose ors, for all imbeds La the study on the Serino MI survey. Although 14911VONOLUMPitilliatt improve meet on this measure is reliable for the Charleston

444 Oet** WOW* echoed* am4 eve 4. stools* three ram and one cooper- 0440t itotaOreeses4 twood em this isms school did improveon this ootoose. Soh oemperteemschool* meovre. It is difficult to deter- thotShme4. t

-Mt

232 PATHE

Attachment to School stands out on all measures of implementation for school-wide activi- The program had positive effects ties, and its improvement from 1981 on this key, intermediary outcome in to 1982 on PATHE goals and objec- the middle schools. All improved on tives was second only to Haut Gap, this outcome while the comparison the middle school which also imple- middle school declined substan- mented the program with excellence. tially. Only one of the PATHE high How might Burke's negative results schoolsimproved onthis outcome, be explained? There are several but of the twothat declined one possibilities: started out and remained above the 1) The control students received 94th percentile. Middle school tar- more treatment than the target students reported higher levels get students, of attachment in every school, but 2) The "worst" control students only in one high schooldid this dropped out of school while comparison favor the program (in a their equivalent target stu- second high school. there was no dif- dents remained in school, ference). The high school with the thus making the groups none- highestschool-wide attachmenthad ! quivalent (because dropouts an apparent negative effect on its were not included in the data target students (p<.01). collection), or the "worst" control students did not take Educational Expectations the spring 1982 survey, and 3) The treatment was harmful. Students in two middleschools Each will be discussed in turn. and two high schoolsin the PATHE - - - - program reported increased educa- Table 11 compares the total num- tional expectations. Students in ber of contacts with specialists one comparison school also reported recorded for the 1981-82 school year increased expectations. In only one for Burke High School and all other treatment/control comparison did the PATHE schools. It shows that con- direction of thedifference favor trol students did not receive more the PATHE target students. services than treatment students at. Burke. This istrue also forthe Belief in Rules subset of targetand control students who took theSpring. survey. All but one of the Charleston The PATHE specialists in the school schools(a PATHE school)increased admitted that theymay have been onthis measure. Roughly half of more careful to record contacts with the treatment/control comparisons target than control students because favored the PATHE students. theywereconcerned with"getting credit for" their target student A Closer Look at Burke contacts. The underreporting of control student contacts would have The mostsurprisingfinding of had to have been blatant in order to this interim evaluation is a consis- explain the differences on outcomes tent trend toward a negative effect favoring control students, though. on target students in Burke, one of the PATHE high schools. The result The secondhypothesis, that of is doubly troubling because the differentialattrition from school Burke operation is regarded as the or from the survey administration, closest approximation (amonghigh is also not supported bythe data. schools) to the ideal implementation The "original" and "final" columns ofthe PATHE program. The school

-356-

o33 PATHE

in Table 2 show no pre-treatment high numbers of contacts withthe difference between CTBS scores for specialists. Although the distribu- target and control students who left tions for most ofthe students in school before Spring, 1982 when the Burke and other schools are similar, CTBS test was administered. Accord- the top 5% of the distribution ing to thesefigures, dropout did received between 45 and 60 contacts not change theequivalency of the at Burke and only between 18 and 45 target and control groups. Similar contacts at the other schools. Tar- analyses comparing 1981CTBS meas- get students were more likely to be ures for target and control students among those students receiving who took the Spring, 1982 survey extreme amounts of service. The alsodisconfirmed thedifferential hypothesis that a high amount of attrition hypothesis. Among stu- contact with specialists causes tar- dents who tookthe 1982 surveyat get students to dislike school was Burke, control students did not have examinedand disconfirmed. Number a pre-treatment "advantage" over of contacts with specialists is target students. slightly positively related to attachment to school for target stu- Might the treatment at Burke have dents (r=.07, p=.08) and slightly caused target students to report negatively related for control stu- less attachment to school, score dents (r=-.09, p=.06) for all PATHE lower on achievement tests, have schools. The associations are lower expectations for completing stronger at Burke, reaching statis- high school and have fewer jobs than tical significance at the p<.05 equivalent students who received one level (target: r=.21; control: third as much treatment? Observa- r=-.28). Target students who tions of the Burkespecialists in received extreme amounts of service action imply that the opposite liked schoolmore, not less, than should occur. Both specialists are those receiving less service. experienced teachers, have excellent rapport with the students,teachers One final difference between the andadministrators in the school, way direct services are provided at are perceived as extremely hard- Burke and other schools is notewor- workingindividuals, areadvocates thy. Tables 15 through 21 show that for thestudents while being firm Burke is the only PATHE school which disciplinarians,and aboveall are provided more academic than affec- dedicated to their work. Their tive types of services to target teaching style seems to accord with students. The difference may have that suggested byresearch: Their resulted from differences in record- instructional pace is rapid and ing practices across schools, but their tutoring group size is always observations and interviews with the sufficiently small to allow for specialist at Burke also suggest individualized attention. that the emphasis is on academics. The Burke specialists frequently Two hypotheses about the way the work as a team with small groups of PATHE treatment was administered at students, tutoring and providing Burke were examined: Differences counseling as needed. Tutoring between Burke and other schools in seems to be the major focus in these the standard deviation for number of small groupsessions. PATHEspe- contacts (see Table 11) raises sus- cialists at other schools work more picion. The frequency distributions independently and schedule entire for totalnumber of contactsshow sessions solely for counseling. that at Burke a few students (both treatment and control) had extremely

-357 234 PATHE

The hypothesis that the ratio of School-level academic to affectiveservices is central to the success of the PATHE Comparisons of 1981 to 1982 programis currently being tested. changes in school averages on This ratio is a variable central to PATHE's goals and objectives must be the PATHE philosophy: The program approached with more skepticism than rests on thenotionthat students comparisons of randomly assigned need both academic and affective treatment and control students. support. The program managers chose Even when improvement is observed to compromise this notion when they from year to year we cannot be sure responded to the federal budget cut that the PATHE program caused the by severely curtailing affective improvement. Schools were not ran- services in half of the PATHE domly assigned to treatment, and we schools for the 1982-83 school year. have only two Charleston schools with which to compare the PATHE Summary of Outcome Evaluation schools.

The effects of the PATHE program School-wide improvements on PATHE differaccording to target (i.e., goals and objectives are generally treatment-control comparisons vs. more evident at the high than at the school-level change) and level middle school level, although the (i.e., middle vs. high schools). "most improved" school isa middle school. PATHE high schools improved Target Students on80%of the outcomes examined, while the PATHE middle schools Program effects ontarget stu- improved on 69%.- -A1:1-three- PATHE dents are more evident at the middle high schoolsand two of the four than at the high schoollevel. In PATHE middle schools improved siAni- general, target students in the ficantly on several of the outcomes PATHE high schools seem not to have examined in Tables 4a and 4b. benefited from the program any more thandid thetypical, nontargeted The Sign Test (Siegal, 1956) can student in the school during the be applied to the results in Table 1981-82 school year. Theresults 4c to determine the probability that for academic achievement are an the observed pattern of changes exception, though: Target students could have resulted by chance. The in all PATHE schools save one high direction of 1981-1982 change is school fared better than their ran- treated as a binomial variable with dom-equivalent controls on this out- aprobability of improving of .5. come. Target students in all middle Several of thefindings discussed schools attended school more often aboveare found to be improbable and out-achieved the controls,and when this test is applied. For they reported higher levels of example, the observed decrease in school attachment and more positive studentalienationin every PATHE self-concepts. Only for the school and increases in teacher achievement outcomes do the differ- morale and student positive self- ences reach statistical signifi- concept in every PATHE school would cance, though. occur by chance only eight times in one thousand. The patterns of increases and decreases for students' and teachers' reports of safety, student victimization and belief have only a .06 probability

-358- 235 PATHE

of occurring by chance. The PATHE Program Implementation school improvements for teacher reports of smooth administration and Information onlevelof program planning and action, student reports implementation comes fromtwo main of rewarding school experiences, sources. "Intensity" standards, clarity of rules, interpersonal com- (standardsfor the amount of ser- petency, and attachment to school vices provided by PATHE personnel), also have a low likelihood of occur- and "fidelity" standards (standards ring by chance given that both com- for the quality of the services ren- parison schools regressed on these dered) were developed by the program measures. All of the above-men- managers to assistin project man- tioned findings favor the PATHE pro- agement and evaluation. Each school gram. None ofthe variables exam- in the PATHE program was rated for inedin Tables 4a through 4c show intensity and fidelity -twice during disadvantages for the PATHE schools. the 81782 school year, once at mid- These probabilities and those year and again at theend of the reported in the following paragraph school year. A fivepoint scale are not corrected for possible ceil- rangingfrom zero (doesnotmeet ing effects which, as I mentioned in standard/not implemented) to 4 the previous section, sometimes make (exceedsstandard) wasused. The improvement from 1981 to 1982 impro- ratings were based upon documented bable. contacts with students, teachers and parents,minutes andagendas from A gross measure of program effec- meetings, logs of resources used and tiveness is simply the ratio of out- interviews with PATHEspecialists. comes on which a given school The standards are shown in Tables12 improved from 1981 to 1982. If we and 13, and the averages of the two assign a score of one to each out- ratings for each school are shown in come examined in Tables 4a through Table 14. 4c that moved in the positive direction and a score of zero to those The second major source of pro- that did not, and thenrank the gram implementation data for 1981-82 seven program and two comparison is information about the nature and schoolsfrom lowest to highest on duration of PATHE specialist-student the degree of positive change from contacts. Thesedata comefroma 1981 to 1982there is no overlap management information system devel- between the average rankings for the oped by Ann Birdseye, the former comparison andthe PATHEschools. PATHE evaluator. Data from contact That is, the average improvement for logs and sign-in sheets were entered the highest-ranking comparison into computer files and reorganized school is lower thanthe average to facilitate analysis. Tables 15 improvement forthe lowest-ranking through 21 compare levels of treat- PATHE school. PATHE schools ment for target and control students improved on 64% to 82% of the out- and for the entire school. Several comes in Tables 4a through 4c with points can be drawn from these an average of 74% improvement. The tables: comparison schoolsimproved on 24% and45% of the outcomes with an Experimental design breakdown. average of 34%. The experimental design was not upheldin most schools. The most severe breakdowns occured in Courte- nay and Rivers middle schools, where controlstudents received close to

-359-

236 PATHE one half as much treatment as target interventions lagged behind. students. In both of these schools, According to the ratings, the middle control students received more schools implemented the program with direct services than did other non- less fidelity than the high schools. targeted students in the school. In other words, it appears that control The most vigorously implemented students were singled out for treat- components includeStudent Leader- ment or that the specialists in ship Teams and Curriculum Review and these schools had special difficul- Revision for all schools, and Parent ties withholding treatment from con- Leadership Teams, School Pride Cam- trol students referred for service. paigns, Field Trips, Reading Experi- Control students in Burke also ence Program, Job-Seeking Skills and received more treatment than other Faculty Inservices for high schools non-targeted students,and received only. The least well-implemented about one third as much direct ser- components included services to tar- vice contact as the target students. get students and extracurricular All otherschoolsexceptfor Haut activities (allschools) and Busi- Gap treated control students one- ness-Education Partnerships, Peer 1p fifth to one-third as much as the Counseling, Tutoring, Job-Seeking targetstudents, but these ratios Skills and Faculty Inservices (mid- would beexpected if control stu- dle schools only). dents received onlythose services directed at all students in the Caution shouldbeexercised in school, which was in accord with the interpreting the implementation experimental design. data. Only a moderatedegree of correspondence between outcome meas- Intensity of direct services. ures and implementation levels was The level of direct services to tar- ,:served: School-wide academic and get students did not reach accepta- affective components were imple- ble levels in most schools. Accord- mented best, and the outcome data ing tothe intensity standards set show most improvement in these by the program managers, target stu- areas. By and large,however, the dents wereto have at least eight schools which would be expected, on contacts with the specialists during the basis of the implementation the year. Rhett exceeded this stan- data,to show the most improvement dard but was the only PATHE school in certain areas often did not. The to meet it. Haut Gap and Brown fell moderate level of correspondence is seriously short with less than half explained by two factors: The proj- of the required number of contacts. ect never established a workable system for collecting data on some Overall program accomplishments. PATHE activities--notably extracur- On the average,PATHE schools met ricularactivities andschool-wide about 75% of the program standards. activitiessuch as the study and The academic interventions were test-taking skills programs. Also, implemented most faithfully and the the intensity and fidelity standards services to targetstudents least were sometimes unrealistic and were faithfully program-wide. The inten- not well-understood by the PATHE sity and fidelityratings suggest specialists. This problem was cor- that the team structure and the rected forthe 1982-83 year. The career interventions were imple- standards are now more realistic and mented almost up to standard in the have been communicated to all spe- high schools, and that the affective cialists.

-360-

237 PATHE

Discussion techniques on information about the child's background may in fact serve The establishmentof an experi- to foster low expectations. mental design,implementation standards and a system for monitoring An example of subtle undermining program activities during the of the program objective comes from 1981-82 school year strengthened the an interview with one PATHE special- PATHE programand its evaluation. ist who reported frequently taking Theprogram wasmorefocused and teachers aside and "filling them in" systematicallyimplementedthan in about the child's home life so that its previous year. Specialists the teacher might react more appro- /reported that the monitoring system priately to thechild's behavior. facilitatedtheirjobs because it Although offering this type of helped them to organize and priori- "assistance" is probably not an tize activities and to manage their uncommon practice among counselors, time better. it certainly is not in accord with the PATHE model. Educational expectations did not improve. One objective central to Projected Program Improvements the PATHE program was not met during the 1981-82 year. The expectations The program managers have already of studentsand teachers for the taken steps to improve the program students' academic success did not in areas identified as weak during increase. In 1981, teachers in the 1981-82 school year. The most PATHE schools reported that between dramatic change is in the target 28and 39% of the students they student- service -component : Blue- teach are low ability students. prints for services totarget stu- Between 55 and 77% ofstudents in dents have been developed and shared these same schools reported that with the specialists, and these ser- they expected to continue some type vices are monitored carefully by the of schooling after high school. The program manager and evaluator on an 1982 figures are roughly the same. ongoing basis. Early signs from the Target-control student comparisons first quarter of the 1982-83 school on this outcome also tended toward year suggest that target students the negative: Controlstudents in are receiving much more service of a five of the seven PATHE schools had higher quality than they were last higher educational expectations than year. This improvement also seems did the target students, and the tobe solving the problem of the difference reached statistical sig- experimental design breakdown: The nificance in one school. increased expectations for services to target students made it necessary Although the largest school-wide for specialists to establish treat- gain inthis area was observedin ment schedules for students. In Burke--the school which implemented order to keep up with the schedules, theschool-levelcomponents of the specialists are withholding services the program with most fidelity to to nontarget students more often. the PATHE theory, the absence of evidenceof overall program effec- An improved system for monitoring tivenessin this area suggests the program activities was developed and need for better-defined program com- implemented in Fall, 1982. The new ponents aimed at increasing expecta- system identifies problem areas tions. The "individualized" earlyso that the program manager approach taken by many educators and can work with the specialists to the reliance of many counseling

-361--

238 PATHE correct implementation breakdowns in PATHE goals and objectives are gen- a timely fashion. Early evidence erally more evident at the high than alsosuggests that this system is atthe middle school level. PATHE strengthening the PATHE program. high schools improved from 1981-1982 on80% of the outcomesexamined. Conclusion PATHE middle school improved on 69% on the average. Interim results of the evaluation of Project PATHE suggest the follow- 6. Every PATHE school improved ing: from 1981-1982 on student Alienation and Positive Self-Concept and 1. Program effects on high risk Teacher Morale. Six of the seven students targeted by the PATHE pro- PATHE schools improved on students' gram are more evident at the middle reports of Victimization, and than at the high school level. Safety., Belief in Rules, and teachers' reports of Safety. 2. Target students in all middle and two high schools scored higher 7. Every PATHE high school exper- on standardized achievement tests ienced a reduction in delinquent and received higher grades than did behavior and suspensions from random control students. These 1981-1982. For one the reduction in results reached statistical signifi- Delinquency was statistically signi- cance. ficant,and forthe other twothe reduction in the number of suspen- 3. Target students in all four sions was statistically significant. middle schools attended school more often, reported higher levels of 8. School-wide academic interven- attachment to Pchool, and more positions were implementedwithmost tive self-concepts than did the con- integrity during the 1981-82 year. trols, although' these differences The highschools generally metor did not reach statisticalsignifi- exceeded program standards inthis cance. area, andthe middleschoolsmet about 75% of the standards. 4. Target students in one middle schoolreported significantly less 9. Target student services were serious delinquent behavior than generally weak, compared to program random-equivalent control students. standards. Onlyone middle school met the intensity standards estab- School-v.de improvements on lished by the program managers.

-362- 23D PATHE

References

Birdseye, A. T. Evaluating the Implementation of a Delinquency Prevention Program: The PATHE Experience. Paper presented at the Annual Meeting of the American Educational Research Association, New York, March, 1982.

Gavurin, S., Hybl L., & Gottfredson, D. C. Project PATHE: Interim evaluation results. Baltimore: Center for Social Organization of Schools, 1982.

Gottfredson, D. C. Interim evaluation of project PATHE--Charleston. In Gott- fredson, G. (ed.), The School Action Effectiveness Study: First Interim Report. Baltimore: Center for Social Organization of Schools, 1982.

Gottfredson, G. D., Gottfredson, D. C., & Cook, .M. S.(eds.). The School Action Effectiveness Study: Second interim report. Baltimore: The Johns Hopkins University, Center for'Social Organization of Schools, 1983.-

Gottfredson, G. D., Ogawa, D. K., Ricker, D. E., & Gottfredson, D. C. Stu- dents and teachers in context: The measures used in the School Action Effectiveness Study. In, Gottfredson, G. (ed.), The School Action Effec- tiveness Study: First Interim Report. Baltimore: Center for Social Organization of Schools, 1982.

Siegal, S. Nonparametric Statistics for the Behavioral Sciences. New York: _ _ McGraw-Hill, 1956.

Slavin, R. E. Cooperative Learning. New York: Longman, in press.

-363-

4,4u0,1:1 Ch3rleston Table PA' Student Selection Form

00 you think this

'student should be I PATHS SELECTION CRITERIA

13 PATH!: target 1 (check all criteria that Istudent next year?' apply to each student:-)

I NO NAmo I I YES I I'D 1112131415 1 61'

4. « 4. 4-4.---4---«__-4.---4.---+- (students' names were preprinted here) ;.-11 :::is., list ,t11 ctner stui,cnts in your homeroom not included in atove list:

I I I I I I I 1 I I .1

4., 4.- 4. * « 4. « -«___«_.

I I I I I I I I 1 I I ' « « « « - - 4.- « ---4._. .

I I I I I I I I 1 1 I .

4. 4. « «.--4.---4._--4.---«---4.---4«.

I 1 I I I 1.1 I 1 1 I

« « 4. « « « -«---«---«---«-.

I I I I I I I.I . I I I I « , « - « « « « -4. « « «-. L'1,.:Ise list ..,,ny other stuents not in your homeraom who....should he selected as PAT(

1-..:Arvi' stud,..ints next year: .------« « « « «- -4.---4.---«-.

I I :I 1 1 1 1 ! 1 I I « « « « .: M +.

I I I I I I I I 1 I I « « « « « « -4. _4.-__4__-4.-

I I 1 I I I I I I I-I c « « « « -4. « 4. «

I I I 1.1 I I I I I I « « « « « «- « -« « 4.---4.-

I I I 1 I I .1 I I I I « « « 40,!umbers one throu.p.seven reFer to the-followingPATHEtsqlection'criteria: 1* previous expulsion.' suspension. or involvementin a number- of .disciolinary actions 2. social promotion or previous retention 3. decline in performance level over past :3,school years . 4. previous request for guidance orcounseling'due to a s9ecial need or proble.m: F. tendency to cut class (truant) 6« difficulty with the .law (delinquent) 7. referral by a school/community agency 241 Table 2

Spring 1981 California Test of Basic Skills Standard Scores

for Target and Control Groups, by School

Total

Reading Math LLEPEL Original Final

x SD N x SD N x SO N x SD N x SD N

Courteney

Target (N=83) 376 68 79 389 56 78 402 65 79 363 68 78 365 68 74

Control (N=80) 374 62 77 388 53 76 397 60 77 361 60 76 362 60 73

Rhett

Target (N.62) 415 58 60 407 55 61 423 65 60 390 62 60 389 63 55

Control (N=54) 406 64 50 405 42 49 420 59 50 388 51 49 38952 46

Rivers

Target (N=86) 400 65 77 396 50 79 414 59 77 380 54 77 382 53 68

Control (N=75) 399 70 65 400 43 66 419 63 66 383 56 64 382 59 56

Haut Gap m Target (N=65) 407 69 59 401 46 58 422 60 59 387*58 57 385 54 51 0 Control (N=23) 378 48 21 378 51 21 392 57 21 354 53 20 358 47 18

Brown

Target (N=83) 449 77 73 456 70 70 466 69 73 445 75 70 448 74k58

Control (N=80) 450 68 71 455 66 70 457 66 71 440 69 70 439 68 4 61

Burke

Target (N=80) 442 82 79 424 63 74 444 78 79 414 69 73 418 72 61 420 68 57 Control (N=77) 436 81 75 424 62 71 i 441 71 75 414 71 71

St. Johns

Target (N=78) 469 87 65 458 78 64 466 74 66 45083 62 449 85 43

Control (N=76) 464 69 64 453 71 62 454 72 65 442'73 60 430 69 48

All Schools 410 Target (N=540) 422,79492 418 66 484 434 71 493 403 74 477 . 402 73

Control (N=464) 419 75423 418 63 415 430 68 425 402 71 410 400 69 359

Difference between target and control groups is significant at the p<.05 level.

a Figures under the "original" column represent group averages for all 1981-82 target andcontrol students who

took the CTBS test in Spring, 1981.Figures under the "final" column represent averages for all target and control students who took the Spring, 1981 CTBS test as well as the Spring, 1982 CTBS test. Total N's (in24i parentheses) are the number of cases initially randomized into treatment and control conditions. 242 Table 3

Means and Standard Deviations for Teacher Referrals to PATHE and Number of Suspensions, 1980-81

Number of Numbe'r of Referrals of to PATHE Suspensions Nr SD X SD

Courtenay Treatment (N =83) .61 .49 .06 .24 Control (N=80) .54 .50 .08 .31

Rhett Treatment (N=63) .46 .50 .25 .57 Control (N=52) .58 .50 .17 .43

Rivers Treatment (N=85) .53 .50 .25 .53 Control (N=75) .63 .49 .12 .40

Haut Gap Treatment (N=64) .50 .50 .12 .33 Control (N=23) .39 .50 .00 .00

Brown Treatment (N =81) .63 .48 .63 .86 Control (N=78) .50 .50 .58 .75

Burke Treatment (N=86) .20 .40 .24 .51 Control (N=80) .18 .38 .31 .61

St. Johns Treatment (N=78) .46 .50 .77 1.02 Control (N=76) .43 .50 .88 1.36

All Schools Treatment (N=540) .48 .50 .34 .68 Control (N=464) .46 .50 .35 .78

Note--Total N's (in parentheses) are numbersof cases initially randomized into treatment and controlconditions.

-366- 2,4 Table 4a

1981 to 1982 Difference Scores for School Averages on Survey Measures of PATHE Goals and Objectives--Middle Schools

Laing Courtenay Rhett Rivers Haut Gap (control)

Goal or objective measure diff t diff t diff t diff t diff

Total Delinquency (-) Serious Delinquency (-) Drug Use (-) Suspensions (-) .01 .19 -.08-2.62 .01 .37 -.01 -.18-.03 -.96 Victimization (-) -.03-1:61 -.01 -.86 -.02-1.33 .02 1.69 .00 .05 School Nonattendance (-) .02 .57 .03 .67 .00 -.00 -.03-1.06 .01 .27 School Rewards (+) .02 .71 .02 .60 -.02 -.88 .02 .65 -.03-1.19 School Punishments (-) -.01 -.30 .02 .98 .02 .71 -.02 -.87 .05 2.45 Interpersonal Competency (+) .02 1.29 .05 2.68 .02 1.15 .03 1.31 -.02 -.88 Positive Self-concept (+) .00 .26 .02 1.04 .03 1.69 .04 2.18 .02 1.52 Alienation (-) -.01 -.36 -.07-2.97 -.03-1.21 -.05-1.03 .00 -.15 School Attachment (+) .00 .05 .02 1.17 .01 .27 .04 1.76 -.05-2.65 Educational Expectations (+) -.06 -.46 .01 .11 -.14 -.92 .29 1.92 .10 .69 Belief (+) .00 -.11 .03 1.39 .03-1.26 :05 2.32 .04 1.85 Low Expectationsa (-) -4.17 -.72 .30 .04 .90 .17-5.51 -.97-10.99-1.57 Involvement (+) .02 1.41 -.00 -.10 -.03-1.64 -.01 -.49 .01 .41 Victimizationa (-) -.01 -.25 .00 .07 -.01 -.32 .02 .29 -.00-.10 Safetya (+) .34 2.28 .32 2.17 -.03 -.19 .07 .49 .12 .83 Teacher Moralea (+) .07 1.26 .09 1.67 .05 1.03 .15 2.78 .00 .07 Planning and Actiona (+) -.02 -.24 .06 .87 .10 1.44 .03 1.17 -.07-1.01 Smooth Administrationa (+) -.02 -.34-.10-1.68 .06 1.03 .15 2.46 -.09 -.31 Rule Clarity (+) .00 -.10 .01 .23 .02 .51 .02 .47 -.04 -.90 Rule Fairness (+) -.01 -.22 .02 .37 .00 .03 -.01 -.10 -.04 -.67 Planning and Action (+) -.02 -.34-.05-1.01 .03 .73 .00 .10 -.06-1.30 Safety (+) .04 1.44 .03 1.01 .05 1.70 .00 .06 .02 .76

Pct. measures improved .68 .f8 .64 .82 .45

Note. The desired direction of change is indicated in parentheses after each scale name. t-statistics for compositional measures (Total Delinquency through Involvement) are based upon 1981 and 1982 means and standard deviation's for each school. For psychosocial climate measures, the "t-statistic" is the ratio of the difference between 1982 and 1981 scores, to the standard error of measurement of the 1982 scores for all non-St. Paul schools in the OJJDP initiative. As a rule of thumb, psychosocial climate measure differences that are twice the standard error of measurement may be regarded as dependable. For compositional measures, t values of 1.96 and 2.54 are statistically significant at the p<.05 and p<.01 level, respectively. All measures are taken from the SAES student surveys unless otherwise indicated. Dashes indicate that the goal was not measured in 1981.

a Measure taken from SAES teacher survey

-367- %Isle 4h

)$4 i* Oittotooto Stereo for School Averages 044 Nolk-voe useeetes et tent Coale and Ohjetlives--1110 Schools

ttss s s.m.ssimissacusess Charleston Orates Berke It. Johns (control) MMUWW 1114G16110110.11114.

644114 tithiswei is* lettMovie gift t dttt t dill t diff t 31,407.42,1:ZiZAIAZtat,l,Z4,111141L11,111.3111,A1.711P.r,S,SMONaPaPoSultirligio

*stet Optiteheessev 4.1 a.415 -1.92 -.01.4.041 -.02-2.05 .00 .09 dot*** Opiteesesoba 4.1 ,00) 441 -.01-1.12 -.01 -.72 .00 .40 Oesei tee 121 =.03 -1.04 .01 .24 -.05-2.27 .02 .92 Oft ehe 4.1 *4.20 -1.06 -.10-3.084.42-.55 .01 .38 tleitstaettes 4.1 ,-.01 *.46 -.06-4.01 -.01 -.99 .00 .21 006smi Ossitivedsotio 4.1 .00 iii.O5 -.04 -.94-.05-1.18-.08-1.77 6sheei Opsittle 1.) a.02 ...le .00 .16 .01 .25 -.02-.75 *shoot Pestahmeets 4.) si.OS 'L40-.02.4.40-.01 -.61 .01 .45 404 otipoteseei feepeteeep is) .00 s..22 .02 1.46 -.01 -.31 .00-.20 'settees thoitistossest is) .01 .44 .04 2.97 .01 .70 .03 2.08 ditssalthes is/ -.04 441 -.OS -2.31 -.02 -.76 .03 .98 *shoat Aelseemsed is) Al .83 - .02.1.36 -.01 -.63 -.03-1.23 8 esetiese) tasystattoso (s) is.10 s.069 .25 1.88 .06 .38 -.23-1.52 6s44e1 (41.4 .01 .52 .02 1.21 .03 1.26 .01 .34 -1.46 2.39 .33 Cm lispeitetwo4 I ) .94 .20-5.10-1.26-8.23 Vitt MOM We 1- 0 -.02 -1.10-.Al -.32-.01 -.26-.05-1.08 ksmoissmeet is) is.02 -1.07 .01 .73 .03 1.56 -.01 -.56 letete is) .02 .11 .21 1.40 .04 .29 .27 1.82 %Mot ilhorelea (a) .0) .12 .01 .83 .11 2.12-.08-1.46 Pismo. owl Art test tit) .01 .13 -.01-.20 .01 .20 -.04-.61 lOodt* ddishaistuttito0 '0 .04 1.25 .00 .02 .04 .68 -.17-2.88 bete elsettl is) .03 .10 .01 .25 .00-.09 -.05-1.13 Oells Pattspos ist .04 AO-.03-.57 -.01 -.15 -.03 -.52 Pisan* tad Attie* it) .03 .55-.10-2.19-.03 -.72 -.06-1.17 Weds is) .02 .76 .06 2.21 -.04-1.49 -.01 -.49

P.m. imetesemse »a r me *4 .so .80 .80 .24

paw 1101 dotted direettes sit thefts is indicated in parenthesesafter each 'tele some. g*etettetite tee emmpeilti.sal messeres (Total Delinquencythrough lismoiressosta me hosed epos 1101 sod 1102 moons old standarddeviations for each POW pepsheessis4 elisate feeieeeMI the "rstatistic"is the ratio of the 44fterssee issesess IOW end 10411 stores to the standard errorof measurement of "Ne 1444 sees** tar all seepOt. Peel schools is theWO" initiative.As a rule of Wash, serichseettai el into moaaere differfoces that aretwice the standard *MR of 4.0441101004 1114,:be ffterded asdependable. For compositional measures, 4 ,444104 of 1.16 4011 1.54 are statisticallysigaificant at the 1)4.05 and 1)(41 weepectiveiy. 611 sesames are takes tress the SAESstudent surveys eidees meter tadicated.

tiseest tales ties 8418 teacher seem

- Table 4c

Number of PATHE and Control Schools that Improved from 1981 to 1982 on Survey Measures of PATHE Goals and Objectives

PATHE Schools Control Schools

MiddleHigh Total Middle High Total Goal or objective measure (n=4) (n=3) (n=7) (n=1) (n=1) (n=2)

__b Total Delinquency (-) __b 3 3 0 0 __b Serious Delinquency (-) __b 3 3 "0 0 __b Drug Use (-) __b 2 2 0 0 Suspensions (-) 2 3 5 1 0 1 Victimization (-) 3 3 6 0 0 0 School Nonattendance (-) 2 , 3 5 0 1 1 School Rewards (+) 3 2 5 0 0 0 School Punishments (-) 2 3 5 0 0 0 Interpersonal Competency (+) 4 1 5 0 0 0 Positive Self-concept (+) 4 3 7 1 1 2 Alienation (-) 4 3 7 1 0 1 School Attachment (+) 4 1 5 0 0 0 Educational Expectations (+) 2 2 1 0 1 Belief (+) 3 3 :- 1 1 2- Low Expectationsa (-) 2 2 4 1 0 1 Involvement (+) 1 2 3 1 0 1 Victimizationa (-) 2 3 5 1 1 2 Safetya (+) 3 3 6 1 1 2 Teacher Moralea (+) 4 3 7 1 0 1 Planning and Actiona (+) 3 2 5 0 0 0 Smooth Administrations (+) 2 3 5 0 0 0 Rule Clarity (+) 3 2 5 0 0 0 Rule Fairness (+) 2 1 3 0 0 0 Planning and Action (+) 2 1 3 0 0 0 Safety (+) 4 2 6 1 0 1

Pct. measures improved .69 .79 .74 .50 .20 .34

Note. The desired direction of change is indicated in parenthesesafter each scale name. All measures are taken from the SAES student surveys unless otherwise indicated.

aMeasure taken from SAES teacher survey.

bDelinquency was not measured in 1981 for middle schools.

-369- 247 Table 5

Rates of Suspension, Expulsion, Attendance and

Withdrawal from School, 1979-1982

Suspensions Expulsions Average Daily Withdrawals AttendanceL

79-80 80-81 81-82 79-80 80-81 81-82 79-80 80-81 81-82 79-80 80-81 81-82

C. .rtenay M.S. .26 .13 .08 .00 .00 .00 .93 .93 .94 .03 .01 .01

Rhett M.S. .15 .09 .10 .00 .00 .00 .93 .95 .94 .03 .01 .01

Rivers M.S. .11 .18 .25 .01 .00 .00 .94 .94 .94 .02 .02 .03

Haut Gap M.S. .04 .10 .20 .00 .00 .00 .93 .93 .93' .01 .01 .01

1 Laing M.S. .08 .13 .15 .00 .01 .00 .93 .94 .94 .02 .01 .02

Brown H.S. .46 .66 .32 .01 .01 .02 .89 .90 .91 .14 .10 .14

Burke H.S. .09 .18 .14 .01 .01 .02 .90 .92 .11 .10 .10

St. Johns H.S. .49 .90 .87 .02 .02 .02 .93 .91 .92 .10 .09 .10

CharlestonH.S.1 .24 .34 .48 .01 .01 .00 .92 .93 .94 .11 .05 .07

1 Control School

2 Average daily attendance of average daily membership.

249

248 Table 6 California Test of Basic Skills PATHE and Control School Scores, 1981 and 1982

scoring in Scale Score bottom national 1981 1982 quartile

School H SD 11 SD 1981 1982

Reading

Courtenay 436.1 80.8 459.5**82.7 42.9 32.9 Rhett 458.4 92.7 481.8**92.2 31.0 19.1 Rivers 446.2 75.8 466.6**71.6 39.0 31.7 Haut Gap 475.8 77.9 488.0* 73.5 34.1 27.2 Laing 514.4 101.6 536.7**88.9 21.7 11.5 Brown 483.2 86.3 488.9 87.7 57.6 57.1 Burke 509.5 90.4 523.1**91.8 50.2 45.4 St. Johns 519.4 94.3 528.6 98.8 41.7 38.7 Charleston 567.0 90.5 575.7 53.2 36.2 30.0 PAM Schools 484.3 91.9 495.3**90.6 43.8 36.3 Control Schools 543%7 99.0 554.9* 92.9 29.8 19.8

Language

Courtenay 452.9 76.5 470.4**81.1 36.0 25.4 Rhett 471.7 92.4 489.1**85.6 26.4 18.4 Rivers 462.1 76.4 474.1* 70.9 29.5 24.9 Haut Gap 480.1 73.6 490.0 64.6 29.3 18.6 Laing 513.6 99.8 527.6* 81.0 18.3 11.7 Brown 501.0 84.8 504.7 87.6 43.6 41.4 Burke 516.0 90.1 527.8**91.2 37.3 37.1 St. Johns 520.1 89.2 522.1 93.3 39.8 39.3 Charleston 573.9 95.6 578.7 92;1 -22.4 21.1 PATHE Schools 494.3 88.6 501.1**86.9 36.0 30.4 Control Schools 547.1 101.9 551.5 90.0 20.6 16.1

}lath

Courtenay 437.5 65.2 451.6**70.2 37.6 28.8 Rhett 448.1 75.3 465.5**71.4 31.9 23.8 Rivers 440.4 56.9 448.5* 59.1 39.6 32.9 Haut Gap 450.2 65.4 463.5**60.9 33.1 26.6 Laing 485.9 88.8 508.7**82.8 23.0 13.4 Brown 473.8 77.9 477.2 80.4 53.6 52.1 Burke 488.7 81.7 504.5**81.9 47.6 44.7 St. Johns 509.1 85.8 509.0 94.9 41.5 37.1* Charleston 531.8 85.3 546.1* 88.3 36.1 33.8 PATHE Schools 472.0 80.0 478.6**80.1 42.0 35.8 , Control Schools 511.2 89.8 526.3**87.3 30.0 22.5

Total Battery

Courtenay 422.9 78.9 445.0**83.0 46.3 34.5 Rhett 442.1 95.5 463.8**89.1 32.8 23.4 Rivers 430.9 72.9 445.9**71.8 44.8 35.8 Haut Gap 453.1 76.7 466.1* 69.0 39.2 29.6 Laing 496.0 107.0 518.5**93.6 25.5 13.0 Brown 477.7 86.4 482.1 90.8 60.4 59.4 Burke 499.1 91.3 515.1**92.5 53.6 50.4. St. Johns 513.2 94.0 519.0 102.9 46.3 44.3 Charleston 559.6 92.1 571.6 96.5 38.9 33.6 PATHE Schools 472.7 92.8 481.9**92.3 47.7 40.3 Control Schools 530.8 104.0 543.3* 98.5 32.9 22.4

Note: Table entries are based on all cases in the school each year for whom a valid test score was provided. N's range from 277 to 634 for 1981, and from 338 to 661 for 1982. Control School *Difference between 1981 and 1982 is significant at the p<.05 level. **Difference between 1981 and 1982 is significant at the pc.01 level. Table 7

Means and Standard Deviations for PATUE Target and Control

Croups on Pleasures of Goal 1: Reduce Delinquency

Number of Disciplinary Infractions Self-reported Delinquency Number of Suspensions Number of Major Minor Total Serious Luna Official Self-reported Expdlsions SD N SD N M SD N M SD N M SD N M SD N M M SD N M SD .N M

Courtenay 5.62 2.54 81 .18 62 .14 .22 62 2.64 1.40 81 82 .15 .16 62 .13 Target .12 .36 82 .35 .48 65 .01 .11 55 2,82 1.48 80 5.392.52 80 55 .10 .18 55 .10 .21 60 ,00 80 .11 .15 Control .09 .28 80 .30 .46 .00

Rhett 3.92 1.51 61 .20 42 .16 .26 41 2.28 1.27 61 63 .14 .21 42 .12 .11 .51 63 .21 .41 43 .01 .33 Target 1.14 49 4.26 1.52 49 .23 31 .12 .24 31 2.28 51 .13 .22 30 .11 Control .12 .43 51 .37 ,49 30 .00 .00

81 ) Rivers .18 53 2.62 1:14 81 4.27 1,47 .13 .13 51 .09* .14 51 .15 .37 .86 84 .38 .49 47 .00 .00 84 Target 48 2.61 1.23 12 4.53 1.66 72 .19 .19 46 .16 .20 46 .21 .27 .39 .49 41 .01 .12 74 I Control .50 1.20 74 r I 2.42 1,15 59 4.51 1.49 59 1 uliaut Cap 40 23 4.13 1.49 23 .12 37 .05 .10 31 "2 .23 2.96 1.40 .38 30 .00 .00 62 .09 1 "I .32 .78 62 .17 Target 13 13 .11 .16 13 .18 .22 .27 .46 23 .15 .11 Control .17 .65 23 15 .00 .00

1\11 ..7 70 2.44 1.22 10 3.99 1.59 C. A. Brown 53 .26 .28 54 ..97 70 4.41 1.43 70 .13 .15 53 .06 .12 . 2.26 .43 .81 75 .33 .48 54 .00 .00 75 Target .28 55 .17 55 .09 .15 55 .22 .43 56 15 .14 Control .35 .60 75 .23. ,03 .16

2.25 1.12 75 4.15 1.53 75

Burke .31 .33 60 72 4,24 1.73 72 .22 59 .12 .21 59 2.44 1.28 .25 .44 55 .01 81 .19 Target .17 .47 81 .11 .30 , 53 .17 54 .08 .16 54 .25 .42 .14 Control .08 .27 78 .22 46 .01 .11 78

St. Johns 51 2.26 1.39 69 4.23 2.00 69 .13 52 .06 .10 52 .25 .26 .52 .50 52 .12 .971,67 16 01 .11 76 70 Target .27 47 2.13 1.05 70 4.27 1.57 .11 .13 47 .06 .12 46 .20 Control 1.26 2,19 16 .51 .50 51 .01 .11 76

1.87 496 All Schools .27 361 2.43 1,25 496 4.41 .14 .17 356 .09 .16 356 .20 .36 .92 523 .33 .47 346 .01 .09 523 Target .27 302 2.47 1.24 436 4.53 1.84 436 .14 .17 300 .10 .17 300 .19 .47 299 .01 Jo 457 Control .40 1.15 457 .33

is significant at the p

Measures not available.

and withdrawals are based is all casesrandomly of official suspension::, expulsions Note - -The base N on which computations The base y1,0 to ro IP,Ihlraled in the schools as of October, 1981. assigned to treatment and controlcondition; I randomly assigned to treatment andcontrol conditions minus other outcomes are based is all cases N on which computations of all from school as of 2/82 and for whom no2nd semester data were provided, 72 cases who had withdrawn

fir,. 9 4,10.1. 25 Table B Means and Standard Deviations for POE

Target and Control

Croups on Measures of Coal II: Improve Attendance

% Days Absent - String '82 % Days Tardy No, of Attendance Number of Self-reported Non-attendance

Total Unexcused to Class -arinf, 182 related disc. infrar.. 011hdrawals Cutting School Cutting Class Total Index

N II SD N SD N M SD N H SD N M SD N 11 SD N M S0 N ti 5b Courtenay

.53 .67 74 Target .01 .08 81 .06 .06 81 .04 .08 81 1.54 1.48 81 .11 .31 82 ..47 .96 75 .49 1.04 15

Control .01 .09 77 .07 .08 77 .05 .10 77 1.38 1.29 80 .11 .32 '80 .531.06 72 .36 .83 72 .46 .69 72

Alit

Target .06 .07 56 .06 .06 56 .01 .04 56 .67 .03 61 .22 .42 61 .37 .85 51 .85 1.23 53 .65 .80 51

Control .08 .10 47 .06 .07 47 .03 .09 47 .82 .90 49 .16 .37 51 .38 .95 40 .68 1.47 41 .45 .68 40

;beers

Target .09 .14 80 .07 .12 80 .04 .06 80 .91 1.04 81 .17 .37 84 .20 .65 64 .401.01 65 .27 .60 61

Control .13 .18 70 .10 .16 10 .04 .08 10 .80 .04 72 .08 .27 74 .20 .56 55 .25 .82 55 .28 .60 54

Haut Cap .49 55 Target .08 .08 59 .03 .04 59 .02 .03 59 1.07 .96 59 .08 .27 62 .10 .41 51 .24 .55 53 .26

Control .09 .11 22 .04' .05 22 .01 .02 22 1.13 1.22 23 .00 .00 23 ..42 1.12 19 .63 1.30 19 .42 .77 1

.4 1., A. Brown a a 1.10 1.32 61 .78 .78 60 1.0) Target .10 .10 70 .09 ,10 70 1.28 1.26 70 .19 .39 75 .33 .60 61 a a a 1.28 .84 1 Control .11 .14 69 .10 .13 69 1.24 .98 70 .25 .44 75 .52 .88 62 1.19 62 .90 62

Burke a a Target .09 .10 72 .08 .09 72 .48 1.19 75 .20 .40 81 .39 .58 61 1.27 1.05 63 1.03 .80 8r a a a Control .11 .11 69 .10 .15 69 .12 1.17 72 .29 .45 78 .45 .83 62 1.04 .99 56 .93 .79 55

St. Johns

Target .09 .10 62 .07 .09 62 .09 .10 62 1.38 1.35 69 .18 .39 76 .30 .78 56 .59 .95 56 .54 .71 56 1.39 54 .56 .71 54 Control .09 .08 61 .07 .07 61 .10 .12 61 1.60I.42 70 .17 .38 76 .29 .78 55 .87

All Schools 426 .58 .74 415 Target .08 .10 480 .07 .09480 ,05 .07 330 1.20 1.22496 .16 .97 573 .32 .71 419 .11 1.11 .59 .77 356 .10 .13 .10 217 416 .88 359 .12 1.18 359 Control 415 .08 .12 415 .11 1.18 1.18 .17 . 457 .40

a Measures not available.

b Withdrawals are cases that left school before the close of the 81-82 school year for reasons other than graduation from the twelfth grade.

Note-The base N on which computations of official suspensions, expulsions and withdrawals are based Is all cases randomly assigned to

treatment and control conditions minus 24 cases who were not located in the schools as of October, 1981.The base N on which computations of all other outcomes are based is all cases randomly assigned to treatment and control conditions minus 72 cases who had withdrawn from school as of February 1982 and for whom no second semester data were provided.

253 1 of 3

Table 9

Means and Standard Deviations for PATI1E Target and Control Groups on Measures of Coal IV: Decrease Academic Failure

Number of Academi. Percent Receiving Failing Grades During Spring '82 Semester Self-reported Related Disc. Infra, English Math Social Studies Science Average Grades N M SD N M SD N M SD N M SD N M SD N M SD N M SD Courtenay 2.37 .25 .54 81 Target .40 .49 81 .36 .48 81 .31 .46 81 .38 .49 81 .42 .50 81 .80 76 2.62 72 .28 .59 BO Control .40 .49 78 .36.48 78 .26 .44 78 .27 .45 78 .44 .50 78 .97

Rhett 52 .23 .50 61 Target .24 .43 55 .07 .26 55 .36 .0 55 .17 .38 53 .22 .42 55 2,75 * *,74 .35 49 .35 .48 46 2.36 .62 41 .14 Control .38 .49 45 .15 .36 46 .49 .50 45 .20 .40 45

Rivers 65 .16 .37 81 .37 .49 73 .42 .50 .26 .44 73 .44 .50 73 2.35 .72 I Target 73 .46 .50 72 .18 .39 72 L.; Control .45 .50 65 .42 .50 65 .44 .50 64 .40 .49 65 .54 .50 65 2.27 .85 55 ..,1 4s Illaut Gap 2.56 .75 52 .25 .54 59 Target .38 .49 55 .11 .31 55 .38 .49 55 .18 .39 55 .16 .37 52 19 .13 .34 23 Control .45 .51 22 .14 .35 22 .41 .50 22 .23 .43 22 .27 .46 22 2.52 .90

C. A. Brown 2.13 61 .06 .23 70 Target .52 .50 62 .38 .49 48 .12 .33 33 .32*.47 44 .47 .50 64' .50 .49 2,21 52 62 .06 .23 70 Control .67 .48 63 .38 .49 55 .13 .34 30 .50 45 .58 .50 65

Burke 2.36 ,66 63 .08 .27 75 Target .39 .49 64 .31 .47 55 .30 .46 53 .53 .51 36 .40 .49 65 2.43 .60 56 .11 .36 72 Control .40 .49 60 .30 .46 47 .17 .38 47 .30 .46 40 .31 .47 64

St. Johns 2.32 .64 56 .14 .39 69 Target .20 .41 59 .35 .48 49 .41 .50 56 .43 .50 30 .40 .49 60 .11 .32 70 30 2.34 .67 55 Control .31 .47 58 .48 .50 52 .27 .45 55 .37 .49 .41 .50 i61

All Schools 425 .16 .42 496 .48 453 ' 2.40 .71 Target .36*.48 449 .29 .46 416 .35 .48 405 .31 .46 312 .37 .40 436 2.39 .75 360 .15 Control .44 .50 391 .34 .48 365 .31 .46 341 .33 .47 325 ,43* .50 401 *Difference between target and control groups is significant at the pe..05 level.

AmDifference between target and control groups is significant at the p,..01 level.

Note--The base N on which computations of official suspensions, explusions and withdrawals are based is all cases randomly assigned to treatment and control conditions minus 24 cases who were not located in the schools as of Octo-

ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treatment and control conditions minus 72 cases who had withdrawn from school as ofFebruary, 1982 and for whom no second 256 semester data were provided. 2 ot 3

Table 9 (cont' d)

Standard ScoresElpliag '82 CTBS Test

Reading Math Language Total

M SD N M SD N M SD N M SD N Courtenay

Target 408.37 73.68 79 405.32 60.61 78 416.94 78.87 79 387.45 75.37 78

Control 406.90 60.85 75 406.25 53.62 7 415.36 64.16 76 386.73 62.10 75

Rhett

Target 441.05 70.36 58 436.83 55.26 58 444.71 69.07 58 421.83 67.96 '58

Control 422.11 77.72 47 429.81 50.90 47 430,36 57,57 47 407.72 62.22 47

Rivers

Target 432.74 66.06 72 421.74 47.73 72 445.85 58.25 72 411.51 56.41 72

Control 437.12 69.52 64 423.89 48.66 64 440.67 57.64 64 412.80 61.48 64

Haut Cap

1 L.) Target 437.36 60.21 55 420.35 59.18 47 437.67 57.71 55 409.41 59.51 54 .1 ui Control 410.19 49.72 21 402.00 49.26 21 432.14 52.64 21 392.57 46.47 21

C. A. Brown

Target 474.26 76.09 62 465.58 76.82 62 493.63 73.94 62 467.56 78.32 62

Control 475.34 80.50 64 457.66 65.71 64 479.47 72.48 64 459.50 74.97 64

Burke

Target 473.66 82.24 65 459.11 75.56 65 466.94 82.16 65 454.54 79.75 65

Control 482.21 85.70 58 470.96 62.06 '57 478.86 83.03 58 466.16 74.82 57

St. Johns

Target 492.10 90.02 50 465.94 94.52 154 478.76*82.14 50 470.34 102.60 47

Control 465.85 81.74 52 453.98 78.22 148 458.62 76.06 52 441.57 74.78 47

All Schools

Target 448.64 78.72 441 437.21 70.67 436 453.06 76.06 441 429.04 79.87 436

Control 445.04 79.44 381 436.53 63.70 377 448.64 71.97 382 425.50 73.32 375 *Difference between target and control groups is significant at the p<.05 level.

**Difference between target and control groups is significant at the p<.01 level.

Note--The base N on which computations of official suspensions,, explusions and withdrawals are based is all cases

randomly assigned to treatment and control conditions minus 24 cases who were not located in the schools as of Octo-

ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treat- 5 ment and control conditions minus 72 cases who had withdrawn from school as of February, 1982 and for whom no second258 semester data were provided. TO14. $ (coned) 3 of 3

0101311111Vahr4M the Settee larti1e on the Spring82 CTBS Test Self-reported lOta Reading Ability 0 N M SD N M SD N

.69 .46 78 1.40 .98 63 1/2 .411 PI .61 .47 71 .66 .48 79 .79 .41 75 1.45 .99 62 ,7t 15 44 .41 76 .70 .46 76

58 .48 .50 58 1.62 .85 42 ot4 ,41 .40 59 .40.49 54 .43 .50 47 .68 .47 47 1.60 .97 38 1404 .04 .40 41 .50 47 .57 .50

.50 72 .64 .48 72 1.35 .80 54 *1-64 .4$ .411 72 .00 .49 72 .42 .52 .50 64 .64 .48 64 1.30 .95 47 4S.01 .$4.$0 64 AO40 64

54 1.38 .79 42 04 At' .40 $$ .54 .50 34 .49 .50 55 .70 .46 18 .57 .51 21 .86 .36 21 1.44 .70 444 .14 . 21 .76 .44 21

40464 .77 .42 62 1.60 .75 57 04 .02 .21 61 .72 .45 62 .69 .46 62 .84 .37 64 1.38 .84 52 Owl .04 .40 44 .6) 31 64 .76 .43 64

65 .88 .33 65 1.50 .84 54 .44 .2$ 0$ .$2 .29 6$ .80 .40 .86 .35 57 1.39 .70 49 ,40 54 .14 .44 57 .69 .41 58

Omo * 50 .77 .43 47 1.33 .71 52 ,f) .6$ $0 JO .46 47 .66.411 rio4 49 41 .46 .34 52 .91 .28 47 1.41 .84 0104 .11 ,44 51 .77 .42

4044* 441 .70**.46 436 1.45 .83 364 .0 .44 441 .66 .414)6 .5944.49 .79 .41 375 1.42 .88 315 01, .47 3112.-- ,",,f1111rPOP efir2 (antral grow Is Writ Icaot at the pc.05level. 114.00040:0 4 001000, 41:144,f4 44f04 *44 roam! imp*le significant at the oe.01 level.

withdrawals are based is all cases ert* 4**0 $1 04 4404 00,4440404of official olossioneloos, exclusions and (Petrol 44440444 SOW 24 cases who wore notlocated in the schools as of 0cto- If 004004 40 100410041 and of all other outcomes ate based Is all casesrandomly assigned to treat- 016 tto boo limillAkh pompetstiome February, 1982 and for whom no second hi WNW witittomo Woo if 40441414 hadvithdrown from school as of 260 win Ionfir. 1 of 3

Table10 'Means and Standard Deviations for PATE

Target and Control Croups on other Survey Measures

Interpersonal Attachment to Educational School Involvement In Practical Self-concept Competency Parents School Expectations Effort Extracurricular Activ, Knowledge

M SD N M SD N M SD N M SD N M SD N M SD N M SD N M SD N Courteuy

Target .73 .25 62 .70 .21 51 .66 .22 76 .11 .20 69 2.44 1.11 73 .54 .26 63 .38 .21 60 I.24*.44 60 Control .78 .19 60 .67 .19 51 .66 .25 72 .16 .21 61 2.57 1.88 10 .55 .27 67 .36 .23 65 1.40.37 57

Rhett

Target .78 .19 31 .75 .15 33 .65 .28 52 .16 .23 45 2.92 1.80 52 .63 .30 49 .32 .24 49 1.35.36 37 Control .71 .22 27 .10 .14 27 .68 .19 39 .14 .24 34 3.41 1.82 39 .67 .24 34 .24 .25 32 1.40.41 23

River;

Target. .75 .22 46 .73 .19 38 .62 .23 61 .10 .22 52 2.76 1.85 62 .61 .29 61 .31 .22 56 1.28.42 46 Control .73 .28 39 .10 .19 32 .65 .25 54 .68 .24 43 2.76 1.90 55 .56 .29 48 .24 .20 43 1.31.50 31

Haut Cap

1 Target .79 .22 30 .11 .19 27 .64 .21 50 .16 .27 34 2.56 1.91 52 .58 .29 46 .25 .22 42 1.17.47 33 1.4 Control .10 .22 12 .67 .4 .22 13 .68 .26 11 .63 .28 16 2.59 2.03 17 .56 .30 14 .28 .21 15 1.25.35 14 .4

I C. A. Brown

Target .83 .18 51 .16 .18 45 .66 .24 60 1.63 6'4 .65 ,77 .20 51 2.08 .26 60 .22 .11 56 1.47.40 54

Control .85 .17 52 .80 .14 46 .59 .22 59 .17 .20 58 2.20 1,38 56 .60'.29 60 .26 .18 58 1.38.48 51

Burke

Target .19 .21 44 .16 ,16 41 .60 .22 64 .75**,18 59 2,05 1.51) 64 .58 ,26 63 .21 .11 58 1.40,41 43

Control .71 .24 45 .79 .14 39 .62 ,27 56 .83 .13 48 2.70 1.65 54 .64 .28 55 .11 .15 55 1,46.36 45

i St. Johns

Target 47 1.41 AO .21 .15 .18 41 .59 .25 55 .72 .22 53 2.41 1.55 54 .66 .27 52 .21 .18 44 .40 47

Control .79 .24 50 .73 .14 )45 .66 .23 55 .65 .28 52 2.07 1.59 54 .69 .26 51 .25 .22 49 1.40.39 41

All Schools

Target .78 .21 317 .74 .18 276 .63 ,2. 418 .74 .21 3691 2,1,5 1.13 417 .60 .27 394 .21 .21 365 1.34.42 320

Control .18 ,22 285 .73 .11 253 .64 .26 152 .73 .23 112 2.58 1.76145 .61 .28 329 .26 .21 311 114.42 274 *Difference between target and control groups Is significant at tie -.15 level,

Note--The base N on which computations of official suspensions, explusions and withdrawals are bayed is all cases

randomly assigned to treatment and control conditions minus 24 cases who were nnt located in the schools as of Octo-

ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treatment and control conditions minus 12 cases who had.withdrawn from school as of February, 1982 and for whom no second semester data were provided.

262

26.; 2 of 3

Table 10(cont'd)

Parental Emphasis School School Perceptions of School Climate Disrespect Control of on Education Punishments Rewards Rule Fairness Rule Clarity St-Tch Interaction for Students Environment

N SD N M SD N M SD N M SD N M SD N M SD N M SD N M SD N Courtenay Target .63 .29 68 .40 .25 69 .37 .31 69 3.11 1,41 66 3,67 1.06 66 1.56 .95 68 ' 2.831.3966 .56 .23 64

Control .63 .29 62 .45 .30 62 .43 .32 61 3.44 1.22 61 3.67 1.06 57 1.691.00 64 2.411.30 59 .62 .20 60

Rhett

Target .59 .26 39 .33* .29 45 .36 .33 45 3.40 1,47 42 3.79 1.14 43 1.611.00 46 2.771.34 43 .56 .24 37

Control .66 .49 30 .26 ,27 32 .33 .26 31 3.48 1.50 29 4.00 1.19 28 1.271.23 33 3.171.29 30 .62 .20 28

Rivers 1 .ti

(.k) Target .64 .30 50 .36 .28 53 .36 .34 53 3.36*1.20 52 4.04*1.09 51 1.511.10 57 2.791.2843 .54 .22 46 .4 Control .60 .29 42 .37 .22 37 .35 .31 38 2.78 1.53 36 3.57 1,14 37 1.241.04 41 2.541.3835 .57 .18 38 CO

1 Haut Cap

Target .58 .29 33 .24 .25 35 .34 .35 36 3,03 1.54 35 3.69 1.18 321 1.421.20 38 2.561.64 .14 .58 .26 31

Control .57 .30 15 .32 .27 15 .32 .29 15 3.00 1.57 14 3.60 1.18 15 1.501.10 16 3.001.51 15 .64 .21 14

C. A. Brown

Target .50 .33 54 .21 .25 56 .30 .30 56 3.29 1.28 51 3.61 1.06 51 1.531.24 57 2.251.15 52 .58 .24 53 55 .24 .28 .50 1 Control .50 .32 55 .20 .26 56 3,36 1.24 50 3.47 1.38 53 1.531,04 56 2.481.42 52 .23 53

Burke

Target .47 .32 49 .24 .26 58 .24 .28 59 3,10 1.38 57 3.85 1.18 53 1.501.18 59 2.651.36 55 .57 .22 43

Control .49 .29 50 .22 .21 47 .2 5 .30 61 3.51 1.10 43 4.09 1.09 46 1.671.1546 2.261.42 43 .51 .21 45

St. Johns

Target .40 .26 49 .20 .28 52 .24 .25 52 2.85 1.38 48 3.86 1.11 51 1.631.07 52 2.881.42 50 .56 .23 47

Control .42 .28 51' .19 .27 53 .27 .32 54 2,61 1.50 51 3.54 1.26 52 1.421.15, 53 2.881.50 52 .54 .22 50

All Schools

Target .55 ,DI 142 ,29 .28 368 .31 .31 170 3.18 1.38351 1.-1 1.1114. 1.''i 1.10.371 2,6P1,37343 .56 .23 321'

Control .54 .111 307 .31, .29 299 J,19 1.33284 3.69 1.20 2r. 1.111.10 309 2,611.41256 .56 .21 288

Note-The base N on which computations of official suspensions, explusions and withdrawals are based is all cases

randomly assigned to treatment and control conditions minus 24 cases who were not located in the schools as of Octo- ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treatment and control conditions minus 72 cases who had withdrawn from school as of February, 1982 and for whom no second semester data were provided.

264 3 of 3

Table 10 (Cont'd)

Belief in Negative Working Havea Rebellious

Rules Peer Influence Alienation Victimization for Pay job Autonomy M SD N M SD N M SD N M' SD N M SD N M SD N M SD Courtenay

Target .62 .24 61 .27 .23 68 .37 .22 62 .25 .22 69 .60 .49 70 .36 .66 70 .50**.30 5 Control .57 .25 60 .24 .18 66 .43 .27 60 .24 .27 60 .58 .50 71 .48 .74 70 .68 .28 6

Rhea

Target .63 .30 49 ,62 39 .22 .21 .33 .26 38 ,19 .25 44 .49 52 .46**.61 . 50 .57 .35 1 Control .59 .19 29 .18 .16 36 .44 .20 29 .18 .20 31 .50 .51 40 .18 .38 40 .67 .31. 2

Rivers

Target .55 .26 47 .28 .22 58 .40 .20 44 ,27 .25 51 .61 .49 64 .29 .52 65 .57 .27 .4 Control .60 .22 38 .24 .19 46 .38 ,22 37 .22 .22 37 .47 .50 51 .19 .52 52 .60 .29 3

Haut Cap

Target .59 .30 30 .21 .22 46 .43 .26 29 .16 .21 37 .51 .50 47 .47 .65 49 .61 .36 3 .59 .24 Control 13 .24 .24 14 .39 .35 13 .27 .28 15 .44 .51 18 .39 .78 18 .61 .34 1

C. A. Brown

Target .62 .27 54 .24 , .22 59 .26 .20 51 .10 .17 54 .37 ,49 59 .28 .52 60 .59 .35 5 .22 Control .67 .24 53 .19 59 .27 .23 52 .09 .16 56 .37 .49 60 .33 .54 60 .56 .30 4 tAl Burke

Target .64 .20 43 .21 .20 60 .37 .22 45 . .10 .18 58 .44* .50 63 .26 .51 64 .64 .28 4

Control .24 45 .21 .19 .46 .68 55 .30 .23 43 .10'.16 46 .62 .49 58 .60 57 .56 .34 4

St. Johns

Target .70 .20 49 .17 .14 54 .42 .24 48 .11 .14 52 .55 .50 56 .39 .68 56 .56 .34 4

Control .63 .21 50 .22 .20 53 .41 .24 50 .13 .22 53 .55 .50 54 .28 .60 54 .60 .33 4

All Schools

Target .62 .26 323 .23 .21 394 7' 317 .17 .22 365 .53 .50411 .35 .60 414 .57 .32 30

Control .62 .23 288 .22 .19 329 .16 .22 298 .51 .50 352 .34 .61 351 .61 .31 27

*Difference between target and control groups is significantat the p<.05 level.

**Difference between target and control groups Is significant at thep.01 level;

Note--The base N on which computations of official suspen ,1, explusio:Is and withdrawals are based is all cases

randomly assigned to treatment and control conditions minus . . cases who vere not located in the schools as of Octo- ber, 1981. The base N on which computations of all other outcomes are ba ed is all case!; randomly assignedto treat-

ment and control conditions minus 72 cases who had withdrawn from school ,1 of February, 1982 and for whomno second semester data were provided.

,oJ Table 11

Percent Students Receiving PATHE Services and Number of Contacts - Burke High School and other PATHE Schools Number of Contacts %recd Service Mean SD Media

Burke High School All Target Students (N=86) 84.9 7.2 12.4 3.0 Target Students who took survey(N=59) 94.9 8.6 14.1 3.4

All Control Students (N=80) 55.0 2.4 6.5 .7 Control students who took survey(N=48) 62.5 2.a 7.9 .9

Other PATHE Schools All Target Students (N=454) 90.7 6.6 6.4 4.9 Target Students who took survey(N=310) 94.8 7.6 6.8 5.6

All Control Students (N=384) 60.4 2.2 .2 1.0 Control Students who took survey(N=264) 62.9 2.1 3.1 1.0 Table 12 Fidelity Scale

The rater compares implemented activities and tasks as documented on the Implmementation Report to the standards for implementation describe on this and subsequent pages. After checking off which of the standards have been met, a rating is assigned to each intervention using the following scale: 4=exceeds standards

3=meets standards completely or with approved adaptation

2=meets SO-99% of standards

1=meets 1-49% of standards

0=does not meet standards/not implemented

Intervention Standards

Tutoring outside tutors involved - regular' schedule of tutoring contacts by C.S. -target students included

Counseling -RAP sessions plan on file -evidence of RAP sessions -evidence of at least 10 parent .contacts -contact with at least 90% of students suspended -system established to monitor attendance problems -target students included in counseling

Study Skills -plan on file -evidence of implementation

Job Seeking Skills -distribution of materials -presentations to groups on job seeking skills or career awareness

SerVices to Target -individualized instructional plan/needs Students assessment or problem statement for every target student -letter sent to parents -positive parent contact for every target student -at least one contact with each targetstudent -link to summer enrichment program

Resource Room -needs assessment memo used after each grading period -resource room open house conducted -evidence of teacher use

Faculty Inservices -participation in team building conference -teacher participation in Student Team Learning Learning training -inservice plan on file which addresses identified needs -evidence of implementation of plan

.School Pride -plan on file for 0,,1e year Campaign -evidence of implementation

Evaluating the Implementation of a Delinquoncv Source: Birdseye, A. T. Paper presented at the Prevention Proe,ram:: The PATHE Experience. Annual Meeting of the American EducationalResearch Association, New York, March 1982. Tlhlo !'Coont.) Intervention Standards

Curriculum Review -CTBS analysis on file Campaign -plan to improve identified weakness on file -evidence of meetings with teachers re plans and weaknesses -failure data collected and used to identify students and teachers who need help -help provided identified teachers and students -mini-tests in use -instructional materials and teaching strategies distributed -assistance in use of curriculu:. guides -at least one .teacher has adopted S.T.L. in classroom

-S.T.L. monitoring forms completed . -test-taking motivational activities implemented

Field Trips -plan on file -one trip for each of three areas (career, academic, cultural) -scheduled throughout the year -planned trips implemented

Reading Experie:7ce -plan for school-wide program on file Program -evidence of implementation -motivational activities planned and implemented

Exploratory Program -schedule on file -display of exploratory products in school -schedule implemented

Discipline Policy - school and classroom rules developed by teachers Review and Revision and students and posted -referral procedures and discipline referral form approved by principal -referral procedure implemented so that includes S.C.S. -assistance given to teachers with classroom management problems (at least 10)

Peer Counseling -counselors trailed -plan on file - counseling' services provided on regular schedule

Extracurriculuar High school: Activities -club plans or charters on file -club mid-year activity reports on file -end-of-year reports received Middle school: -clubs begun last year continue

Student Leadership -meetings held regularly Team -statisfactory team plan on file -evidence of plan implementation -central leadership conference attended -participated in planning spring conference -attended spring conference

Student Concerns -regular meetings held Support Team -contracts on file -satisfactory team plan on file -evidence of plan implementation -final evaluation meeting conducted Middle school: =team member agrees to serve as liason -members agree to serve without pay next year

Curriculum Support -regular meetings held Team -contracts on file -satisfactory team plan on file (same as above) -evidence of plan implementation Table 13 Intensity Rating Scale

The rater compares intensity of program service's as documented on the Imple- mentation Report to the standards given below and rates each intervention using the :allowing scale:

4=exceeds standard by more than 5t

3=meets standard + 50

2=meets 50-95t of standard

1=meets 5-490 of standard

0=meets 4% or less of standard

Intervention Intensity Standard

Tutoring 400 incidents of tutoring

Counseling 600 incidentsof counseling

.Study Skills 100 incidents of participation

Job Seeking Skills 1 all school activity or 50 incidents participation

Services to target 8 contacts with every target student students

Resource Room and # teachers checkouts=2 X services to teachers # teachers

Faculty Inservices 3 school level, 5 teachers attend S.T.L., and 5 attend-team building --

School Pride 6 all school activities Campaign

Curriculum Review incidents C.S. faculty contact=4 X # teachers

Field Trip Program 3 field trip conducted

Reading Experience entire school, monthly, for more than Program 15 minutes

Exploratory ?rogram entire school, weekly

Discipline Policy # discipline referral forms.# suspensions Review and Revision +h# suspensions

Peer Counseling 100 incidents contact related to peer counseling

Extracurricular H.S. - 10 club plans Activities M.S. - 100 incidents contact

Student Leadership' incidents attendance = # team members X 6 Tears

Studer,:: Concerns 40 incidents attendance Support Team

Cuv:iculum Support 40 incidents attendance Team Career Explmration 120 incidents attendance programs 1T.T_C.)

Business Education 6 meetings or documentedcontacts Partnership (assembly, planning session)

Parent Leadership 30 incidents attendance Team Source: Birdseye, A. T. Evaluating the Implementation of a Delinquency Prevention Program:The PATHE Experience. Paper presented at the Annual Meeting of the American Educational. Research Association,

.-i New York, March1982...... Je44.., , _A....,] -383- .. el '- P---i 0 .ze....,. ,o, , ,1 le,nt.) Intervention Standards

Career Exploration - students recruited to attend Trident Program Technical College fall program -follow-up meeting with students held -students recruited for spring program - students recruited for FACET

Business Education -team formed with school and business members Partnership -PATHE specialists included on team - plan developed to improve school management -implementation of plan evident - plans for summer meetings on file

Parent Leadership - regular meetings held Team -satisfactory team plans on file -evidence of implementation of plan -leadership training attended -participation in .spring training Draft Table 14

Average of Intensity and Fidelity Ratings for Each PATHE Intervention

All All AIL Courtenay Rhett Rivers Haut Gap Middle Brown Burke St. Johns High ' Schoo

Team Average 1.7 2.7 1.9 2.2 2.1" 2.4 3.3 2.9 2.9 2 Student Leadership 3.0 3.5 3.5 3.0 3.2 4.0 4.0 3.5 3.8 3. Student Concerns Support 1.5 3.5 2.0 2.5 2.4 2.0 2.0 3.5 2.5 2. Curriculum Support 1.5 3.5 2.0 2.5 2.4 2.0 3.0 3.0 2.7 2. Business-Education Partnership 0.0 0.5 0.0 0.0 0.1 1.0 4.0 2.0 2.3 1. Parent Leadership 2.5 2.5 2.0 3.0 2.5 3.0 3.5 2.5 3.0

Affective Average 2.8 2.3 2.6 1.9 2.4 2.2 3.2 2.2 2.6 ., Counseling 3.5 2.5 2.5 1.5 2.5 1.5 3.0 2.5 2.3 School Pride Campaign 3.5 0.0 3.5 2.0 2.2 3.5 S.5 2.5 3.2 Field Trips 2.5 2.5 2.0 3.0 2.5 3.5 3.0 3.0 3.2 Discipline Review & Revision 3.0 3.0 3.0 2.0 2.8 2.0 3.0 3.0 2.7 .2. Peer Counseling 2.0 2.0 0.5 1.5 1.5 0.0 3.5 2.5 2.0 1. Extra-curricular Activities 2.0 4.0 4.0 1.5 2.9 3.0 3.5 , 0.0 2.2 2.

..1 Academic Average 2.8 3.0 1.5 2.8 2.5 2.7 3.6 2.7 3.0 2.. Tutoring 1.5 3.0 2.5 0.5 1.9 2.5 3.5 2.0 2.7 2. Study Skills 3.5 3.5 1.5 2.5 2.R 2.5 4.0 2.0 2.8. 2. Resource Room 1.5 2.5 2.5 3.0 2.4 2.0 3.5 2.5 2.7 . 2. Curriculum Review & Revision 3.5 4.0 2.0 4.0 3.4 4.0 4.0' 3.5 3.8 3., Reading Experience Program 3.5 2.5 0.5 3.5 2.5 2.5a 3.5 3.0 2. 2a __a a Exploratory 3.0 2.5 0.0 3.0 2.1 -- -- 2.

Career Average 2.0 0.0 0.0 3.0 1.2 2.8 3.0 3.0 2.9 Job-Seeking Skills 2.0 0.0 0.0 3.0a 1.2 3.0 3.5 3.0 3.2 2.i a a a a Career Exploration Programs 2.5 2.5 3.0 2.7

Other Average 1.8 1.0 1.0 2.0 1.4 2.5 2.5 1.8 2.2 Services' to Target Students 1.5 1.0 1.0 1.5 1.2 1.5 1.5 1.5 1.5 1., Faculty Inserviees 2.0 1.0 1.0 2.5 1.6 3.5 3.5 '2.0 3.0

Total Implementation Average 2.4 2.4 1.8 2.3 2.2 2.5 3.2 2.6 2.8

aThis intervention was not implemented at this level. fah*. 1$ thlki 0410044101 Osamu, 1911412 toletterep 044411, Wool 14.464)

L.vstiv suwitizmvmmuvle-somoszsgswoalp-watwaswasigus

011410* 00001 taut Stwitott CoMful fitutl.ntb MIIZZZZ-.1:2A:W:Z-Xsa4aZWAkSoSt.*WWIDWIt, ,r&salm=s=sc aaaaaaa am* Mg. Avg. Avg. 00s NO. os I OW 0010 twitting contacts Poppet* 9110* et otroloate pot Se. at per No. of per 04 **0440 Memiatt40ottottedetedtet oottott Hotfoot contacts st, wous-austuu.,42,4,=1-1,-muzzosupwww*mamemwwwwelm.....w......

IOW 414441i4, Ore .47 /094 475 5.72** 243 t

timeroolia4 10 017 A/ 61 Al 53 . ,, 14**4044or 440 AO 1.54 1// 2.13 146 1.62 144404 toodet0 10$ *14 ,k, 39 .47** 7 .09 lea 21444 *4444414, 0 001 A/ 3 .04 1 .01 hot romeeetle 12 .01 .03 0 OA 1 .01 644$ P. 56 AO .00 30 .36** 2 .02 imeteeerre POI 0/5 ,41 ISO IM0* 12 .15 146644401***iielae 44 A/ *13 10 .I: 12 .15 414#.14* lOokkil pttetto 0 *SO *OS 0 .00 0 .00 'wrote& 4 .16 .16 14 .19 9 .11 04404 .1600.044 161 ,10 40 47 .51** 16 .20 Opeehatiol tesivire 01 .10 .19 SI .37 11 .14 Peet me. 0 .00 *OS 0 .00 0 .00 PW104 00440 t .01 .01 1 .01 1 .01 4144111440 ta4 et4es It .04 .06 2 .02 3 .04 *ear milt, 040116104 0 .00 .00 0 .00 0 .00 0004 0 .00 .00 0 .00 0 .00 1464004 Sechrtral 0 *SO .00 .0 .00 .0 .00 644440 1444ca1 kelt It *SS *04 13 .16** 1 .01 000000 14441 3.19 517 6.35** 26$ 3.35 IMMO 1 20110410=e1:02231tiiiisagaiodamaistse4usoAmmossaima.spowalossamaroa. 111411141.411.1114

*141Swerree Seemerft tempt sr il 4444441 stedeste to sissifitaat at the p4.0$ level. 440411444444 66,46606 Wen art 4464,44 strie444 tr eigattiaaat at the 14.01 level. 273 Table 16

PATUE Specialist Contacts, 1981-82 nett Middle School (N=510)

Entire School Target Students Control Students

Avg. Avg. Avg. no. no. no, % contacts contacts contacts Purpose No. of students per No. of per No. of per of contact contacts contacted student contacts student contacts student

Total Affective 1986 .65 4.04 498 7.90 ** 130 2.50 Counseling 480 .32 .98 220 3.49** 37 .71 Discipline 218 .29 .44 53 .84 28 .54 Student Leadership 234 .19 .48 25 .4r 8 .15 Team Peer counselee 0 .00 .00 1 .02. 0 Peer counselor 58 .03 .12 9 .14 0 -- Field trip 0 .00 .00 0 .00 0 .00 Conference 174 .20 .35 98 1.56** 17 .33 Clubs/extracurricular 179 .06 .36 2 .03 0 activity Special program 565 .18 .15 72 1.14 32 .62 Rap session 78 .14 .16 18 .28 8 .15

Total Academic 641 .22 1.30 , 359 5.69** 17 .33 Specialist tutoring 639 .22 1.30 359 5.70** 17 .33

Peer tutee 0 .00 .00 , 0 .00 0 .00 Peer tutor 0 .00 .00 0 .00 0 .00 ''Outside" tutoring 0 .00 .00 0 .00 0 .00 Study skills training 0 .00 .00 0 .00 0 .00 1 Test 0 .00 .00 0 .00 0 .00 Trident Technical 0 .00 .00 0 .00 0 .00 College Central Staff 0 .00 .00 0 .00 0 .00 program 1 Total 2637 .68 5.36 861 13.67** 148 2.85

*Difference between target and control students is significant at the p<.05 level. **Difference between target and control students is significant at the p<.01 level.

274 Table 17 PATHE Specialist Contacts, 1981-82 Rivers Middle School(N=530)

Entire School Target Students Control Students

Central Staff 34 .06 .06 7 .08 3 .04 program Total 1588 .65 2.95 572 6.73** 213 2.84

*Difference between target and control students is significant at the p<.05 level. **Difference between target and control students is significant at the p<.01 level. 27i Table 18

PATHE Specialist Contacts, 1981-82 Haut Gap Middle School (N=446)

Entire School Target Students Control Studer

Avg. Avg. Avi no. no. n( % contacts contacts cons:

Purpose No. of students per - No. of per No. of IN of contact contacts contacted student contacts student contacts stu(

Total Affective 510 .36 1.10 237 3.70** 2 Counseling 112 .14 .24 57 .89** 0 Discipline 70 .08 .15 7 .11 1 Student Leadership 53 .04 .11 5 .08 0 Team Peer counselee 4 .01 .01 0 .00 0 Peer counselor 43 .04 .09 21 .33 0 Field trip 33 .07 .07 17 .26** 0 Conference 154 .14 .33 120 1'.88** 0 Clubs/extracurricular 0 .00 .00 0 .00 0 activity Special program 37 .08 .08 8 .12 1 Rap session 4 .01 .01 2 .03 0 Total Academic 113 .07 .24 7 .11 0 Specialist tutoring 86 .02 .19 0 .00 0 Peer tutee 0 .00 .00 0 .00 0 Peer tutor 0 .00 .00 0 .00 0 "Outside" tutoring 2 .00 .00 0 .00 0 Study skills training 23 .05 .05 6 .09 0 Test 1 .00 .00 1 .02 0 Trident Technical 1 .00 .00 0 .00 0 College Central Staff 0 .00 .00 0 .00 0 program Total 623 .38 1.35 244 3.81** 2

*Difference between target and control students is significant at the p<.05 level. **Difference between target and control students is significant at the p<.01 level. 276 Table 19

PATHE Specialist Contacts, 1981-82 Brown High School (N=659)

Entire School Target Students Control Students

Avg. Avg. Avg. no. no. no. % contacts contacts contacts Purpose No. of students per No. of per No. of per of contact contacts contacted student contacts student contacts student

Total Affective 760 .38 1.15 197 2.43** 34 .36 Counseling 210 .18 .32 55 .68** 16 .20 Discipline 27 .04 .04 8 .10 2 .02 Student Leadership 276 .08 .42 33 .41** 2 .02 Team Peer counselee 0 .00 .00 1 .01 0 .00 Peer counselor 0 .00 .00 0 .00 0 .00 Field trip 103 .15 .16 20 z..25* 9 .12 Conference 138 .14 .21 77 .95** 5 .06 Clubs/extracurricular 6 .00 .01 3 .04 0 .00 activity Special program 0 .00 .00 0 .00 0 .00 Rap session 47 .05 .05 2 .02 7 .09 Total Academic 424 .20 .64 116 1.43 28 .36 Specialist tutoring 257 .10 .39 100 1.23 18 .23 Peer tutee 29 .02 .04 10 .12 9 .00 Peer tutor 60 .03 .09 0 .00 1 .01 "Outside" tutoring 2 .00 .00 2 .02 0 .00 Study skills training 0 .00 .00 0 .00 0 .00 Test 4 .00 .01 1 .01 0. .00 Trident Technical 72 .10 .11 3 .04 9 .12 College Central Staff 0 .00 .00 0 .00 0 .00 program Total 1203 .46 1.82 314 3.88** 62 .79

*Difference between target and control students is significant at the p<.05 level. * *Difference between target and control students is significant at the p<.01 level. 277 Table 20

PATHE Specialist Contacts, '1981-82 Burke High School (N=1005)

Entire School Target Students ontrol Students

Avg. Avg. Avg. no. no.. no. % contacts contacts contacts Purpose No. of students per N.:., of per No. lf per of contact contacts contacted student corts student cont. -4tv..1.ent

Total Affective 1106 .49 1.22 22, 2.64** 1 0,) .:.".:5

Counseling 165 .14 .18 73 .85** ll, .18 Discipline 494 .33 .54 59 .69 52 .65 Student Leadership. 182 .06 .20 16 .19 1:i .16 Team

Peer counselee 68 .03 .07 8 .09 5 a .06 Peer counselor 60 .03 .07 14 .16 4 .05 Field trip 28 .03 .03 4 °.05 3 .04 Conference 62 .06 .07 51 .59** 2 .02 Clubs/extracurricular 0 .00 .00 0 .00 0 .'00 activity Special program 0 .00 .00 0 .00 0 .00 Rap session 47 .05 .05 2 .02 7 .09 Total Academic 883 .15 .97 391 4.55* 94 1.18 Specialist tutoring 810 .13 .F9 360 4.19* 94 1.18 Peer tutee 0 .00 .00 0 .00 0 .00 Peer tutor 3 .00 .00 0 .00 0 .00 "Outside" tutoring 54 .02 .06 31 .36* 0 .00 Study skills training 0 .00 .00 0 .00 0 .00

Test . 0 .00 .00 0 .00 0 .00 Trident Technical 16 .02 .02 0 .00 0 .00 College Central Start 0 .00 .00 0 .00 0 .00 program Total 1996 .53 2.20 621 7.22** 194 2.42.

*Difference between target and control students is significant at the p<.05 level. **Difference between target and control students is significant at the p<.01 level. 1278 Table 21

PATHE Specialist Contacts, 1A1-82 St. Johns High School (N=808)

Entire School Target Students Control Students

Avg. Avg. Avg. no. no. no. % contacts contacts contact. Purpose No. of students per No. of per No. of per of contact contacts contacted student contacts student contacts studen.

Total Affective 1660 .70 1.99 375 4.181 ** 113 1.49 Counseling 411 .27 .49 47 .60 48 .63 Discipline 69 .06 .08 8 .10 7 .09 Student Leadership 151 .08 .18 53 .68** 3 .04 Team .01 Peer counselee 6 .01 .01 1 .01 1 Peer counselor 81 .02 .10 4 .05 0 .00 Field trip 0 .00 .00 0 .00 0 .00 Conference 290 .12 .35 205 2.63** 0 .00 Clubs/extracurricular 5 .01 .01 0 .00 0 ;00 activity Special program 59 .02 .07 4 .05 11 .14 Rap session 588 .49 .70 53 .68 43 .56 Total Academic 491 .31 .59 105 1..75** 29 .38 Specialist tutoring 250 .16 .30 72 .92* 20 .26 Peer tutee 4 .00 .00 2 .02 0 .00 Peer tutor 47 .02 .06 11 .14 1 .01 "Outside" tutoring 0 .00 .00 0 .00 0 .00 2 .03 Study skills training 33 .04 .04 i 5 .06 Test 62 .07 .07 7 .09 2 .03

Trident Technical 94 .08 .11 I 8 .10 4 .05 College 0 .00 Central Staff 1 .00 .00 0 .00 program Total 2279 .77 2.73 495 6.34** 155 2.04

p<.05 level. *Difference between target and controlstudents is significant at the p.01 level. **Difference between target and controlstudents is significant at the 273 Virgin Islands Alternative Education Project

J. St. John

Abstract

The Virgin Islands Alternative Education Project is one of a number of demonstration projects sponsored by the Office for Juvenile Justice and Delinquency Prevention as part of its Program in Delin- quency Prevention through Alternative Education. Itis seeking to implement two interventions disseminated by the National Diffusion Network (NDN)--Focus and PATL. Interim evaluation results suggest that Focus is being implemented largely as intended, but with some changes from the original Focus model. PATL is being implemented to a more limited degree, due largely to a limited number of teachers implementing the model. Despite some weaknesses in the evaluation design and measures, results also ,suggest that the modified Focus intervention resulted in students receiving higher grades than they otherwise may have received. Although project implementers believe they perceive positive results as a result of their activities, no other consequences ofthe Focus intervention were detected using measures obtained in the evaluation. This outcome may be a result of the small number of teachers implementing the intervention or to incomplete implementation. Some positive changes in the school climate were observed, but there is no evidence in a comparison of students exposed in differing numbers of teachers trained in PATLind that the limited implementation of PATL has had detectable effects on students. Program development is continuing, as are efforts to strengthen the evaluation.

Major Program Design Changes structural beginnings of a school- within-a-school were launched for The Virgin IslandsAlternative the Focus Program. In addition, a Education project continued its teacher coordinatorwas appointed efforts to implement the National for the PATL Program and peer train- Diffusion Network (MN) Focus (Focus ing among teachers began anew. Dissemination Project, 1979, 1980) and PATL (Pelan, Zirges, Scroggins, Thestaff could not attend the & Arterberry, 1975) programs during August, 1981, planning workshop. the 1981-82 school year. Major Consequently their Program Develop- changes in the program included the ment Evaluation (PDE) worksheets did opening of an Alternative Education notdetail plans for the 1981-82 building on the Elena Christian academic year. Without these plans Junior High School campus. Although it was difficultfor us to know thebuilding hadonly two of the which program components they four needed classrooms, the intended to implement or the degree of implementation success. For example,an intensive public relations campaign was a component during the first implementation year. This report covers the period of The campaign took on a new slant the September 1981 to June 1982. second year because the school

-393- 250 Virgin Islands system incorporated alternative edu- fer to a n- school expected to open cationandthere wasnolonger a in the fall. Allbut oneof the need to lobby for survival. This Focus-tra. ted staff requested trans- development resulted in a change in fer, and by Augu....t. the new in-school project priorities. coordinator knew she would have at least three untrained teachers join- Major Forcefield Changes ing the staff. Fortunately, new teachers whowereacquaintedwith In the fall, the director of the Focus and were committed to its project, representing the Commis- ideals joined the staff. New train- sioner's Office, recommended the ing sessions were held with NDN advisability of the services of a trainers during the first weeks of clinicalpsychologist onthe proj- the 1982-83 school year while stu- ect. He suggested that psychologi- dentrecruitmentwasin progress. caltesting (including intelligence Staff .began the upcoming school year testing) be used asa part of the with excitement and optimism. selection process. and that indiVi- dual behavior management programs be Implementation based onthe use of psychological tests. The underlying philosophy of We attempted to assist in program this approach is counter to the phi- planning and PDE updating during losophy of project staff andwas 1981-82 by phoneand mail. This regarded as counter to the philoso- method was time consuming, and phy of Focus as well. In addition, resulted in incomplete documentation the cost of implementing this recom- of project plansandtheirexecu- mendation was prohibitive.. There- tion. As the -school" year-pro- fore, afternegotiations with the gressed, the Focus staff devoted ps -Thologist, the project decided considerable timeto discussion of not to implement most of the recom- theProjectDirector'srecommenda- mendations, although changes in tions (discussed earlier), and this record-keeping procedures were may have slowed further development adopted. of the Focus intervention.

The State Department of Education Our obF-lvations suggest that created a division of Alternative more monitoring of the Focus inter- Education in the summer of 1981. vention inthecl ssroom; more in- During the school year the depart- school cool,inator attention to ment recruited staff, chose cl.rricu- attendance, discipline; and more lum, and began program planning with attention to seeking family assist- several junior high schools in the ance with school discipline problems system. The funded in-school coor- would have been useful. dinator helped the department establish its new division and consulted PATL Component with other schools. PATL is a teacher training pack- By late Spring, the in-school age intended to a) helpteachers coordinator had beenpromoted to learn alternative teaching and dis- Director of Alternative Education ciplinary methods,and b)confront forthe Virgin Islands school sy.3- their attitudes toward students and tem. One of the program's teaches._ the effects these attitudes have on took on the project coordinator students. The training program pro- responsibilities. Thespringalso ceeds in several steps. First, brought faculty requests for trans- teachers who volunteer to teach the

-394- 28 Virgin Islands curriculum participate in an all teachers who had participated in introductory four-day workshop. the PATL training were implementing After the workshop, traineesform the PATL techniques due to insuffi- groups of about five members. PATL cient knowledge. Therefore, imple- hasfour training "kits"--training mentation was probably at level 3 of group members work together to com- stage 1 for some teachers. plete the kits. Each kit has learning objectives that must be mastered Focus Component sequentially. Mastery is demonstrated by a 90% proficiency rating InMarch, 1982, I visited the on an in-classroom observation of project site to learn what had been thetechniques. The PATLprogram implemented. The assistance of an has four stages of implementation NDN consultant on focus was sought that correspomi with the completion to prepare to assess implementation of each kit. In addition, there are ofthe major program components of three levels ofimplementation for Focus before the visit. The consul- eachkit: (1) trainingcompleted, tant supplied a checklist of 11 com- (2) in process of implementation, ,ponents that included ranges of and (3) implemented. PATL develop- acceptable implementation. The fol- ers expect the program to begin lowing on-site observations are out- affecting those students having lined according to NDN's checklist. PATL-trainedteachers in at least three oftheir classes; minimally, 1. Family group meeting. Focus those teachers must be implementing calls for students to engage in PATL at the first of the four imple- daily problem-solving sessions mentation stages. together with Focus teachers. Stu- dents in theproject were grouped Plans to strengthen' the 1981-82 into four "family" units. One class PATL component were made during the period each day wasset aside for previous year when project managers the family groups to meet sepa- realized the component was not being rately. The observed family group given sufficient attention. Manag- was participating in a values clari- ers planned for the appointment of a fication exercise and not observed Teacher Coordinator. The coordina- resolving the problems of members in tor was to oversee the recruitment that particular session. Staff mem- of newteachersfor peer training bers interviewed reported that the and structure the program. The NDN family meetings were handled differ- PATL trainer was scheduled to return ently by each staff member, but they for a refresher workshop in Septem- did not fully describe what occurred ber. Each of these steps were in the meetings. implemented. The Teacher Coordina- tor reported, however, that peer 2. Student leadership board. training was not going as expected Focus calls for a student leadership because peer trainers and trainees board to help resolve problems not had difficulty finding time to meet. solved in family meetings. Project This was largely due to the school's staff thought this component should double session school day resulting not be implemented until the project in rigid schedules for teachers. had been operating for several years Consequently, by latespring many and effects had taken hold. trainees had reportedly covered only Instead, students wereinvited to one or two of the objectives in the attend staff meetings when problems first ,training kit. During an neededattention. This change is August, 1982, planning workshop, viewed bythe State Facilitator as project managers reported that not

-395- 282 Virgin Islands an adaptation ofthe program model dents seemed to be exhibiting rather as an interim strategy loyalty and cited student statements required while other aspects of the of support. At the end of the project develop. Indeed, the faci school year, however, some students litator views the entire Focus model requested they not be listed as as open to adaptation rather than as Focus students on the school promo a prescriptive modelfor interven tion list. tion design. 7. Use of reality feedback and 3. Mini school. The Focus model confrontation. Focus calls for the calls for a designated Focus area be consistent application of realityo established within the school, and riented feedback, and staff and stu for Focus staffto have their own dent confrontation of inappropriate offices. At the project site, a behavior. Such an apprOach seeks to campus Alternative Education build insurethat studentsreceive spe ing had been opened but two ofthe cific and immediate feedback on four Focus classes were being held their behavior and are given an elsewhere in the school building. ,opportunity to change. Some of the Division of the two Focus classrooms teaching staff were using these intofourrooms wasscheduled for techniques and others were not. In completion during the summer of one Focus classroom a student 1982. entered late and was wearing a hat. The teacher confronted his lateness 4. Parental involvement. Focus and the student gave the teacher a calls for "some kind of positive late pass. Wearing a _hat was support" on the part of parents. At against school rules and the teach-er the Virgin Islands project, parents asked him what the hat was doing on of project participants must sign his head. He promptly removed the permission slips before their chil hat. In another Focus classroom the dren can beconsidered for selec same behaviors bythe same student tion. In addition, project plans went without comment. call for teachers to report to parents their students' progress, 8. Scheduling student's day. and to seek parents' help with dis Focus calls for at least four hours cipline problems. Records reviewed of instruction a day in the program. on site showedthat at leastsome Focustrained staff should teach parents were contacted by some Focus students a Focusdeveloped teachers, but no systematic report family curriculum, and three of the ing occurred. students' regular academic subjects. Each project family group was sched 5. Program contracts. Focus uled in five Focustaughtcourses calls for students to set individual and in regular schoolcourses for contractual goals. At least one the rest of the day. The Focus project staff person had contracted courses were occupational relations, withstudents in hergroup. The family, English, math, and social contracts served as a means of moni science. The occupational relations toring and measuring progress. course was designed to expose stu dents to careers of all types, help 6. Student ownership of the pro students develop job skills, and gram. Focus sets up the expectation help students gain everyday practi that students "will demonstrate loy cal skills. alty to the program in some way." Project managers reported that stu

396

283 Virgin Islands

9. Community involvement. Focus instance, the English teacher recommends that junior high students reported using a highly individual be involved in one community service ized approach that was structured by project each semester. Senior high test results, while the math teacher students should eitherparticipate reported usinga modelwhere stu in jobsor community service proj dents studied when they wanted. In ects. Over the course of implemen the English classroom, the teacher tation, fiveprojectstudents have and student developed learning had jobs.A publicrelations cam objectives,and students worked on paign implemented byprojectstaff individualized materials. In the was substituted for the recommended math classroom, the teacher reported student community service projects. that students were given a specific time period to master assigned 10. Assessment. Focus recom materials from the regular math mends that Focus participants be book, .and weretestedafter each pre and posttested in "three main unit. In both classrooms the teach academic areas". Project managers ers gave students individual atten improved on this simple prepost tion. evaluation design byattempting to implement a more rigorous quasiex On the day I observed the perimental design. They added to classes, attendance was low. While the assessment measurements relevant observing oneclass withabout10 toa delinquency prevention demon students present, students from the stration project. Next year the other class in the alternative edu project will be implementing a true cation building were milling about experiment and readopting the pre on the porch. In a laterFocus post approach as a method of moni class, onlythreeof the students toring student progress. present in the class of 10 observed earlier were on time. One more of 11. Admission policy. Focus the students came to class later. recommends that current participants andstaff decide which new appli Observation limited to a single cants should be admitted. The proj day is not, of course, a dependable ect staff substituted an admissions way of describing a school. The policy that both identifies the typical day may have looked either troubled students the program is better or worse than the one intended to serve, and allowsfor observed. the formation of participant and control groups necessary for an Effectiveness evaluation (see below fora more detailed description ofthe selec tion process). Focus Evaluation Design

Other observations. The Focus The national evaluation staff and program does not prescribe curricu project staff agreed to study Focus lum or classroom methods beyond the effects by using a regression dis use of reality feedback and confron continuitydesign (Cookand Camp tation; consequently each teacher bell, 1979). This design was chosen was using either the required Virgin because itallowedservices tobe Islands curriculum, or curriculum of given tostudents mostin need of their own design for the family the program. The project staff classes. Teaching methods were up developed a screening device on to the teaching staff. For which referring teachers were asked

397

284 Virgin Islands to rate students' academic ing Alienation, Belief, School abilities, and indicateattendance Effort, Practical Knowledge,Posi and behavior problems. Students' tive SelfConcept, School Attach scores in thefirst two waves of ment, Selfreported Total and Seri eligible referrals were ranked, and ousDelinquency, andSelfreported the median score determined the Drug Involvement. Items on selfre selection pool. Those scoring above ported school and class skipping, the median became Focus partici and suspensions were also included. pants, and all othersbecame the These scalesand itemswere used control group. Students referred to either because the project was the program later had to score above expectedto affectthese outcomes, the original median score to be orbecause they areknown to be selected as participants. This strong correlates of delinquency. score was intended to serve as pre Scores were derived as they had been participation information to be used forSAES scales in theSpring of in statistical analysis. 1982--summing the standardized scores ofthe items comprising the Project staff executed each step scales (Gottfredson, Gottfredson, for design implementation as Ogawa, and Rickert, 1982). planned, butthe design broke down because ofa weak relation between School data from the first school screening device scores and out quarter were also used as preparti comes. (Interpretable regression cipation measures because the proj discontinuity designs depend upon a ect staff were recruiting and sche strong relation between pretreat dulingstudents atthattime, and ment measures and outcomes.) formal amplementatiOn-did not begin until the secondquarter of the Selection procedures for the school year. Although the question regression discontinuity design had naire pretest did not occur until resultedin nonequivalent groups. Jgnuary, it was also used as a Table 1 illustrates that small dif source of "control" variables in the ferei.ces between Focus and compari analyses because ofthe nonequiva son students are apparent for grades lence of the treatment and compari in the first quarter ofthe school son groups. Its use, is a conserva year (before the Focus interventions tive procedure. Any were begun in earnest) and a moder preparticipation measures (either atedifference inattendance. We school or questionnairebased) that had forseen the possibiliry of correlated with both treatment and design failure and had administered outcome were used as statistical a brief version of the School Action controls for outcomes measured at Effectiveness Study(SAES) student the end of the year. questionnaire in early January, about five school weeks after most students were admitted to the pro Outcome Measures gram. The SAES questionnaire was admin Administration of the SAES. ques istered in late April. The project timnaire pretest, coupled with' the evaluator reported school and class controls originally identified for room attendance, schoolgrades at the regression discontinuity design the end of quarters one and two, and created a quasiexperimental none again after the second semester was quivalent control group design. The completed. School discipline data pretest included SAES scale3 measur- were reported attheendof first

-398

285 Virgin Islands andsecond quarters. Metropolitan isting differences among students Reading Testscores from the fall (beta). When statistical controls and spring administrations and are applied, only the association of police contactinformation forthe Focus participation with GPA remains previous year were reported in June, statistically significant. The 1982. Unfortunately, 47% of the adjustedassociations observedfor Spring achievement test data for the the other two outcomes, although in Focus students were missing, making the postive direction, are not sta these data of little value. tistically significant.

Results PATL. Table4 describes PATL outcomesgrouped according to the Focus. Mean scores on the ques number of teachers exposed to PATL a tionnaireadministered in Spring, student had, That is,if a student 1982, and means for outcomes meas had one teacher who had completed a uredthrougharchival records are fourday workshop, he or she is shown in Table 2. The grade point listed under "One PATL Teacher." If average (CPA) for Focus participants the student had two such teachers, was significantly (p<.01) higher he orshe is included inthe "Two than the grade point average for the PATL Teacher" group. The table comparison students. No other dif classifies students in this way, ference shown in Table 2 is statis irrespective of the extent of con tically significant. Two other out tinued peer training and ofdegree comes measured by spring of implementation of PATL methods by questionnaire items differed signi teachers. No differences are sta ficantly. Selfreported grades tistically signficant. There is no favored the Focus group (p<.05), and consistent relation between the num Focus students reported less class ber of teachers exposed to PATL and skipping than did the comparison student outcomes. students (p<.01). Discussion Recall that the comparison group was not equivalent to the Focus group atthe outset, however. In Focus other words, the treatment and com parison group differed systemati . New project managers were asked cally atthe outset in waysthat whether the improved grade point would lead them to differ at a later average of Focus participantswas timein systematic ways regardless indicative of mastery of the subject of whether or not they were exposed matter, or reward for effort. They to anyintervention. In such cir believed that the students' grades cumstances it is necessary to use were based more on effort than mas statistical techniques to adjust terybecause the Focusphilosophy postintervention differences calls for the use of such rewards as differences to be expectedon the motivators for further improvement. basis ofthese preexisting differ Because of this, the results for GPA ences. Table 3 shows the unadjusted may represent a measure of implemen association (r) of Focus participa tation, andmay not address the tion with GPA, selfreported grades, question of mastery. and selfreported class skipping. It also shows the association when Themost directway to assess statistical controls are used to mastery is through the use of stan adjust for the measures of preex- dardized achievement tests. Unfor-

-3997 286 Virgin Islands

tunately, standardizedachievement pleted. The new in-school coordina- test scores were available for only tor plans to work closely with the 53% ofthe students, and could not PATL teacher coordinator toremove be used to address the issue ofmas- obstaclesto peertraining. Peri- tery. Theevidence is clearthat odic visits to classrooms of trained grades where higher than they other- teachers are expected to improve the wise would have been for treatment the match between program design and students; the interpretation of this implementation. result is necessarily ambiguous. School-Level Results Observations of the implementation of Focus are encouraging. Results reported elsewhere (Gott- Although not yet implemented asper fredson, Gottfredson,& Cook, 1983, NDN specifications, 1981-82 Focus Chapter 5) show some- changes in implementation improved substan- school, climate between 1981 and tially over the previous year. For 1982. Specifically,increases were 1982-83, the new in-school coordina- observed in teacherjob satisfac- tor is committedto implementation tion, teacher non-authoritarian improvements. She is especially attitudes, and students' reports of committedto developingstrategies school rewards. Decreases were for increasing class attendance. observed in self-reported school Finally, the managers' commitment to grades, students' reports of commu- a stronger experimental design nity crime, and students' reports of should reduce evaluation problems drug involvement and delinquent and give a better picture of 1982-83 behavior more generally. Although effects. the detailed interpretation-of-these results is difficult, they are encouraging. Decreases in reports PATL of classroomdisruption,which we expected to observe because project PATL implementation did not occur staff believed this to be a conse- asplanned. The projectplans to quence of placing disruptive stu- strengthen both the PAIL training dents in the Focus program, were not component, and the implementation of observed. PATL concepts after training is com-

-400-

287 Virgin Islands

References

Cook, T. D., & Campbell, D. T. Quasiexperimentation. Chicago: Rand McNally, 1979.

Focus Dissemination Project. Focus curriculum manual (2nd ed.). Hastings, Minn.: Educational Growth Exchange, 1980.

Focus Dissemination Project. Focus training manual (3rd ed.). Hastings, Minn.: Educational Growth Exchange, 1979.

Gottfredson, G. D., Gottfredson, D. C., & Cook, M. S. The School Action Effectiveness Study: Second interim report (Report No. 342). -Baltimore: The Johns Hopkins University, Center for Social Organizationof Schools, 1983.

Gottfredson, G. D., Rickert, D. E.. Jr., Ogawa, D. K., & Gottfredsdh,D. C. Measures used in the School Action Effectiveness Study. In G. D. Gottfred son (ed.), The School Action Effectiveness Study: First interim report. Baltimore: The Johns Hopkins University, CSOS, 1982.

Pelan, C., Zirges, J., Scroggins, F., & Arterberry, M. Positive attitude toward learning. Washington, D.C.: Office of Education, Department of Health, Education, and Welfare, 1975.

401

28R Table 1

Means and Standard Deviations for Focus and Comparison Group First Quarter 1981-82 Pre-Intervention Measures

Focus Comparison

Measure ti SD N M SD

Grade point average- all subjects .1.59 .76 51 1.41 .72 49

Grade point average- core subjects 1.43 .80 51 1.19 .73 49

Total school days absent 7.10 8.16 41 3.68 6.03 37

-402- 2RQ Table 2

Means and Standard Deviations for Focus and Comparison Group Spring 1982

Focus Comparison Outcome measure N SD N M SD

Questionnaire-based Alienation .47 .24 25 .49 .25 28 Attachment to parents .52 .28 30 .57 .28 31 Belief .57 .28 28 .64 .26 28 Interpersonal competency .68 .27 28 .72 .24 26 Negative peer influence .28 .20 30 .21 .18 30 Parental emphasis on education .52 .30 29 .49 .34 28 Positive self-concept .61 .15 24 .56 .20 24 Practical knowledge 1.31 .45 28 1.13 .45 26 Rebellious autonomy .50 .30 26 .53 .28 27

School attachment .58 ,:i .22 26 .66 .27 31 School effort .52 .32 29 .61 .27 29 School involvement .22 .19 29 .25 .17 26 School punishment .27 .33 29 .26 .26 26 School rewards .39 .32 29 .35 .36 26 Self-reported delinquency- total .12 .15 25 .10 .12 27 Self-reported drug involvement .14 .22 26 .12 .15 27 self-reported serious delinquency .10 .15 25 .07 .13 27 Victimization .22 .25t 29 .20 .26 26 School records Disciplinary infractions a a a a a a Grade point average 1.44 .92 49 .97 .74 45 Total absence 20.00 22.78 49 15.55 20.13 44 Police records Police contacts .09 .29 54 .12 .48 51

a Data not reported

-403- 29n Table 3

Correlation Coefficients and anti Standardized Regression Coefficients in the Regressions of Outcomeson Focus Participation Spring 1982

Coefficients Measures r beta

School report of gradesa .267** .274* 58 b Self-report of grades -i300* .189 -45 b Self-report of class skipping .324** .220 46

Note. Table shows the three outcome variables with significant correlations. N shown is the lowest pairwise n.

a First quarter school reports of grade pointaverage and absenteeism, and January SAESmeasures of Alienation and Serious delinquencywere used as statistical controls. b First quarter school reports of absenteeism and class skipping, and January SAES measures of Alien- ation, Serious delinquency, School attachment,and Self-reported class skipping were usedas statistical controls.

*p<.05 **p<.01

-404- 2q1 Table 4

Means and Standard Deviations for Students by Number of PATL- Trained Teachers, Spring 1982

No PATL One PATL Two PATL Three PATL Teacher Teacher Teachers Teachers Outcome measure SD N M SD N M SD N M SD

Questionnaire-based Alienation .41 .28 18 .40 .22 47 .43 .21 44 .42 .27 24 Attachment to parents .71 .18 20 .63 .24 50 .64 .29 50 .57 .29 25 Belief .62 .22 19 .63 .26 50 .66 .24 44 .62 .25 23 Interpersonal competency .83 .17 19 .77 .24 49 .77 .28 41 .74 .24 24 Negative peer influence .15 .17 21 .17 .18 49 .21 .20 50 .20 .15 23 Parental emphasis on education .63 .29 21 .60 .29 50 .61 .31 44 .61 .34 24 Positive self-concept .72 .20 19 .68 .21 45 .71 .19 39 .67 .17 20 Practical knowledge 1.33 .36 19 1.24 .44 49 1.14 .41 42 1.19 .44 23 Rebellious autonomy .32 .36 19 .54 .37 49 .58 .33 43 .61 .27 22 School attachment .76 .23 20 .66 .23 49 .68 .23 49 .65 .21 24 School effort .68 .26 20 .66 .29 48 .64 .26.7 49 .62 .29 24 School involvement .23 .18 18 .21 .18 45 .23 .15 47 .28 .22 20 School punishment .17 .26 18 .20 .22 45 .25 .28 46 .25 .29 23 School rewards .24 .28 18 .39 .34 45 .25 .27 46 .38 .33 23 Self-reported delinquency- - total .08 .10 18 .07 .11 .46 .07 .10 47 .07 .07 18 Self-reported drug involvement .09 .10 16 .09 .14 46 .08 .15 48 .10 .14 19 self-reported serious delinquency .05 .09 18 .05 .11 .46 .04 .08 47 .02 .04 18 Victimization .09 .14 19 .16 .21 45 .14 .18 45 .19 .23 22 School records a a a a a a a a Disciplinary infractions a a a a Grade point average 1.76 .94 42 1.49 1.01 193 1.72 .95 89 1.73 .82 46 Total absence 8.52 16.24 40 16.29 23.15 93 9.0114.16 88 6.48 10.35 46 Police records I Police contacts .02 .15' 45 .11 .49 100 .02 .14 95 .06 .25 47

a Data not reported

.`4, 9 2 Interim Quantitative Evaluation of the Academyfor Community Education

D. E. Rickert, Jr.

The Academy for Community Educa- considerableeffortin creating a tion isa small alternative school matched control group. Essentially sponsored by the Office for Juvenile the same screening procedures used Justiceand Delinquency Prevention to select students for participation as part of that Office's Program in in the Academy were used (ata later Delinquency Prevention through time) toscreen studentsfor the Alternative Education. The Academy control group (Daniels, 1983). uses a token economy system, academic education, professional/voca- Analyses proceeded in several tional curriculum, and other inter- steps. First, treatment and control ventions describedmore fully by group.students were compared on the Daniels (1982, 1983) inproviding available pre-intervention data. services to youths at high risk.of And, because many students partici- delinquent behavior drawn from the tipated in theAcademy for only a Dade County (Miami) Public Schools. short period oftime (68.8%,or 75 The present brief report summarizes of109, hadatleastsix months' some interim quantitative results of involvement in the Academy),compar- the evaluation of this project as it isons between long-term (six months operated during the 1981-82 school or more) and control group students year. Qualitativeresults of the were also made on pre-intervention evaluation aredescribed elsewhere measures. Second, comparisons of (Daniels, 1983). Academy persisters and dropouts were made to determine whether differen- Method tial attrition might bias the results. (Only 2.2% of students Planswere made to assess the ever enrolled in the project dropped project using a non-random but outduring the year,a relatively closely matched controlgroup for small percentage in projects of this whom both pre- and post-intervention kind.) Third, comparisons of treat- data were to be available.1 This ment and controlgroups were made design was partially implemented: with and without the application of Pre-and post-interventiondata on covariates to increase the statisti- absences, suspensions, tardiness, cal power of the analyses.3 academiccredits earned, standardized achievement test performance, and police contacts were available for analysis.2 The Academy expended measures were available for the control group, and post-intervention questionnaire measures were available for only 37% (21 of 57) of the 1. In itsinitialgrant applica- control group students. An account tion, the Academy was to operate as of the difficulties in implementing a true experimental field trial, the planned design may be found in with eligible youths randomly Daniels (1983). The present report assigned to treatment and control is limited to the most meaningful group conditions. For a number of comparisons: those involving treat- reasons described by Daniels (1982), ment and contr-1group students on this expectation was not realized. archival information. 3. A covariate is a variable known 2. No pre-treatment questionnaire to becorrelated with the outcome

-407- ACE

Results A comparison between students persisting in theAcademy for at No statistially significant dif- least six months and students who ferences were found between treat- dropped out with fewer than six ment and control groups on any of months' participation is shown in the pre-intervention measures avail- Table 2. The table shows that per- able foranalysis. Nor were any sistence inthe Academy is essen- statistically significant differ- tially unrelated to any of thepre- ences found between long-term Aca- intervention measures examined. demyparticipants and the control group on any of these measures. Post-intervention comparisons, These results, summarizedin Table between treatment and control group 1, imply that the matching procedure students are summarized in Table 3. implemented by the Academy resulted Treatment students were suspended in a good match between treatment significantly fewer times than con- and control groups. Furthermore, trol group students, withdrew invo- such differences among the groups as luntarily significantly more times, are observed are relatively small in rand earned significantly more aca- comparison to the individual differ- demic credits. Although not quite ences within 'groups (compare the significant by conventional stan- differences between means in Table 1 dards, treatment students were also with the standard deviations which tardy less often than were control are indicators of the extent of students. No significant differ- individual differences). ences were found for absences, num- . ber of police contacts, or standardized achievement test scores. Table 3 also shows that the same general pattern of resultsis present when only the long-term Academy participants are included in the analyses, variable of interest which is some- except that(of course) the exclu- times introduced into an analysis to sion of students who withdrew invo- adjust for pre-existing differences luntarily from the Academy from the between groups when the groups are analyses alters the "result" for not randomly equivalent. In the this form of withdrawal. present case,as we shall see, the treatment and matched control groups A statistically more powerful did not differ greatly on pre-inter- form of analysis that takes pre-ex- vention measures. Here the primary isting individualdifferencesinto purpose in the use of covariates was account is summarizedin Table 4. to increase statistical power: The In these analyses the results signi- probability that a "true" difference ficantly favorthe treatment group between the groups will be detected for number ofabsences,number of by statistical analyses. The logic suspensions, number of times tardy, of the analysis of covariance, as credits earned, and the Math Compu- this technique is called, isthat tation and Reading sections ofthe some of the differences among indi- Stanford Achievement test. The dif- vidualsin both thetreatment and ference in number of police con- control groups can be accounted for tacts, although favoring the treat- bypre-existing individual differ- ment group (especially for long-term ences. Bytaking theseindividual differencesinto account, analyses can sometimes focus in more closely on group outcomes due to the treatment.

-408- ACE

persisters), is not significant. treatment group was found for two of Even in these very powerful analy- the four test scores examined. ses, no significant differences were found for standardized test perfor- Despite these limitations, the mance in the areas of math concepts following interpretations appear or language. reasonable.

Discussion 1. Participation in the Academy results in less absenteeism than The present results are positive participation in the public schools. and impressive, but before elaborat- The difference, although not large, ing on their implications, some is significant. limitations must be described. The most important limitation isthat, 2. Participation in the Academy with the exception of police records results in fewersuspensions than and standardized testscores, these would occurin the public schools. archival measures 'mayinvolve some The difference is of moderate size. unknowndegree of instrumentation bias. That is,it is possible that 3. Participation in the Academy records of absences, suspensions, results in less tardiness than would tardiness, and credits earnedmay be occur if thestudents remained in kept in somewhat different ways by the public schools. The difference the public schools on the one hand is of moderate size. andthe Academy onthe other, or that the public schools and the Aca- 4. Participation in_the Academy _ demy may differ somewhat in the ways results in earning more academic these outcomes aredefined. This creditthan wouldbe expected if limitationis, of course, largely students remained in the public unavoidable in a field trial suchas schools. The difference amounts to this one. A second important limi- abcat 1.77 credits for studentsever tation isthat these analyses bear participating in theAcademy, and only on a part of the outcomes the amountsto about 2.74 credits for evaluationseeks to examine. The students who persist in the Academy unavailablity of questionnaire meas- for at least six months. ures is a serious shortcoming. 5. During the 1981-82 school The standardized achievement test year, however, Academy participants data are especially important withdrew involuntarily more often because, lacking questionnaire data than students remaining in the pub- collected in parallel ways for both lic schools. treatment and control groups the only other data that were collected 6. Participation in the Academy in perfectly parallelways for both results in higher scores on the Math groups (police contacts) did not Computation and Reading subtests of show significant differences. A the Stanford Achievement Test. significant difference favoring the

-409- ACE

References

Daniels, D. Academy for Community Education: First interim report. In G. D. Gottfredson (ed.), The School Action EffectivenessStudy: First interim report. Baltimore: The Johns Hopkins University, Center for Social Organ- ization of Schools, 1982.

Daniels, D. Academy for Community Education: Second Interim Qualitative Report. San Rafael, Calif.: Social Action Research Center, 1983.

/ Table 1

Pre - Treatment Comparisons of ACE and Control Students 1980-81 School Year

Total ACE Long-term ACE Control Variable SD N N SD N N SD

Number of days absent 20.7 19.8 108 21.0 19.7 75 19.9 17.7 5E

Number of times suspended 0.7 1.2 104 0.7 1.2 75 0.5 1.4 5E

Involuntary withdrawal 0.0 0.0 103 0.0 0.0 75 0.0 0.0 5f

Number of days tardy 7.2 7.4 82 6.7+ 6.6 59 6.4 6.1 44

Credits earned 7.4 3.4 109 7.4 3.4 75 8.5 3.4 5E

Number of police 0.1 0.4 109 0.0 0.1 '468 0.1 0.2 57 contacts

Stanford Achievement Test, national percentile

Math computation 37.0 22.5 105 36.9 23.3 72 43.0 24.9 5] Math concepts 37.7 22.9 99 38.1 23.4 70 40.5 28.8 5( Language 33.2 25.0 106 30.1 23.7 73 33.5 23.4 5] Reading 34.9 23.3 83 33.0 22.7 64 35.8 26.4 4]

Note. N's differ among outcomes due to missing data. N's are as follows' disregarding missing data: ACE total, 109; ACE long-term, 75; control, 57. Long-term ACE participar had at least six months of treatment.

+This mean is almost significantly different from the control group mean:Q < .06.

% 9 Table 2

Pre-Treatment Comparisons of ACE Persisters and ACE Dropouts

ACE Persisters ACE Dropouts Variable M SD N M SD

Number of days absent 20.7 18.9 80 21.3 23.2 26

Number of times suspended 0.8 1.2 79 0.5 1.0 23

Involuntary withdrawal 0.0 0.0 75 0.0 0.0 26

Number of days tardy 7.0 7.0 60 8.2 9.0 20

Credits earned 7.6 3.3 80 6.6 3.5 27

Number of police 0.0 0.2 74 0.3 0.8 22 contacts

Stanford Achievement Test, nationalpercentile

Math computation 38.1 21.7 76 33.3 24.8 27 Math concepts 35.9 21.9 72 44.2 25.0 25 Language 31.9 23.5 77 36.0 30.0 27 Reading 34.9 23.6 70 34.5 22.6 13

Note. N's differ among outcomes due to missing data. N's are as follows disregarding missing data: ACE persisters, 80; dropouts, 27. Long-term ACE participants had at least six months oftreatment. Oetiwtokele r2011 end (esteil iliedmeits UNA! Odes. War

DallettiSMI*0111141116100

$0 o II SO allalaWagaMilltlfaNtiltrnetitoisa.auarsairtariamisahpnrroprpx* 11100.11111.111.111411.041.1114MOMINIF

00 ilow* moos 1,40 10.2 *L4 ISA 13 21.11 21.$ 3

411000 410 0400,iosopisAliwo 10.410 0,f 100 OA*.0.3 13 0.4 1.4 3

"441100144 06440 0.O 101 0,0 0.2 73 0.0 0.0 3

44,419 'jai LA 911 1,4. OA Hi 11.6 12.0 3

40104,441 SA* SA WI 10.0**1.0 H 7.2 LI 3 "Wm tot 0J 0.1 10 0.0 04 6$ 0.1 O./ *ammo flowhoof fliotwoolo optimpos444

ft* ompsoaro. 4010 42.6 214 50 9.3 24.0 4$ gooshoo 40*.1 11.$ 144 41.3 211.1 *I 41.2 26.2 44 WINO 04 104 334 23.4 20 9.9U.1 41 itoo*** IIVAA VOA MI WO NA 14 11.0 U.) 41 vairsmatoryorgiemvrnt,,trilpprinprorroppeorrximprrisrersortwunretaverisaersiagribeseuraiveimmoamerris.rn,...-a......

sio .44441_4041.Ora u* se follows dieropriltas 140111114.09,14 3 cosuol. 91.lom,-trei putitetpe

4, 0494$44144 tom 44. 44414-4.4 sm. wow s c .06. oroop woe at tise $ Taut. poop ems eit ilea.01 WM. 299 Table 4

Adjusted Post-Treatment Means for ACE and Control Students with Pre-Treatment Data used as Covariates: 1981-82 School Year

110011.111. Total ACE Long-term ACE Treatment Control Treatment Contr6 SIIMMIIVariable

Member of days absent 16.82* 108 22.06 52 18.37 75 22.06

Member of times suspended .16** 103 .65 55 .05** 75 .64

Member of days tardy 7.86** 75 13.33 31 7.91** 55 13.32

Credits earned 9.25** 102 7.48 56 10.18** 73 7.44 number of police .17 98 .19 56 .03 68 .18 contacts

Stanford Achievement Test, national percentile ,-.

Math computation 42.07** 96 35.46 44 46.08** 56 34.47 Math concepts 40.42 101 40.46 44 44.00 64 40.24 Language 35.35 111 38.67 44 36.59 69 37.93 Reading 38.90* 83 32.60 41 38.90* 64 31.70

MONIIIMON11.

Nag. N's differ among outcomes due to missing data.N's are as follows disregardini missing data: ACE total, 109; ACE long-term, 75; control, 57. Long-term ACE participants bad at least six months of treatment.Analysis of covariance was performed usil the 1980-81 variable corresponding to each 1981-82 outcome as a covariate. No expulsions occurred for any control or treatment group student for whom pre-treatment data mere available.

*This mean differs from the control group meanlat the < .05 level.

**This mean differs from the control group mean at the 2 < .01 level. 300 Alternative Education for Rural Indian Youth: Lac Courte Oreilles

M. Cook

Abstract

The Alternative Education for Rural Indian Youth project (here- after referred to as the LCO project) wasone of several projects funded by the Office of Juvenile Justice and Delinquency Prevention (OJJDP) aspart ofits Program in Delinquency Prevention through Alternative Education. The LCO project operated on the Lac Courte Oreilles Indian reservation, located ina remote part of.north-central Wisconsin. The project maintained two drop-in Youth Centers, installed PLATOcomputerized instruction. in the Youth Centers, developeda Youth Outreach effort,and partially supported PLATO instruction in the LCO reservation schools.

The project sought to decrease delinquency on the reservation by providing recreation services to youth, and by decreasing drop-out from the LCO schools through the support of PLATO instruction. A variety of implementation difficulties including loss of funding for other reservation projects, staffing problems, unclear specification of goals andobjectives, and a scattershotapproach to program development resulted in the program not beingfully implemented nearly two years after initial funding. The outcome evaluation is inconclusive due toa lack ofdata,butlogicaland theoretical analysis suggests that the project failed to implementa program that might reduce delinquency.

Theory Letters of support for the grant application advocate forfunds to The LCO grant proposal was a mas- maintain drop-in youth recreation sive 500+ page work. Despiteits centers (Youth Centers) to control length this volume contains no cohe- delinquency. Although the propo- rent statement of what the project sal's goals and objectives and the intended todo,or whyit intended proposed workplan do not mention the to do it. An intent toimplement. recreation centers,the letters of PLATO,a proprietary computer-based support suggest that the Youth Cen- instruction system, and to evaluate ters were to be included as part of its effectiveness, was stated in the the,,program. (More importantly, the proposal; the objective of imple- Youth Centers evolved into the pri- menting PLATO is puzzling, however, mary project intervention). because the LCO school had had PLATO instruction available for three Vagueness about what the program years prior to the submission of the was about,and what it should do grant proposal. extended throughout the first proj ect year. Week-long conferences in August1980 and January1981 were insufficient for evaluation staff to determine precisely what the project This report covers the period of intended to implement. Over time, September 1980 to June 1982.

-415- 301 LCO

projectgoals and objectives were The foundation for strategies to developed, but their theoretical meet these objectives is the belief rationale remains sketchy. The fol- that youth on the reservation get in lowing account is constructed from trouble when they have nothing to do Program Development Plans developed but "hang out." Delinquency should over the project's history, and from be reduced if the youth haveoppor- implications of thestatements of tunities available for constructive project personnel. Bear inmind (or least not destructive) activi- that this account is constructed ties: school or supervised recrea- post hoc, and was probably not used tion. Youth in school or at a Youth to guide day to day project decision Center should be protected from making. delinquency.

According to project staff, TheLCO schools (secondary and delinquency onthepart of Indian elementary) were founded in 1976as youth is due to 'having "insufficient an alternative to the Sawyer County opportunities for productive, rele- public schools which,it was felt, vant, and enjoyable activities" (LCO did not provide an education in program development plan, dated 'keeping with the Ojibwa Indian cul- March16, 1982). Thereare "few ture. opportunities for adolescents to engage in productive activity during Early on, PLATO instruction was weekend and evening hours" (same emphasized in an attempt to provide source). individualized instruction in a medium(visual and motor)that was Although alcoholism and drug believed to mesh with Indian youths' abuse were frequently mentionedas behavioral and cultural style. The causes of delinquency by project and project proposed to expand the use LCO personnel, the project only of thePLATO system,and evaluate obliquely attempted to address these its effectiveness. PLATO's theoret- problems (see the discussion of the ical import for delinquency preven- Group Home, below). tion is that increased PLATO availability will result in decreased Goals abseenteism anddrop-outfrom the school, whichwouldproduce less The goals of the project were to delinquency. (The appropriateness reduce delinquency among reservation of computerized instruction is youth, and reduce drop-outs from the called into question because LCO schools. (a) prior to the start of the project the LCO high school already had Objectives eightterminals which were not in constant use, and (b) the principal Objectives of the LCO project of the LCO high school stated that included reducing the suspension absenteeism and drop outfrom the rate among LCO high school students, school were not really a problem.) reducing over the life of the project the number of students referred The project contributed to PLATO to the project from the reservation instruction on thethe reservation school, decreasing in-school absen- by placing four PLATO terminals in teeism among project-referred youth, the Youth Centers,and temporarily increasing the number of youth par- putting fourterminals in theLCO ticipating in Youth Center activi- schools to supplement the eight ties, and increasing the self-esteem theyalreadypossessed. The four of tribal youth.

-416-

302 LCO project terminals in the schools conclude that "...there is no were ultimately withdrawn and placed evidencethatthese (recreational) in the Youth Centers. Because PLATO programs in any way alterdelin- instruction was alreadya signifi- quency" (p. 37). Similarly, the, cant part of the LCO schools' cur- background paper for the Delinquency riculum, and the PLATO terminals in Prevention Through Alternative Edu- the Youth Centers were almost exclu- cation initiative (OJJDP, 1980) does sively used for computer games (see not include recreational programming below), the importance of PLATO as among possibly effective instruction as a part of the project approaches to the prevention of is unclear. juvenile delinquency. More to the point perhaps, there is scantreason The actual primary project inter- why recreational programs ought to vention was the staffing and mainte- affect delinquency. -Delinquency, nance of two recreational Youth Cen- and crimein general,takes little ters. The objective here was to time. Vandalism is usually an provide a place where youth on the impulsive acttaking only seconds. reservation could engage in healthy Shoplifting requires at most several recreation. The explicit rationale minutes. Unless one wishes recrea- was thatthe reservation was iso- tion personnel to supervise youth lated and that youth on the reserva- every minute of every day (and call- tion needed structured recreation to ing such a program recreational protectthem from delinquency (see would be strange), there is no rea- Carlton,1982,for a more detailed son to believe that youth will not analysis of the social and geograph- have time_available_to commit crime ical environment of the LCOreserva- if they so wish--no matter how much tion). basketball or pool they might play.

Is This Sensible? Obstacles to Program Development

Delinquency is a tough problem. The LCO project suffered from a It has not proven amenable to quick succession of difficulties. From fixes or to ad-hoc programming. funding to staffing to politics, the Evaluations of delinquencypreven- project confronted a continuous tion projects have consistently series of obstacles in the path of noted that projects have not been program development. The following powerful enough to solve the problem list of events, although not exhaus- that they were intended to address tive of the trialsthat theLCO (Wright and Dixon, 1977, Lundman & project went through, illustrates Scarpitti, 1978). Theprogram as the project's struggles. designed and implemented by LCO does not appear to us to bea credible Project Unfocused approach to the prevention of delinquency. The primary theoretical The project's fundamental imple- principle behind the project is the mentation problem was that its goals maxim, "Idle hands are the devil's and objectives, and the strategies workshop." Acting on such a theo- to achieve them, were diffuse. As reticalpremise, oneintervenes by noted above, the original grant pro- providing work or play for the idle posal did not clearly outline a pro- hands: Youth Centers and computer gram. Neither the original project terminals. Yet, in anexhaustive director, nor the director that review of delinquency prevention shortly took over were involved in efforts, Dixonand Wright (1974) the proposal writing or project

-417- 303 LCO

design. They were notsure what encompass the myriad ofactivities they were supposed to be doing. that the project wasinvolved in. Youth Outreach became a major focus After meeting with projectper- of program development efforts. sonnel in August 1980, and January Unfortunately, going from paperto 1981, it was apparent to the evalua- program takes time--interventions tion staff that the project hadno must be specified,agreements must clearly specified aims. The project be made, and staff must be hired and hoped to help youth on thereserva- trained. For the LCOproject, it tion stay out of trouble--anything was a matter of too little too late. that seemed to address that goalwas As ofApril, 1982, when refunding fair game for project attention. decisions for the third project year However, after these conferences,we were being made, Lca was still hir- had no clear picture ofthe proj- ing new project staff-'-more thana ect's implemented or intended inter- year and a half afterthe initial ventions. grant award.

Because the grant application The unfocused nature of the ori- t.e stressedthe importance of PLATO, ginal project proposal was exacer- and contained the stated objective bated by the governing tribal coun- of evaluating itsusefulness with cil's view of the project. They may Indian children, a decisionwas made have seen itas another aspectof to evaluate PLATO instruction in the _general reservation activitiesand high school. Although PLATO had perceived little uniqueness to the beenanintegral part of the LCO delinquency prevention project as a curriculum for some time before the project. Thus funds were spent on a projectbegan, it wasdecidedto range of thingssuch as repairing administeran abbreviated form of the school van, providing transpor- the SAES national questionnaire to tation for school and Group Home all students at the LCO high school events, staffing the Youth Centers, once at the beginning of the school hiring alcoholism counselors, hiring year and again atthe end ofeach outreach workers, taking youthon academic quarter. This plan would field andactivity trips, renting have allowed the project managers to PLATO connect time, buying PLATO monitor the usage of the PLATO ter- terminals, and developing PLATO minalsover time, and wouldhave Ojibwa languagesoftware. While a made possible a relatively weak qua- subset of these activities might si-experiment of the effect of have been reasonable program PLATO. This quasi-experimental efforts, the shotgun approach design was not realized because of assured that no single intervention poor response on the early pretest. received enough attention to have a Early evaluation plans also called well defined impact. for monitoring the use of the youth centers in order to learn about what Loss of Other Federal Funding kinds of youths used the centers at which times. A serious obstacle to the project was the large cuts that the reserva- At an August, 1982 workshop, tion was receiving in other federal project staff, along with evaluation funding concurrent with the start of and technical assistance staff, the delinquency prevention project. designed a new intervention: Youth Instead of being ableto focus on Outreach (see below for a descrip- additional services to youth on the tion). An outreach effort seemed to reservation, the project fell back

-418-

304 LCO

to maintaining some of the services Cuts in educational and social no longer funded under other grants. programs appear to have elicited a backlash on the part of the LCO As the project began CETA funds reservation population toward whites ran out. CETAworkers had been that worked on the reservation. A staffing thetwo reservation Youth tribal election in thewinter of Centers. Concurrent with these 1982 changed the composition of the cuts, the Youth Employment Training tribal council. The new members had program, and the Youth Community campaigned on an "LCO jobs for LCO Conservation Improvement program Indians" platform. In April of were discontinued. These federal 1982, 11 of the 17 white teachers in projects had also provided some the LCOschoolsystem were fired, funding to maintain the Youth Cen- (with no replacements identified for ters and their activities. At the the following year) along with the statelevel, the Wisconsin Gover- principals of the elementary and nor's Council on JuvenileDelin- high schools, the school superinten- quency Prevention;which, hadpio- dent, and the director of education vided the funding to build the Youth (who was a Native American, butwas Centers in the firstplace, also perceived as having let whites withdrew support funding. As a become too involved in the education result of these fundingcuts, the of reservation youth). The alterna- Youth Centers were closed on eve- tive education project director, who nings and weekends--the timesthat was white,was convinced that the the project felt were most crucial onlyreason he was not fired was for delinquency prevention. There- that the tribal _council knew that fore, the project, instead of his dismissal would be the last extending the services of the Youth straw as far as possible third year Centers as they had hoped, fell back funding for the projectwas con- to simply keeping them open. cerned. The project director believedthat if the project had Toward the end ofthe project, been refunded for the third year, he several other youth and education would havebeenreplaced shortly programswere being discontinued. after the grant was signed. As cuts in federal programs continued throughout the life of the proj- Although the project director did ect, projectstaff became demoral- not give up on the project during ized. They found themselves these events, it is fair to say that subjected to requests to fund a he was less invested in project variety of youth related events, efforts past April 1982. The poli- whether they were linked to project tical shake-up particularly hurt the objectives or not. evaluation of the project, since the reservation firings were concurrent The situation deteriorated toward with attempts to survey users of the the end of the project. Along with Youth Centers, and survey Indian the loss of the delinquency preven- youth who attended the local public tion project,the reservation lost schools. Lacking the concerted funding for a fine arts project in attention ofthe project director, the school, a youth athletic intra- neither of these surveys met with murals project, and an adult educa- much success (see below). tion project.

-419- 305 LCO

Succession of Project Directors did not appear to OJJDP or the evaluation staff to be directly As lateas December 1980, four related to theinterventions that months into the project, no the project was trying to implement. accounting procedures had been instituted andthepaper work for An example of this situation was the release offunds had not been the development ofthe Ojibwa lan- completed. Theproject couldnot guage program for use on the PLATO pay itsbills, and wasthreatened system. The argument advanced for with being dropped from the initia- underwriting the development of this tive. The project director's duties computer software was that learning were assumed by the evaluation coor- Ojibwa would put the reservation dinator in December, 1980, after he child more in touch with his or her had been associated with the project heritage, resultin mbre self-es- for onlyone month. The initial teem, .and lead to less delinquency. project director resigned in January This causal chain is plausible if a of 1981, and the evaluation coordi- project is making a full-scale nator became the acting project ,effort to improve the children's director. The changein status of self-esteem. Butthat was not the the evaluation coordinator produced project'sintention. The point is three management difficulties: that instead of concentrating on one (a) there was now no one with the task or another, the project delved primary duty of overseeing the eval- into a variety of things whose only uation, (b) having just been hired, interconnection was that they had the new director knew even less something to do with Indian youth. aboutthe project's goals than the original director, and(c) the new Not Expecting Rigorous Evaluation director was white, which may have limited his ability to negotiate A further organizational problem with the tribal governing board. was thatLCOgrantadministrators were familiar with a certain kind of Project Not Viewed as Distinct From federal evaluation. The LCO tribal Regular School organization was accustomed to federal evaluators making site visits, Largely due to the unfocused and judging projects on the basis of nature of the project, there was no interviews and audits. The evalua- clear distinction made between the tion of thealternativeeducation alternative educationproject and project was perplexing. After the LCO schools. It may be that the decades of federally fundedproj- original intention of the grant ects, the project director commented writerswas that the LCO schools that the evaluation of the alterna- themselves were the "alternative" tive educational project was the education part of the project. This most rigorous that LCO had ever been is certainlyplausible given that subjected to. the LCO schools were indeed established as a culturally-oriented The consequence of the reserva- alternative tothe public schools. tion's past history with evaluation Yet, the project asdeveloped and was that evalution tasks were "just understood by evaluation and OJJDP more paperwork." When problems staff wasan entity distinct from developed, or research designs broke the school. The confusion (whatever down, theresponsewas to assume the source) resulted in the project that it didn't make any difference having to justify expenditures that anyway. One result: Evaluations of

-420- 306 LCO the Youth Centersandof the LCO in the evaluation staff assigned to high school school fell apart due to the project. nonattention to data collection details. The result of all these obstacles wasthat at theclose of the LCO Change in Evaluation Staff project it was still in an early stage of development. Staff was A final obstacle to program suc- still being hired in March 1982, and cess was thechangein evaluation all project components were not staff that the project had to implemented until May of that year. endure. Project personnel were dis- By this time the decision had been satisfied with their original field made not to renew the project's worker. They were puzzled when he grant award. showed up for a site visit without a clear set of objectives for the Interventions visit. This event reinforced the project's belief that evaluations Youth Center were generallymeaningless. This attitude made evaluation efforts Thefundamental intervention of subsequent to the replacement of the projectwas the staffing and that field workerin thefall of maintenance of the reservation Youth 1981 more difficult. Then, the Centers. There are two Youth Cen- replacement field worker resigned in ters on the LCO reservation, located January, 1982. In the two years of about 15 miles apart in the two most their project, therefore, theyhad populous communities on the reserva- three evaluation field workers. tion. The project initially hoped This lack of continuity, coupled to expandthe services offeredat with the project's previous experi- the Youth Centers, but following ence with federal evaluations, cutsin other sources offunds to served to undercut attempts at for- the Centers, had to be content with mal, data-based evaluation. installing PLATO and paying the salaries of the Youth Center staff. Summary of Obstacles to Implementa- tion TheYouth Centers aredrop in facilities thathave grills, pool The major obstacles to the imple- tables, video games, and meeting mentation of the LCO Alternative rooms. Space can be cleared to hold Education Project were (a) the proj- dancesor otherlarge gatherings.. ect had unfocused goals and objec- The project hired persons toopen tives and expended funds on a grab- and close the center, maintain the bag of interventions, (b) other physical plant,and serve as cooks federally funded projects were being (see descriptions of the Support discontinued, and the delinquency Specialists and Youth Outreach Work- prevention project picked up some of ers, below). Project personnel also their expenses, (c) project adminis- supervised the use of PLATO at the tration was allowed to slip in the Youth Centers. early months of the project, (d) the project was not viewed as an entity The first direct project inter- distinct from the LCO schools, vention at the Youth Centers (other (e) project personnel did not expect than keeping the doors open) was the arigorous evaluation,and had no installation of on-line PLATO termi- experience in conductingone, and nals. This intervention was (f) there was a lack of continuity intended to provide another source

-421- 307 LCO

of recreation at the Youth Centers not available for the self-contained (on-line PLATO terminals allowone The only software was to access a variety of computer designed for the teaching ofbasic games), and tomake PLATO educa- academic skills--items of little tional instruction available for interest to the Youth Center clien- youth notinvolvedinthe regular tele. Project personnel estimated LCO school program. that use of the terminals dropped 80% when the self-containedcompu- Installation of the terminals got ters were installed. off to a slow start. The PLATO terminals had to be connected to a Project-sponsored Youth Commit- "parent" computer in Minneapolis, tees were developed in the summer of and the phone linesnecessary to 1981. These committees were organ- carry the information are different ized by project personhel andwere from the regular long-distance open to all youth on the reserva- lines. New lines wereinstalled, tion. They operated out of the but this took fbur months. The Youth Centers. Participation in the lines were connected directly to the meetings themselves was light,but Reserve YouthCenter, andregular the committees--or the project per- long-distance lines carried the sig- sonnel running them--did design and nal to the New Post Youth Center. carry out a number of recreational activities for youth. From January The necessity of sending the sig- 1981 until April 1982, a total of 14 nal by long-distance telephone lines events and trips were sponsored by from New Post to Reserve caused the the project. These included dances, project quickly to remove terminals skating parties, field trips, skiing from New Post. Long-distance rates expeditions, and canoe trips. were too expensive tojustify the terminals' use at New Post. There- Youth Outreach after, until the end of the project, the New Post Youth center was not a In August 1981 a new Youth Out- prime focus ofproject activities. reach intervention was designed that Project funds were used to staff it appeared to encompass a variety of andkeep it open, but PLATO and project interventions. The LCO other project activities were una- project decided to locate those vailable there. youthonthe reservation who were unemployed and not enrolled in Because rental of connecttime school, or whowere enrolled in with theMinneapoliscomputer was school but in danger of being expensive, the project eventually expelledfor discipline or atten- purchased self-contained PLATO dance reasons. The Youth Outreach micro-computers. Four of these were effort was designed to identify such installed in the Reserve Youth Cen- youth,offer counseling and tutor- terin January of 1982. Although ing, and involve these youth incon- they were less expensive to run, the structive activities. Twocatego- self-contained units had one major ries of workers were responsible for drawback: little software was thiseffort: SupportSpecialists, available. and Youth Outreach Workers.

The primary use of the PLATO The first Support Specialist units in the Youth Center was game- began work in November of 1980, and playing. They were seldom used for a second in April of 1981. They instruction. But game packages were were originally responsible for

-422- 308 LCO

staffing and supervisingthe youth Outreach Workers were also centers, and the PLATO terminals in responsible for conducting a census them, and for providing drugand of Youth Center users, administering alcohol abusecounseling to Youth an evaluation survey to Youth Center Center clients. With the advent of clients, and carryingout a field the Youth Outreach effort, they survey of reservation youth who acquired the additional responsibil- attended the SawyerCounty public ities for making home visits to Out- schools. reach youth, and of supervising the Youth Outreach Workers. By January 1982, three Youth Out- reach Workers had been hired, and It is difficult to say how quali- four more were hired in March 1982. fied the Support Specialistswere to carry out their duties. Their sta- Alternative Education tus as tribal members was certainly an asset; it would be difficult for Prior to the start of the alter- outsiders to understand the economic nativeeducation project, students and social problems that thereser- .at the LCO high school who exhibited vation youth faced. Ontheother chronic absenteeismwere expelled hand, although drug and alcohol from the school. The project estab- abuse counseling was one of the Sup- lished a policy whereby students who port Specialists' primary responsi- attained sevenunexcused absenses bilities (indeed they viewed itas were suspended from the school, and their most important repsonsibil- required to attend PLATO tutoring at ity), neither Specialist had any the Youth Centers. If thesestu- training in counseling. Although dents performed satisfactorily they academic training neither guarantees could be reinstated inthe regular nor prevents one from attaining the LCO curriculum at the beginning of skills necessary to carry out effec- the next academic quarter. This tive substance abuse counseling, a referraland PLATO, tutoring compo- lack of trainingin counseling is nent was called the "Alternative not indicative of a strong counsel- Education Program." ing component. Other Interventions The Youth Outreach Worker position was created to establish a Project funds contributed toward liaison between the project and res- the PLATO instruction offered at the ervation youth. The Youth Outreach LCO high school and elementary Workers were to be the people in the school. Project funds paid for part field identifying reservation youth of the rental of the PLATO terminals in need of service, and organizing inthe LCO high schcol. In addi- and supervising recreational activi- tion,in January 1982,the project ties. The Youth Outreach Workers purchased eight PLATO self-contained also staffed the Youth Centers, and microcomputers. Twoof these were were responsible for unspecified placed in the LCO high school, and "peer counseling." two in the LCO elementary school. In April of 1982, all of these The Youth Outreach Workers were self-contained PLATO computers were all high schoolgraduatesor were installed at the Reserve Youth Cen- currently attending the LCOhigh ter. school. Youth Outreach Workers ran the YouthCommittee meetings, and Theprojectalsosupportedthe oversaw their activities. The Youth development of an Ojibwa language

-423- 309 LCO

program for use on the PLATO system. nity for adolescents suffering from The package describes in Ojibwasev- drug or alcohol dependency. The eral traditional Ojibwa events (e.g. project provided transportation for "The Sugar Camp"), while on the Group Home residents to Youth Center screen the scene is drawn and activities, and offered PLATO tutor- labeled in Ojibwa. Headphones allow ing (althoughwe do not know if the student to listen to the speech. tutoring was utilized). The Support Studentsrespond byrepeating the Specialists provided some counseling sentence they have heard (but with to these youth. no feedback as to correctness), and engagein multiple-choice tests of The project also provided trans- matching written Ojibwa to drawings portation for LCO school children to of objects named in the previous project activities. lesson. Afundamental difficulty with the program is that Ojibwa A HumanServicescommittee was grammar is substantially different established. Chaired by the Project from that of English--thelanguage Director, the committeewas charged of all children on the reservation. with the coordination of socialser- For example, in Ojibwa, the verb is vice delivery agencieson the reser- alwaysthelast word inthe sen- vation. The project director felt tence. Unfortunately, the program that his time as chair of the com- doesnot addressgrammar at all. mittee was a project contribution. Because they are not instructed to the contrary, children make the rea- Evaluation Results sonable assumption that the the _ first word spoken in Ojibwacorres- At the outset,it should be made ponds toto the first, word of the clear that we have little data by English translations.' It doesn't. which to evaluate the LCO project. Observationsindicatedthat inter- The project was still hiring staff acting with the program degenerated as late as March 1982, and had not to a guessing game of randomly yetfullyimplemented one of its assigning Ojibwa labels to drawn most important interventions, Youth objects in the hope of matching them Outreach. Evaluation issues took on correctly. Any learning that took secondarystatus to program imple- place appeared to be rote memoriza- mentation. In addition, for reasons tion of chancecorrectmatchings. outlined above, at the close of the Project staff admitted that children project when summative evaluation were a little "confused" by the lan- data were being collected,project guage lesson. personnel were not invested in the project itself, much less in its A second Ojibwa lesson was sup- evaluation. The project director posed to bein development at the was cooperative with evaluation close of the project. We do not efforts, butthe data provided are know whether it wascompleted or insufficient to make strong conclu- not. sions. Therefore, the information that followsis mostly descriptive Project staff provided some ser- in nature. Itis offered more to vices to the reservation Group Home aid other program implementers and and to the LCO school. TheLCO evaluators than to evaluate the LCO Group Home isa residential commu- project.

-424- 310

11. LCO

Youth Center differences between the youth using thetwo Youth Centers, otherthan In March of 1982 the first task that the New Post Youth Center might assigned to the Youth Outreach Work be serving slightly older youth. ers was to undertakea census of With the poor response rate,it is Youth Center users. For four weeks, impossible to say with any cer every user of the two Youth Centers tainty. had to sign in. From this signin sheet a roster of Youth Center Table 2 suggests that youth visit clients was developed. The Reserve the ReserveYouth Centersomewhat Youth Center (the foremost site of more often than the New Post Youth projectevents)had 125 different Center (but the differenceis not users, and the New Post Youth Center significant). Satisfaction with the had 81. Because the New Post Youth youth centers does not differ much. Centerhad little to do with the project, other than being staffed Table 3 gives the stated reasons through project funds,it serves as for visiting the youth centers. Of a kind of nonequivalent control for ,particular interest here is the low Reserve, the project Youth Center. percentage of youth who go the Youth Centers to use PLATO (by the time of In consultation with the national thesurvey in thesummer of 1982 evaluation staff,, a survey was PLATO had been installed at New designed to identify the personal Post). Youth mostly attend the characteristics of the Youth Center Youth Centers to do what one might users, and to assess their satisfac expect: play games, visit friends, tion with the Youth Centers. The and get something to eat. The two Youth Centers were based about results donotsupportthe notion 15 miles apart. Since transporta that making computerized basic tion around thereservation is a skills instruction available at substantial problem for reservation recreational sites will affect many youth, it might be assumed that youth. attendance atone Youth Center or the other woulddepend mainlyon Alsointerestingin Table 3 is geography. Differences in satisfac that almost no youths report tion with or use of the Youth Cen involvement in Youth Council Meet ters might be attributable to proj ings. ect effects. One possibly positive effectof Table 1 gives some descriptive the project's attention to the information onthe clients of the Reserve Youth Center can be found in twoYouthCenters. The N's (the Table 4. Table 4 gives the place of number of people responding to the residence of each survey respondent. survey) tells themost important Whereas all (N = 17) of the New Post story. Of the 125 identified Youth Centerrepondents were from clients of the Reserve Youth Center, New Post, Reserve served youth from 78 weresurveyed, for a response all 10 of the major reservation com rate of 63%. Only 17 of 81 New Post munities. It may be that the proj youth completed their surveys, for a ectandits Outreach efforts suc response rate of 21%. Not only is ceededininvolving more youth in an N of 17 too small for results to Youth Center activities. Alterna be dependable,but a 21% response tively, the project may have chosen rate suggests that selection biases to operateprimarily in the more mayexist. Nevertheless, Table 1 popular center. does not suggest any major 425 311 LCO

Finally,Table 5 gives descrip- 1. As just discussed, the tive information aboutthe employ- greater geographical distribution of mentand schoolingstatus of the the Reserve Center's users may be Youth Center users. No differences due to Outreach efforts, since Sup- aresignificant,although somewhat port Specialists and those Youth more of the youth at New Post may be Outreach Workers that had been hired attending the LCO high school. were based there. The evidence for

this in only circumstantial. , We can make noinferencescon- cerningthe delinquencyprevention 2. The Youth Center survey efficacy of the youth centers. administration was the single most Project personnel never provided important task of the Youth Outreach court contact data on the Youth Cen- Workers, and it .was performed ter users as agreed. poorly. This suggests that high school students require careful supervision to carry out such a What can we conclude about the field study. Similar supervision Youth Centers?Reserve, the primary would probably be required for the project site, seems to serve youth other duties,including peer coun- from a wider geographical area than seling and family visits, with which theNewPost YouthCenter, which they were charged. received less attention from the project. This may or may not be a 3. Support Specialists may projecteffect. According to the require more preparation in drug project director,the Reserve Youth dependency counseling. Because they Center was more active than New Post werecharged withsupervising the prior to the start ofthe project. Youth Outreach Workers in the Youth That was why Reserve was chosen as Center survey administration, which the focus of the project. did not go well, theymay have required more preparation in project To be fair, more important than management or supervision as well. anydifferencesbetweenthe youth centers is their fundamental simi- Alternative Education larity: they were open for business. They would have been closed We were not provided with infor- or opened on a much curtailed sched- mation needed to identify Youth ule withoutgrant funding. Thus, referred to "Alternative Education" their availabilityto youth was a during the 1981-82 school year. project success. For theoretical Both the principal of the high reasons, however, their usefulness school and the project director in preventing delinquencyis ques- indicated that the program did not tionable. work: Youth suspended from high school for nonattendance did not Youth Outreach come to the Youth Center for basic skillstutoring, or counseling in We have no direct way of evaluat- drug dependency. ing the effectiveness ofthe Youth Outreach component, including the For descriptive purposes, Table 6 Youth Outreach Workers, and the Sup- gives characteristics of the refer- port Specialists. Yet,three com- rals to Alternative Education during ments are in order: the spring of 1981.

-426- 312 LCO

OfiltaAoglooksmo Wool and esperteaces of easpiesika. het pohlie school sta- MOM **it te are mewled gag often and tht401144 xea forittI 'waive twee muds is school than to "mown Ise we do psetbe *Medias the LCO schools. o*Itoolii t s4.ce cles saw to pros Thew **telt* fie, hear directly 4400 4011,14x10101- 0444041r twti*itio, to es the'some it whetherthe LCO 140 oloSPote. ea Amp"to* sod* to wheel' teems* or reduce the risk 046014,* i 04000 ees to *bow fever. ef **wow behavior, Differences 4ms *vote oto solfaiod the ere* ebeervedsay be setirelydue to OPOO*0 whoa .) tipot * ifeeipktiledoe eelf..seleettee, 10400411 etvellsosi 00044s iatorsettea ira thop0000stoostose WON COW tf 114 were Aeseelmksfail4 f3/44,46.444441104 talissa. osaariopreyed, the LCOhigh 4 *.I *sheet to raedomly assiga students to PLATO am Ammo strong claims tte 4 esevey se* etesialletfefd love hoes mode ter the of of ot4e001 ko eta* deltiodiag the #PLATO as is isetrectiesal supdtuu,we Oh 440 40hool* 4 #104444 et "poor= eovertheless decided to. (lupine eiEl*Pe omit 4040064001 the Mow 11411) see es best we could.' Table tomo4o 04434 *shoot* owe 64101001. proefs's staple eerrelatioos between *00 toottoeilb 001/4011 ertoeo aosigeed hours et PLATO use is l'112 aod mess- the look et hativtastily feamister. uroe of academic achievement, bmg *Of *Aft ave*tioemales f these attsSatest to school, psychological efoOpoes oe pet* et thole thatoeth health, mad delisqueucy* Wool. The only stgatlisont correlates $ oho la otbutol PO et $ sits of PLATO use are sbossoss and disci- oaf.* attia4044 Use 016,0*, tee am eliaory referrals. Tout% who bltip 04,0**4* foots** co* of 4)16 Only school sore, and who get in trouble fa it it Ube pahlte *fleet mare* teed to use PLATO were. This mod; speo eerveye d. do ome the of seerse is purely correlation fees tee Ohs tooth Goatee foreey. data* sod prohablyindicates only 4000444011601*. tee OdibStateepi initial differences is *sere of to toe to* #04410110 tot sestideet PLATO. More ie so Wailes toaca- imeolvretoefies it the dots. demie achievementas measured by either grease it self-reported toble t goofs f*Ileito et *shod effortit is met ealy the amsleito at Notloose soopa tos the 00644a *ahead*toilet* to the U -*ago.* oo atoo 44411 sesaass46 The Tao re, mad the Attereettal re- *pew* Moe ease ilt44 ittiele Melte I*'d'e were usable to do multiple 4044 he Sneffteelhi1 eStib gnat regressionamalysisoe PLATOuse ostestoso. controlling for pretreatment differ- oases between individuals because of *woe the Woods Se botorpre- law salbers et 'taints surveyed in AMNON thepsu et molt* mg- sprigs ITU. Pretests administered emote het the atedeets ettostimg in the fell et OS: were also Londe- Oto pohlte wheel ere lee* embelli- lust* became. (a) they were not eme mewl miehee MS 1011 delememeat admisstered Datil November. making 60bosbotwObeekSebOt ptters* 'etas their 'pretest" status opestiosaa aoseetaiod ottb lop aft aabemee I. Ids, sad 00 there is a great deal of sissies data.

313 LCO

less ablestudents who areusing Recommendations PLATO. Neither is there any relationship between PLATOuse and any 1. Proiects should not be funded measure of psychological adjustment. that do not have a clearly specified Wo find no evidence here that PLATO theoretical base supporting a use increased the self-esteem of low rational, planned set ofinterven- skilled learners. tions. Especially given theshort time period (two years) by the end Nothing in these results suggests of which projects are required to PLATO instruction hada measurable havedemonstrated progress toward effect on anything. goals and objectives, programs that are not able to hit the ground run- d the, Program Meet Itoals and ning are faced with animpossible Objectivts? task.

Probably not. We cannot assess 2. Projects should not be the project's impact on delinquency. allowed to expend funds on "scatter- Self-report delinquency was assessed ,shot" interventions. The LCO proj- in the LCO high school, but there- ect spent time, money, and energy on sponse rate for students in the pub- a shopping basket of youth interven- lic school was extremely low,and tions. It would have better concen- besides, the Youth Centers were the trated its efforts on a emaller sub- primeprojectintervention. Since set of interconnected interventions. the court contact data wasnever The Youth Outreach program wasan delivered, we are blind to the attempt to do but there effects of the Youth Centers on remained insufficient time forits delinquency. There is no reason to implementation. believe they had an effect on delinquency. 3. Personnel implementing an intervention should have the requi- We cannot say whether the project site training and experience. Only affected dropout from the LCO when counseling is donewell is school. Sincethe intervention in there much hope of positive effects. the school was weak (paying for com- Indeed, theevidencesuggeststhat puters that were already there) we counseling can be of no help in pre- do not know why dropout should have venting delinquency (Wright and been affected. We do know that the Dixon, 1977), and may have negative "alternative education" interven- effects (McCord, 1978;Cook,1983) tion--referral tothe Youth Center that should be carefullyguarded for tutoring after suspension--was against. Under thecircumstances, not successful. No one came. state-of-the art counseling interventions implemented by well-trained Similarly, we cannot evaluate the professionals are perhaps worthy of project's effects on suspensions and test,but the,effortsof amateurs absenteeism from the LCO school, or are unlikely to be fruitful. ontheself-esteem of LCOyouth. Some weakevidence suggests that 4. The requirement of rigorous youths attending the LCO school are evaluation should be made clear suspended lessthan they might be Prior to funding, and a clearly spe- were they attending the public cified experimental or strong quasi- school. The project certainly did experimental evaluation design provide recreational opportunities should be a requirement for funding. for youth. Federally funded crime prevention

-428- 314 LCO projects should cometo anticipate trol groups and the collection of careful evaluationsinvolving con- adequate outcome data.

-429- 315 , LCO

References

Carlton, R. The Lac Courte Oreilles Alternative Education Project: Interim Report. In G. D. Gottfredson (ed.), The School Action EffectivenessStudy: First interim report (Report No. 325). Baltimore: The Johns Hopkins Univ- ersity, Center for Social Organization of Schools, 1982.

Cook, M. S. Plymouth Alternative Education Program: Second interim report. Baltimore: The Johns Hopkins University, Center for Social Organization of Schools, 1983.

Dixon, M. C., & Wright, W. E. Juvenile delinquency prevention.programs: An evaluation of policy related researchon the effectiveness of prevention programs. Nashville, Tenn.: Office of Educational Services, Box 60, Pea- body College for Teachers, 1974.

Lundman, R. J., & Scarpitti, F. DelinqueTcy prevention: Recommendations for future projects. Crime and Delinquency, 1978, 24, 207-220.

McCord, J. A thirty-year follow-up of treatment effects. American Psycholo- gist, 1978, 33, 284-289.

Office of Juvenile Justice and Delinquency Prevention. Program announcement: Prevention of delinquency through alternative education. Washington, D.C.: Author, 1980.

-430- 316 Table 1

Descriptive Statistics on Youth Center Patrons, Spring 1982

Reserve: New Post: Project Youth Center Comparison Youth Center Characteristics SD N Ti SD

Age in years 16.17 5.77 76 18.50 5.61 17

Male 63% -- 78 63% -- 17

Holding high school diploma 31% -- 78 25% -- 17

Table 2

Frequency of Attendance and Satisfaction with Youth Centers, Spring 1982

Reserve: New Post: Project Youth Center Comparison Youth Center Patron's report of M SD N M SD

Weekly visits to youth center 3.93 1.75 75 3.06 1.71 17

Satisfaction with youth center 2.81a .68 79 2.94 .43 17

a On a fourpoint scale ranging from "Very Satisfied" (4) to "Very Dissatisfied" (1).

431 317 Table 3

Reasons for Visiting Youth Center, Spring 1982 (Column Percent)

Reserve: Project New Post: Comparison Youth Center(n=79) Youth Center(n=17) Reason % Affirmative % Affirmative

Games (Pinball, video, pool) 90 94

Food 68. 47*

Social contacts 66 .41 71

Use PLATO 8 12

Youth Council meetings 1

Other 9

* P< 10

-432- 318 Table 4

Residence of Youth Center Patrons, Spring 1982 (Column Percent)

Community of Reserve: Project New Post: Comparison Residence Youth Center (n=78) Youth Center (n=17)

Bacon Square 4 0

Chief Lake 11 0

Dry Town 1 0

KTown 6 0

New Post 4 100

Reserve 48 0

Round Lake 6 0

SignorCouderay 9 0

Six Mile 4 0

White Fish 4 0

Other 4 0

-433-319 Table 5

School and Work Status of Youth Center Patrons Responding to, "What are you Doing Mostly?", Spring 1982 (Column Percent)

School and Work Reserve: Project New Post: Comparison Status Youth Center(n=78) Youth Center(n=17)

Attending LCO high school 23 41

Attending LCO elementiry school 4 6

1.0 Attending Hayward public high school 4 6

Attending Hayward public elementary school 1

Attending vocational school 1 0

Attending G.E.D. 1 0

Attending college 1 0

Working full or part time 24 24

Doing tribal volunteer work 1 0

Unemployed 15 18

Other 23 6

-434- 320 Table 6

Descriptive Statistics on Referrals to "Alternative Education", Spring 1981

Student Characteristics M SD N

Grade point average, tig Spring 1981 2.00 0.00 5

Hours PLATO use, Spring 1981 7.05 5.17 11

Absences, Spring 1981 9.63 3.07 8

Disciplinary referrals, Spring 1981 1.09 .70 11

Note: Six of the 11 referrals withdrew during the Spring of 1981. Three of these subsequently re-enrolled the following year, and one more dropped. At the end of 1982, 7 of the 11 1981 referrals were still enrolled in school. None had graduated.

-435-321 Table 7

Comparison of Self-report Measures of Indian Youth Attending the LCO High School and the Sawyer County (Hayward) Public Schools, Spring 1982

LCO Public School Questionnaire Measures M SD N N SD N

School variables

Attachment to school .73 ..22 77 .70 .30 20 School rewards.. .43 .33 78 .10 .22 22*** School punishments .10 .17 78 .07 .19 22 Parental emphasis on education ,46 .30 76 .30 .21 22*** School effort .55 .29 77 .51 .28 24 Negative peer influence .29 .19 79 .27 .22 25 Victimization .10 .15 77 .06 .10 21 Involvement .20 .16 77 .15 .14 23 Reading ability 1.92 1.59 79 1.85 1.64 26

Psycho-social variables

Attachment to parents .42 .24 79 .55 .26 25*** Belief in rules .61 .28 76 .71 .28 22 Positive self-concept .69 .17 72 .70 .17 21 Rebellious autonomy .86 .23 75 .71 .36 22** Interpersonal competency .77 .25 73 .76 .27 22 Internal control .44 .24 75 .42 .28 22 Alienation .27 .24 76 .24 .26 21 Practical knowledge 1.34 .40 76 1.22 .61 22 Invalidity .14 .18 77 .05 .12 22**

Self-reported behavioral variables

School non-attendance 1.29 .66 78 1.00 .82 25 Delinquency--total .28 .19 52 .14 .15 25 *** Serious delinquency .14 .17 52 .07 .14 25* Suspensions .62 1.98 79 1.85 3.57 26** Drug use .60 .32 51 .28 .31 25**

*p<.10 * *p<.05 ***p.01

322 -436- Table 8

Correlations between 1982 Hours of Plato Use and Measures of Academic Achievement, School Attachment, Psychological Adjustment, and Delinquent Behavior

Measures r N

Academic achievement 4 Spring 1982 GPAa .06 73 School effort .09 77 Absencesa .29* 89 Withdrawals from schoola .16 95

Attachment to school

Attachment to school .09 77 School rewards .12 78 School punishments .03 78

Psychological adjustment

Positive selfconcept .09 72 Alienation .10 76 Internal control .07 75 Interpersonal competency .18 73

Delinquent Behavior

1982 Discipline referralsa .28* 95 Selfreported delinquency--total .04 52 Selfreported serious delinquency' .12 52 Selfreported drug use .05 51

a Schoolbased data; other measures are SAES ques tionairebased.

* I) < 05

437-323 324 -438- Plymouth Alternative Education Project: Second Interim Report

M. S. Cook

Abstract

The Plymouth Alternative Education Project operated StudentSer- vice Centers, StudentActivitiesCenters, and an out-of-school Learning Options program primarily fortwo high schools and two middle schools. These interventionsprovide educational services, counseling, and recreation for students with disciplinaryand attendance difficulties in this predominantly working andmiddle-class white community. Interim evaluation results imply that the high school Student Activities Center is being implemented withcare, and has some promising positive effectson participants (increased aca- demicperformance,decreased alienation,and increasedpractical knowledge). Interim evaluation results raisesome questions about the Student Service Center implementation,and suggest that some unexpected negative effects of this counseling interventionmay be occurring. The Learning Options program, whichappears to be vigorously implemented, did not participate inan outcome evaluation.

poor academic performance, and who are considered at risk for drop-out and delinquency. The project is The Project composed of four discreet treatment programs: ThePlymouth Alternative Educa- tion Project is one of several proj- 1.The high school Student Ser- ects funded by the federal Office of vices Center is a daily semester- Juvenile Justice and Delinquency long class for which students Prevention (OJJDP)aspart of its receive credit toward graduation. Program in Delinquency Prevention Treatment involves four major inter- through Alternative Education. The ventions: (a) facilitated group Plymouth project operates in two peer counseling, using the students' large highschools that share the day-to-day problems as the source of same physical plant, and in two group discussion, coupled with separated middle schools. Allare affective educational training in located ina mostly white working transactional analysis and interper- and middle-class suburb of Detroit, sonal communication skills, Michigan. The project seeks to pro- (b) close weekly monitoring ofthe vide services to students exhibiting students' academic progress in their truancy, disruptive behavior, or other classes,(c) individual counseling outside of the class period, and (d) a high level ofcounselor- initiated parent contact.

Thisreportcoversroughly the 2. The high school Student Activ- period September 1981 through August ities Center has three distinct 1982. For an account of the project interventions: (a) a competency- during its first yearsee Carlton based remedial writing and study (1982).

-439- Plymouth

skillsclass forstudents showing ect staff also seek to influence the poor academic performance, (b) a general school climate ofthe four supervised setting for student ini- project schools by serving as infor- tiated activities such as clubs and mational sources for alternative discussion groups, and (c) an organ- discipline procedures,and by sit- ized intramural program targeted at ting on appropriate building and students who are uninvolved in con- district-level committees concerned ventional school extra-curricular with discipline, attendance, and activities. school rules.

3. The middle school Student Services Center is similar tothe Program Changes During 1981-82 high school program of the same name. There are three major differ- The only major chanO in the.high ences between the two interventions, school SSC between the 1981 and 1982 all of which are due to the matura- school years was the decrease in the tional differences'of the respective amount of time thatthe class was client populations: (a) students ,team taught. In previous years, the mee,. weekly on an individual basis project co-directors had often team with their counselor for ten weeks taught the class, particularly when prior to the formation ofgroups; students were having difficulty, or this aids in developing bonding to some special problem came up. the counselor so that the counselor Apparently due to time spent engaged has more leverage over negative peer inevaluation activities, the SSC influence effects that could appear counselor was_forced_to work alone inthis age group;(b) the middle in the SSC class more than antici- school program isteam taught;two pated. counselors facilitate the group discussions so that they havemore In the SAC two changes were made direct control over the group pro- in the programduring the year. cess; and(c) the programis more First, a pre- and post-testing structured, relying less on the stu- procedure for the remedial writing dents themselves to bring up prob- class was revised to makeit more lems asa source of group discus- sensitive. It waschanged from a sion. five-part to a seven-part examination. The SAC instructor devoted a 4. Learning Options(LO) is a great deal of time to developing an completely self-contained alterna- adequate diagnostic and evaluative tive school forhighlydisruptive instrument- -onethat paid handsome Students from both the high and mid- dividends(see Evaluation Results, dle schools (thoughmost program below). participants in 1981-82 were middle school students). It is character- A second change in plans for the ized by a high degree of individual- SAC was the emphasis that was placed ized instruction, tutoring, group on theremedial writing classfor and individual counseling, progres- the SAC participants. In fact, sive attainmentof responsibility, "Student Activities Center" was student input into rules and disci- somewhat of a misnomer for the plinary practices, and parental intervention during the 1981-82 involvement in their children's year. The primary intervention of schooling. the SAC was,in fact,the writing class, and student activities tooka Besides the above programs, proj-

-440- Plymouth ilkes4goo to SIP* develegmeet et ibis the class was always team mosimmemem. Dow that Use lately's- temsbt--both Kiddie school SSC 14me 111** two lstemed and easy of instructors participated in each rte Mom* totted imm, ewe time will class meetimg. Iv cosstOWs for stftleet sett Studs. 12111aLliaSiliclidisaLl *be v4A01* Wool. DOC cow a dee- init4c Aimee Om Om mr14044 pre- tligklasstLissitar.litnisaLranur gx*mw OHO' apewleem4 firm ttmt, l4gb loissodi DOC smidel. Omrtog tic ,44- The Sly mouth Alternative Educa- tso4 *toms to apply the Itieb tive project successfully imple- N*04 oedelto the middle embos4 meted is tree experiment in the high dote( Om owing el1DOI,tt toe sebcca Student Services Center Ommed thee middle istibeol otudeets (1K).This experiment was carried temps*. mtgbit, mod sink evaders) set for both semesters of the 446 OW bowie ties perms* maturity 11151-$2 school year. The experimen- 411 ts Ott, (members to deal taldesign for the first semester seemly est* ta> Imterperomsw4 and *broke down sonevbet as a number of totomporosee4 tee** that were the the masts randomly assigned to ombiovi if dhows is is the SOC trosteemt refused the intervention. 411* **P00 leer tise 1,1142 yew. a flisadominatiom was accomplished 01444144re moe developed issibtcb *elms a table of random numbers.)A e4444e 40.04 WIC peritctemete met mere adequate selection procedure, empetweee *40 their rep -tine SSC imvolvial a group meeting to provide oemosotet for 14+ twits prier to the more information about _the program 4044Si it grew ettiettiem. It wee to potential students, resulted in *mod Omit after the otudeste bad fewer refusals by treatment students desedomod e permed Srls6time velem for the second semester experiment. WA* MOP at .eloop mouseelere, Obom semi. be able to VOS104 tbe Results for the year-long experi - poor prom ere to ""are owe durtee moot are presented in Tables 1 and Ow 4044 mod rhos ski gremps mould 2. Table 1 pressing the results for 440 44111 1441$. this &age use em4y analyses of variance comparing the 0404014 sesteeesfel. randomly assigned treatment and controlstudents onthe measures of 40041tems4 shows tree tbe Ugh Plymouth's seals as outlined in **Mod model that mote tosorporsted their PIK plain. Table 2 gives the tato SO* middle *Owl 1K wore tbc result. for (limiter analyses on *04Stbes of a meOtly limey day Is their PD2 plan's objectives. (Sepa- milsimb the memmeelm, owed me es rate saalysea that are limited only 400441M41 Wore mod the use if sere to the second summer treatment and siVtwttred aes4goetsia red saki- control students yield essentially atm* Doris. tbe 11l 10. doe. goateed identical results.) of totriog prtmerily es tbe *tedious mod tbete problem me the se of As footnote Ion Table 1 butt - somtiv4.1 Oer weep disemestee, Op- sates. too few students successfully ts* wee doimootoed beferebood by loopleted the Inpression scale for tie DOC mommelmee, aid promoted to it to be analysed. The Expression 301; groove Also, a greeter emptiest" scale vas a self-report measure of was 'Wed so disci sod astem expressive communication ability goof. at Whe eobool rules dortme the that was developed with project O$C portbd. Is ~lbw satonm, to help. It was added as an addendum rued Che pomp mind its bebevtor. to the regular SAES questionnaire.

32 Plymouth

A moreextensive version of the Interpersonal Competency shows a scale has been developed for negative effect of the treatment. 1982-83, and will be included in the regular version of the SAES student The negative effect on drug use questionnaire. is particularly disturbing, since it suggests that farfromdecreasing The G.P.A. measure reported in the drug use of their clients,the Table 2is only a weak measure of project is increasing SSC students' achievement. This is because it was involvement in drugs. the G.P.A. for the entire academic year. Therefore, any effects of the Turning toTable2, of the 12 secondsemester treatment will be non-redundant measures of the SSC's watered down bythe first semester objectives, there are two dependably pre-treatment G.P.A. Allreported and two marginally significant high school G.P.A. data suffers from effects. Two of these effects favor this defect. the treatment, and two the control. Treatment students are slightly less Table 1 shows that out of the 14 ,,likely to be suspended out of measures of the SSC goals, four are school, and are also somewhat less significantlydifferent fortreat- likely to voluntarily withdraw from ment and control group students. school. These are important, if Only one of those differences favors weak, positive program effects. The the treatment group: Treatment stu- controls, onthe other hand, skip dentsattempted significantly more class less, and report themselves to classes than the control group. be more involved in conventional This,in effect,is an implementa- (essentially school related) activi- tion measure. One of the program ties. No other differences, includ- objectives isto move the students ing four other relevantmeasures toward graduation, and SSC partici- (see the bottom of Table 2) approach pants are required totakea full significance. load of classes,including the SSC itself, for which they receive Several hypotheses were advanced credit toward graduation. Impor- by the project co-directors to tantly, the treatmentstudents do account for these results. They are not pass any more classes than the explained more fully below, but one control students. included the observation that positive benefits of the program may be The SSC results in three negative of a more long-term nature; students effects on goal measures. The SAES may initially have some negative measures of Interpersonal Competency affectas theyare forcedto con- and Parental Emphasis on Education front themselves and their problems both significantly favor the control in the group context, but that the group. The treatment students also self-learning that occurs, along report more drug use than the con- with the support provided by project trol students. staff, should better equip the students to cope with the large, imper- The SAES measures of Alienation sonal school environment. In order and Interpersonal Competency are to examine this hypothesis, a sepa- perhaps the most direct SAES meas- rate series of analyses of variance ures ofthe attitudinal and intra- were run looking just atthe first personal variables that the SSC semester treatment and control stu- seeks to affect. In this data, dents. The treatment students Alienation shows no difference, and should havehadan extra semester

-442- 328 Plymouth for the longer term effects of the The SAES questionnaire includes a treatment to be manifested. Because measure of Negative Peer Influ- of the decreased number ofexperi- ence--the extentto which the res- mentalparticipants, this analysis pondent's peers getin trouble and will be less sensitive than the encourage the respondent to be whole year analysis. anti-conforming. Analyses of variance ofthis measure for the first Table 3 gives the results for all semester treatment and control stu- significant effects and nearly sig- dentsshowsthat there is avery nificant effects, for the same out- weak trend toward treatment students comes listed onTables 1 and 2. (M = .48, N = 17)reporting more Also listed arethe results ofan negative peer influence than control analysis of the Invalidity scale, a students (M = .40, N = 22; p =.19). measure oftruthfulnessin report- For the comparison of all experimen- ing. There are six significant tal students for the whole year, we effects, all of them indicating a find a similar, but even weaker pat- negativeeffect of the treatment. tern of effects. The treatment First semester treatment students as group (M = .46, N = 46) reports very compared to controls reported less slightly more negative peer influ- Interpersonal Competency, lower ence than does the control group (M Self- esteem,more Rebellious Auto- = .41, N = 42; p = .27). This dif- nomy, less Involvement in conven- ference is trivial and non-signifi- tional activities,lower Attachment cant. to School, and less Parental Empha- sis on Education. No other measures Evidence_more strongly _supporting approached significance. a negative peer influence effect comesfromthe difference in drug The hypothesis that the SSC stu- use and class attendance between dents would exhibit increased posi- treatment and controlsin the year tive programmatic effects over time sample. Adolescent drug use is is not supported. If anything, the highly related to peer use and may negativeeffects ontheinterper- be subject to peer influence (Kan- sonal and intrapersonal variables del, Kessler, & Margulies, 1978). measured bythe SAES increase over The treatment students may be influ- time. encing each other to increase drug use. Likewise, the increased class Potential explanations of the cutting is consistent with the nega- negative treatment effects. To what tive peer influence hypothesis. Ix do we attribute the short- and all, the evidence for a negative long-term effects of the SSC treat- peer influence effect is equivocal. ment? Analysis of program implemen- Project personnelshould neverthe- tation, and discussion with program less be attentive to its effects. personnel suggest eight possible explanations for the combination of 2. A second possible explanation effects found: for the negative effects of the SSC treatment lies in the possibiltiy of 1. Composing the groups entirely a counselor-validated norm that, of students with academic and beha- "The school stinks, butyou need vioral problems may have resulted in thatpiece of paper." A primary non-conventional group norms. This goal of the Plymouth projectis to could have produced in- and out-of- increase the graduationrates of school friendship cliques that were potential drop-out students. One of predominantly non-conventional in the ways that the SSC addresses this nature.

-443- 329 Plymouth

goal- isto validate thestudents' attend program functions. Whether perceptions concerning the imper this is a longterm negative effect sonal nature ofthe school and the of the program can only beknown lack of caring of some of their through followup. teachers and parents, while at the sametime emphasizing tothe stu 4. A hypothesis advanced by the dents how important the high school project codirectors forthe nega diploma is. The students are taught tive selfreport measures (excluding to copewith secondaryeducation, selfreported drug use) is that the and survive in the building, but not positive intrapsychic effects of necessarily to enjoyschool. This the. program will only occur over explains how the students can become time. This was the reason that lessattached to school, decrease separatecomparisons were made of their involvement in school activi thefirst semester onlytreatment ties, and skip class more often, and and control groups. That analysis at the same time ,reduce their rate indicated, that if anything, the of voluntary withdiawals. Given the negative attitudinal effects strong relationship between attach .,increasedover time. Yet, it is ment to school and graduation rates stillpossible, although unlikely, (Bachman, O'Malley, & Johnston, that a still longer followup period 1978; D. Gottfredson, 1982) this is will be necessary to demonstrate a risky approach to the problem of positive effects. It must be said dropout. This is particularly true that at the current time the weight since the students are attending of the evidence argues precisely class significantlylessthan the oppositeto a delay of _positive controls, despite programmatic effects explanation. efforts to closely monitor atten dance. 5. Another hypothesis concerning the selfreport measures is that the 3. An explanation for the con treatment students responded more sistent negative effects on the honestly. A large part of the Parental Emphasis on Education affective education instruction pro selfreports comes from the data vided in the SSC class involves found in Table 4. Table 4 shows the teaching students to honestly evalu number of parent phone calls, parent ate their intrapersonal feelings and visits, and parent meetings recorded responses. Students are encouraged for each of the four Plymouth proj to openly express their feelings in ect treatments. Collapsingparent the class. Indeed, one of the pri visitsand parent meetings, it is mary reasons that it became diffi apparent that, although parents are cult to design a middle school SSC highly encouraged by program person that paralleled the high school nel to become involved in their model wasthat middleschool stu children's education, and are conti dents could not as easily trust one nuallyinvited for conferences and another to keep confidences so that program activities, the high school close personal feelings could be SSC treatment parents average less expressed without fear of ridicule. that 1.5 personal visits per semes The project codirectors argued that ter. Throughoutthe semester, the this intrapersonal reflection and studentsare behaviorallyreminded emphasis on honestly would contri' that their parents do not, in fact, bute to differences between treat supporttheir educational efforts. ment and control students through a They are given clear evidence that two stage process: (a) because of their parents do not care enough to the emphasis on examination of them-

-444- 330 Plymouth selves and their problems students were responding more honestly to the would become more aware of the prob- behavioral measures. Why should the lems they have--they would be carry- students honestly report their ille- ing less "false autonomy," and gal drug use, andfalselyreport (b) they would therefore be more other delinquent acts? likely to report honestly about their personal feeling and behaviors Obviously, differences in self- when responding to the question- report style cannot account for the naire. These are two subtly differ- negative programeffects on class ent processes, but they would com- attendance where school records were bine to make thetreatmentgroup the source of the measures. more likely to report negative events on certain measures, even if 6. Anotherexplanation for the they didnotactuallydiffer; in pattern of negative effects is that fact, they may not be as bad off as randomization broke down,andthat the controlstudents. Ironically, the experimental and control groups in the view of the project co-direc- differed at the outset of the evalu- tors such "negative" self-report ation. Table5 gives the results effects of the treatment could actu- for analyses of variance comparing ally be positive treatment effects. all program treatment groups and their respective controls on affec- It is difficultto evaluate the tive pretest measures gathered prior accuracy ofthis claim. Some data to the experiment. Because many of is available from the SAES Invalid- thestudents failed to adequately ity scale. Invalidity is a measure complete the pretest, N's for these of truthfulness in reporting and analysesare very low. Neverthe- contains items such as "I read sev- less, there is no indication from eral whole books every day." these data that the high school SSC Although this measure was designed students differed atthe beginning to detect studentswho were not of the treatment period. Only one responding seriously and honestly to comparisonout of nineapproaches the survey, it shows no difference significance, and this is well between the first semester treatment within the bounds of chance. andcontrolgroups (seeTable 3). But, the argument advanced is that 7. A seventh possible explanation the control students may not be as for the results is that the negative "self-aware" asthe treatment stu- self-reporteffects arereal, and dents, so the question is really not that .they may be the result of one oflyingon the questionnaire. inconsistent implementation of pro- Therefore, the invalidity scale only gram components. Unfortunately, we obliquely addresses the "honest" only have two sources of implementa- reporting issue. tion data. One is records of parent contacts, the other is the actual On the other hand,the "self-a- time, in days, spent in the treat- ware/honesty" explanation does not ment program. The correlations of explain the differences in self-re- parent contacts and time in treat- ported drug use;the measure is a ment with each other and with all factual report of use that does not outcome measures were calculated. relyon being more self-awarefor None of these correlations was sig- more accurate reports. Particularly nificant. Table 6 shows the corre- since there were no differences on lations betweentime in treatment the overall delinquency self-re- and parent contacts. It is apparent ports, it is difficultto hypothe- thatthere is no relation between size Chat. the treatment students

-445-

331 Plymouth

the amount of time thetreatment SSC. Theself-report results are students arein the class, and the seemingly negative(especially for number of parent contacts thatare drug use), and class attendance is made. If contact with the students' negatively affected, yet there is a parents is an integral part of the strong positive trend in the whole treatment process, then at least year sample in the key behavioral this component of the intervention measures of voluntary withdrawals is not being systematically applied. andout-of-school suspensions. We will haveto waitforreplication Relatedto implementationissue results and an examination ofnew is the significant increase in class data with higher control group res- cutting shown by the treatment stu- ponse rates to reach a final conclu- dents. Because class attendance is sion concerning the efficacy of the an important target behavior for the high school Student Services Center program,it is obvious that program model- implementation is breaking down. One ofthe very behaviors that'is One closing point should be noted specifically addressed is moving in ,,concerningthe high schoolStudent the wrong direction. Services Center: the program is very popular in the school system. 8. A final possible explanation It has been funded in the district for the negative self-report effects fornine years,and will be sup- could lie in differential response ported when federalfundingis no to the SAES, questionnaire. Table 7 longer available. School system lists the reponse rates for the var- administrators,at least, view the ious Plymouth project sampling program favorably. When confronted groups. Fewer of the high school with the interim evaluation results, SSC control than treatment students the typical administrator's response completed the questionnaire. Since reportedly is "Johns Hopkins must thestudents who are notsurveyed not be measuring the right things." may show poorer outcomes than more Although wecanhatdly imagine a cooperative students, someof the longer questionnaire, or more exten- negative effects may be dueto the sive datacollection, the project differential survey response rates. has cooperated in helping us design Of course,to explain the self-re- "affective education" measures. ported drug use results one would They have been incorporated into the have to argue that the non-surveyed 1983 version ofthe School Action controls differed from the surveyed Effectiveness Study survey. Further treatment students in their drug speculations await better data. use, but notin their delinquency. In any case, the differential survey High School Student Activities Cen- rates make evident the need for ter strengthenedeffortstoadminister the 1983 SAES student questionnaire A true experiment was carried out to control group students to achieve during the secondsemester of the high response rates for both treat- 1981-82 yearto evaluatethe high ment and control groups. school School Acitivities Center writing class (SAC). Table 8 gives Examining the entire pattern of the results for analyses of variance results, and the possible plausible comparing random treatment and con- and implausible explanations for trol students on measures of the them forces us to conclude that the SAC's goals. jury is still out on the high school

-446--

332 "lymouth

The Capitalization, Punctuation, the SACwriting classes suggests Words, Sentences, Sentence Struc- three conclusions: ture, Study Skills, and Essay measures are all derived from pre- and 1. Thestudents are improving post-tests measuring the writing theirwriting skills, and may be skills coveredduring the writing increasing their overall academic class. Of thesix test component performance, at least so far as it comparisons, (theEssaycomparison is measured by self-reported grades. is based on too fewpersons for Follow-up ofthese students should interpretation) three aresignifi- result in betterinformation con- cant, and one shows a marginally cerning general academic improve- significant trend. In all cases the ments. treatment students performbetter thanthe controlstudents. It is 2. Two psychosocial development apparent that treatment students are outcomes favor the SAC students: improving their writing skills. Alienation and Practical Knowledge. We do not know whether these effects Four other measures of the SAC's a direct result of the class and goals are also significant. SAC 'thethe counseling that accompanies it, clientsattempt moreclasses than or an indirect effectof increased the control students (and the per- academic success. Since the SAC and centage of classes they pass SSC students appear to be very simi- approaches significance,p = .12). larby the pretest measures (see SAC students report themselves to be Table 5),and the SSC program is less alienated, and to have greater explicitly a counseling program, practical knowledge. Althoughthe while the SAC is primarily a reme- SAC students report themselves to be dial academicprogram, one cannot receiving significantly higher but be struck by thepossibility grades, the results of the analysis that increased academic achievement of the actual grade data do not sup- results in better "psychological port the self-reports. As noted health"--and that the SAC is a more earlier,however, theG.P.A. data fruitful intervention than the SSC. are forthe entire academic year, andthe experimentalstudents were 3. The trend toward the SAC pro- only in the program during the sec- ducing lower self-reports of Paren- ond semester. Any program improve- tal Emphasis on Education is inter- ments would be diluted by the first esting because it parallels results semester pre-treatment data. The found for the high school SSC. It fourth effect is a non-significant was argued for the SSC that program trend for SACstudents toreport personnel seek to involve parents in that their parents put less emphasis their children's education, but on their education. report that they are largely unsuccessful. This may serve to rein- Table 9 gives the results of force for the students the knowledge analyses of variance comparing the that their parents do not care about treatmentand controlstudents on their educational progress. A simi- measures of the SAC's objectives. lar process may be operating in the None of the measuresthatdo not SAC. The control students in both overlap with SAC goals are statisti- the SSC and SAC may be in blissful cally significant (Alienation is ignorance oftheir parents rack of discussed above). concern about their education.

The experimental evaluation of

-447- 333 Plymouth

4. A fourth conclusion isthat The Middle School Student Services the SAC seems to be well imple- Center mented. This conclusion flows from two pieces of data. The first is Early in the 1981-82year an the observation that the SAC attempt was made to implement a true instructor has spent a great deal of experiment in the middle school Stu- time developing curriculum materials dent Services Center (SSC). This and pre- and post-tests to monitor design brokedown when there were the progress of his students. It is insufficient referrals to enable significant that he has revised his students to be randomly assigned to pre- and post-tests. This indicates treatment and control groups. that heis paying attention to the data he receives from his test, and As a fallback design, the project is responding accordingly. This is co-directors identified- a non-equi- a positive sign for any program valent, control group of students in development effort. the school, and administered the project pretest to them. Table 5 A second piece of implementation ,gives analysesof variance results data can be found in Table 10. This comparingthe SSCparticipants to table shows the correlations between their non-equivalent controls on parentalcontacts and time in the pretest self-report scales. Ofthe SACclass--measures of implementa- nine comparisons, five are signifi- tion--with selected pretest and out- cant. Three of the pretest measures come variables. Parental contacts (Alienation,Attachment to school, are highly correlated with several and Expression) favor the control of the outcome variables,notably group, and two (Belief in the rules those having do with discipline and Interpersonal Competency) favor infractions, and academic perfor- the treatment group. It is puzzling mance. Particularly strong (r = that some of the measures favor the .60) is the correlation between treatment group,and some the con-, classesmissed and parental control, but the consequence of the tacts; the SAC instructor is notif- gross differences between the groups ying parents when their children are is that it will be very difficult to not attending class. Also strongly explain any differences between correlated with parent contacts is groups on outcome measures because the Expression scale measured at it will be difficult to disentangle pretest; tilt more expressive the treatment effects from pretreatment student,the more the parents were differences. contacted. This pattern of correlations is in mrrked contrast to that Tables 11 and 12 give the results found in th SSC data in which the for analysesof variance comparing parental itacts were notcorre- the treatment and non-equivalent lated -ay other measure. Table controlgroupson measures of the 10 also shows that time in the SAC middle schoolSSC goals and objec- class is unrelated to parental con- tives. Of thirty-three outcome tacts, so, like theSSC, parental measures, fourteen show statisti- contacts are not uniformly distri- cally significant differences buted over time. On the other hand, between the groups and five others relationships between parental con- show strong trends. In all cases, tacts and outcomevariables show the treatment group shows poorer that the SAC instructor seems to be outcomes than the control group. responding systematically to the Recall, however, that it is probable behavior of the students. that the groups differed before

-448- 334 Plymouth treatment. The analyses imply that school attendance and withdrawal thetreatment doesnotchangethe policy. The old attendance and behavior of referred students to withdrawal policy was that students match that of non-referred students. were automatically withdrawn from (Analyses of covariance controlling the high schoolif they accumulated for pre-existing differences were eight unexcused absenses. The proj- uninterpretable; too few students ect co-directors,while serving on completed both pre- and post-tests the high school's Attendance Policy to allow for meaningful analyses.) Committee, were instrumental in getting this policy changed to one in One positive note for the program which there are no automatic atten- may be found in Table 13. This dance related withdrawals. Instead, table lists selected correlations students who acquire. eight unexcused between parent contacts and time in absensesarereferred' to in-house the SSC with thevarious outcome suspension upon their return. measures. 'The table indicates that the counselors in the SSC are The projectstaff was also pri- responding to misbehavior and ,,marily responsible for developing truancy by contacting parents. systematic disciplinary record-keeping procedures at the the two proj- Learning Options ect middle schools. Neither school had adequate procedures (one kept no We were unsuccessful in eliciting records at all). In this case,the an agreement from Learning Options national evaluation had an indi- personnel to implement a true exper- rectly positive effect on implemen- iment or any other satisfactory sum- tation. When the school with no mativeevaluation. SinceLearning record-keeping procedures resisted Options serves adolescents with sev- the development of such procedures, ere behavior problems, identifying a it wasrequired to comply by the meaningful non-equivalent control system superintendent after project group is difficult. Therefore, we staff informed the superintendent are left with no control group and that the national evaluation no outcome evaluation of the Learn- required discipline data. ing Options program. This is unfortunate, in that the Learning Optiions Organized student activities programappears to be a theoreti- sponsoredby the SACincluded not cally based and well-implemented onlythe springintramuralvolley- attempt to deal withproblemstu- ball program, buta dance group, a dents in an alternativeeducation "Thursday afternoon discussion format. group," and a club called "Musicians Unlimited." All of these groups For descriptive purposes Table 14 were student-initiated, and the SAC gives the means and standard devia- provided space and organizational tions of relevant outcome measures help. One unexpected positive bene- for the Learning Options students. fit of the dance group isthat it developed into a support group for Other Proiect Accomplishments the small number of blacks that attend Plymouth-Canton high school. Amajor accomplishment of the The center became a place where Plymouth Alternative Education Proj- black students could meet together ectin 1981-82 was the implementa- over lunch, dance, or just talk. tion ofa new Plymouth-Canton high

-449- 335 Plymouth

Major Forces Affecting the 1981-82 referred fromthe Learning Options Program (LO) component, resulted in few students actually being experimental During the 1981-82 schoolyear, treatment and control group members. two major environmental events caused difficulties for the Plymouth The result of these events at the Project: (a) requirements of the middle school was that an experimen- national evaluation, and relations tal evaluation of the middle school withthe nationalevaluation staff Student Services Center was not at the Johns Hopkins University, and undertaken, and the project was (b) a school district reorganization forced to identify a non-equivalent that moved ninth grade students from control group of middle school stu- the high school to the middle dents not referred tothe program. school. Inaddition to these two This procedurewas not successful major forces, the project was (see above). affected in lessdramatic ways by the poor economic situation in At the high school,the project Michigan, and obstacles presented by ,screened students more carefully for the physical education department in the second semester randomization one of the high schools to the process, and developed a group pre- implementation of the Student Activ- sentation procedure wherebypoten- ities Center's spring intramural tial program participants could volleyball program. learn about the program,and could consent to being involved in one or Evaluation Issues the other program before randomiza- tiontook place. This procedure, The relations between the while not eliminating the problem of national evaluationstaff and the treatment refusals, decreased it Plymouth Project remained rocky for sufficiently so that a true experi- much of the year (see the first ment ofadequate power was carried evaluation report), but showed much out in both the high school SSC and improvement bythe spring of 1982. SAC during thesecondsemesterof Efforts in the Fall of 1981 to 1981-82. implementtrue experiments in the high school Student Services Center Inthemeantime, project staff (SSC) and Student Activities Center found the randomization procedure, (SAC)metwith some success, but thecoredata collectionrequire- problems developed because of(a) a ments, and the administration of the drop in referrals (particularly at School ActionEffectiveness Study the middle school SSC) when teachers (SAES) questionnaire to all SSC and and administrators found that access SAC treatment and control students, to the program could not be guaran- LO students,and random samples of teed (due to the randomization pro- the two high and middle schools put cess), and (b) randomly assigned a severe stress on their resources referred students refusing to take (see the first interim report). A the treatment. At the high school, further difficulty wasa change in both the SSC and the SAC did under- the evaluation staff member assigned take experiments duringthe first to the project. The Social Action semester, but many of the students Research Center (SARC) field worker that were randomly assigned to the assigned to the Plymouth Project treatment did not receive it. This, left the evaluation in the fall, and coupled with the reservation of many the responsibility was assigned to of the slots in theprograms for theSARCCoordinator of Formative continuing students and students

-450- 336 Plymouth

Evaluation, who subsequently with planning,this is unfortunate: resignedin January of 1982. The A primary objective ofthe Program author was assigned to work with the Development Evaluation process isa Plymouth project in mid-January systematic increase in planning. 1982. Systemic change activities, The project staff attributes sev- although an interest ofthe project eral negative effects to the demands (see "Policy Changes" above),have ontheir time that the evaluation never been a project priority. imposed in 1981-82: (a) theyhad Project staff have repeatedly stated lesstime to devote to "systemic" that ,heirs is primarily a treatment goals, to working within the school intervention, and in fact, have system at large to humanize the edu- stressed to the national evaluation cational process, (b) the project staff theirdesire to keep a low co-directors could notprovidethe profile in project schools. As personnel support to the SSC and SAC regards the change in the roles of programs as they had intended, theproject co-directors, part of (c) staff coordination and planning ,thiswas to be expected, because decreased as staff meetings were they are,indeed, grant administra- cancelled to make time for evaluators; theaddition of a project tion activities,(d) program refer- staffer in charge of many evaluation rals decreased, (e) and the per- tasks for the1982-83yearshould ceived roles of the project alleviate some of those service co-directorschangedfrom that of delivery vs.data gatherer distinc- direct service providers to grant tions. administrators, evaluators and paper-pushers. The project co-di- Finally, to the extent that eval- rectors feared that this might nega- uation activities contributed toa tively affecttheirfuture in the lack of direct programsupport by school system once the Federal fund- the project co-directors, (and this ing runs out.' issue was raised by virtually everyone connected to the Plymouth proj- It is difficult to assess how ect), then the nationalevaluation serious each of the foregoing hurt the project. The difficulties effects has been. It is certainly derive in part from a lack of proj- true that there were referral diffi- ect staff available to plan for and culties, particularly at the middle accomplish evaluation tasks--a school, but these problems coincided requirement that wasnot clear in witha re-organization that placed the OJJDP request for applications. the projects primary source of Future requests for proposals should referrals, ninth graders, in the make more explicit (a) the time middle schools where the project was necessary for evaluation activities, newandin which referralsources and (b) therequirement that this had not yet been strongly developed. time be budgeted for in the original Concerning time for planning, if request for support. If rigorous evaluation activities did interfere evaluation is a requirement of a grant, as it is in the Alternative Education Program, then it should be a major criterion ofselection for 1 We have recently learned that the funding, as it was not (General direct costs of maintaining the Ply- Accounting Office, 1982). mouth project will be picked up during the 1983-84 school year bythe local school district.

-451- 337 Plymouth

The difficulties the project was plewould be, providing a weaker having in implementing evaluation evaluation of the school change aims requirements, the turnover in evalu- sought by OJJDP, but the project was ation staff, plus the project's held to doing the survey, and meet- feeling that they were spending all ing response rate benchmarks, which of their time on evaluation activi- they did. Johns Hopkins facilitated ties precipitated anger over a draft the Plymouthsurveyadministration of the first interim evaluation by pre-printing mailing labels for report sent to the project for com- the project's parental ccasent let- ments. The reportcriticized the ters. Negotiations between the par- project's compliance with the ties ultimately resulted in the Ply- national evaluation at a time (sub- mouthproject requestingfunds in mission ofthe third-year refunding their third year budget for an eval- request) whenthe project felt it uation staff person, which Johns could least afford such criticism. Hopkins stronglysupported in dis- The project directors therefore cussionswith OJJDPstaff. These responded with a writtendetailed funds were awarded for the third criticism of the report and the .,project year. national evaluation staff responsible for it. The working relationship between Johns Hopkins and the Plymouth proj- This event and its aftermath sym- ect is currently much improved. For bolize the entire year'snegotia- 1982-83 project personnel have tions between the project and Johns developed a more adequate non-equi- Hopkins: evaluation difficulties valent control group for the middle building to confrontation, culminat- school SSC(collecting betterpre- ing in compromise. The first treatment data, and matching the interim report was clarified to non-equivalentcontrols more care- underscore that while there had been fully to the treatment population). major problems with evaluation They have implemented strong experi- activities,the project was making ments in both high school programs, good-faith efforts to correct them, and have aided in the development andthat improvements wereoccur- and pre-testing of an SAES scale ring. The most important change designed to measure attitudinal that occurred was that the Plymouth changes resulting from affective staff began tosee ways to get the education. Continuing compromises various tasks done (if some compro- include the quasi-experimental eval- mises were made), and the evaluation uation of the middle school SSC, the staff was willing to allow the com- continuing use ofthe English class promises andprovidesome help in sample in the high school, withdra- order to see that the most rigorous wal ofthe requirement to follow-up evaluationpossible was performed. the1981-82English class sample, For example,instead ofa adminis- and the dropping or the Learning tering the SAES to a large random Options component from the third sample of the high school population year budget(this wasalso due to (a task that project staff feltwas the 30% budget cut requirement, but absolutely overwhelming), ten Eng- the lack of evaluatability of Learn- lish classes were surveyed; this was ing Options--seethe first interim a compromise for both parties: the report-- contributed to the deci- English classes are not as strong a sion).. climate assessment as a random sam-

-452- 338 Plymouth

School District Re-organization change. The ninth graders were anticipating beginning high school, The second major environmental withthe autonomy it presents,and change influencing the project was a the ninth grade teachers felt that district re-organizationthat took they were "demoted" from the high to ninth graders out of the high school the middle school level. and placed them in the middle school. Ninth grade students were A positive result ofthe re-or- the primary source of referrals for ganization is thatthe SAC program the high school program, so the was given more building space at the treatment population shifted upward high school than it had expected to one year to includemostlytenth receive. It had room to conduct its graders. More importantly, the student actvities, and to have a ninth gradepopulation wasnow in "home" room for the 'SAC writing the middle schools in which the project was new and little known,,so that referral contacts were not Other Forcefield Events strong. It became harder toiden- tify the students the project feels A third negative force inPly- are most suitable for the SSC treat- mouth's field is the severe reces- ment. In addition, the re-organiza- sionary economyin Michigan. While tion plan diverted many of the stu- Plymouth, Michigan was not as dents that the middle school program directly affected as Detroit proper, had been working with in the spring unemployment was up, and school dis- of 1981 to other non-project middle trict funding_ is_ largely, dependent schools where they could not be upon state funding. As a whole, the given the follow-up support that the state of Michigan is suffering a project feels is necessary tosuc- major fiscal crisis. cessful treatment. The recession affects the project The re-organization also led to a in two ways: First, it makes uncer- problem for the students returning tain future continuation of the from Learning Options to the regular project after Federal funding is school curriculum. Since most of discontinued. This affects the the Learning Options students have project now, becausetime must be been eighth or ninth graders, it had spentinvestigating and shoring up been relatively easy to return them additional sources of funds. For to the large, impersonal high school example, LearningOptions was not environment. At the middle schools included in thethird year budget, though,the project ran into diffi- but is fundedin 1982-83 primarily culties when principals ofthe two by local and school systemfunds schools were wary of allowingthe (see Footnote 1). Learning Options participants back in the school. The Learning Options A second recessionary effect students were eventually allowedto reported by project personnel at re-enroll, but project staff feel both the high and middle school that they aresubjected to undue level is that the students are suf- "surveillance." feringmorefamily/home problems, problems directly related to unem- Two additional climate effects of ployment. Duringthe year, unem- the re-organization are that both ployment in Michigan averaged about the ninth grade students and their 12%. Project staffestimate that teachers were angry about the among their client population (pri-

-453- 339 Plymouth

marily working class) that the chase their own volleyballs,as the unemployment rate may be as highas P.E. department would not let them 30 to 40%. While we cannot verify use its equipment. the magnitude of either of these negative effects of the recession, A related problemthat theSAC they certainly seem plausible. ran into was the lack of scheduling flexibility in thehighschool. It As previously noted, a fourth was difficult to schedule activi- negative environmental effecton the ties, and most had to be held after project was the required 30% budget school. cut for the 1982-83 year. This affected the projectin the 1981-82 Changes in the Force Field for year, because hard decisions had to 1982-83 be made about where thecuts should occur. Because the director of the The only anticipated major change Learning Options program and the in the force field isthe markedly Johns Hopkins evaluation staff had improved relationship with the been unable to come to an agreement ,national evaluation. This should concerning an adequate evaluation occur for four reasons: (a) a design by February, 1982, the proj- highly trained evaluation staffer ect decided to drop Learning Options has been assigned to the Plymouth from thethird year funding propo- project as their evaluation contact sal. Inthis manner, theSSC and and better continuity in and a SAC programs could be funded at smoother working relationship with their current level, and funds could evaluation personnel is expected, be acquired for an on-site evalua- (b) a full-time person has been tion staff person. It is unfortu- hiredto perform evaluation tasks nate that we were unable to imple- such as data collection, program ment an adequate evaluation for assignment, and questionnaire admin- Learning Options, perhapsthe most istration, (c) theprocedures for powerful of the Plymouthinterven- carrying out the experiments quasi- tions. experiments, and SAES questionnaire administration have now been "field Two forcefield events had effects tested" for a year, and (d) the primarily on the high school SAC Growth Works component_, over which program. In the spring of 1982, much time had been losttrying to when the project tried to implement develop an evaluation acceptable to an intramural volleyball program for both parties, is no longer a part of students not involved in other the federal initiative. school events,the project ran into difficulties from the physicaledu- Program Changes for 1982-83 cation instructors. They were wary of lending space and equipment toa A new intervention has been added venture designed for what they per- in the high school Student Services ceived to be marginal students. The Center to address the problem of SAC instructor had to see fourper- school non-attendance. The data sons, finally returning to his ini- fromthe 1981-82 experiment indi- tial contact before he receivedper- cated that that the SSC students mission to use the gymnasium were skipping school at-a high rate. facilities after school. This A group competition has been setup resulted in the program starting in which each "group" (class period) later in the semester than intended. competes with theother groups on The project was also forced topur- increasing attendance. This is an

-454- 3,10 Plytoutn

*14 i*.0 4., 19+ tqw-ot rtopoior0 10 target students' lunch tite--a time 44i thot,! 14,/* pietism, that *F4- (Whit which they often get involved 4 ot. 4. «4 0414 1 14** witis 4,40 *4 in disciplinary incidents.

A further chew involves the000 * *0./,440 .4****itkpi14* the vita Ot tutors. In 1981-42, non-program iii .004 4 14,0.144tki=4* t1O4ts seniors were used as tutors, butthe 040414 4,* 4 4474 one ot SAC ieetouctor frit that thisproce- 444: 0444 *« m*«1140-**4 4 ietitely dure WO notsatisfactory, becauue wit 44 tikktiit Slott 4 44iiikt RA% th* pro= tk* SAC students resented theexper- 4*.04.4.4114) it *** bopoilihrt tise of the tutors. In 1982-83 suc- ,4.4+ +r 11+01 'J14 ,oloti 001..ter *or= cessful gritluAtcs of the program 44i 44.17* 1)44* .0#4,4110 f4 44041140411 will do tutoring. 41404 4* 1101410- sorVO4 11pullet. #104** 4* 14p Kw's fee a 1000* +1 Pinally,the SAC instructor was '-44* *4*** bilk et4pt siellarstr- the senior class advisor during the 440) ttiWti sort14 test the benefits 19111-82 paT, and theirrepresenta- et.** onpvtience. sad **bid mat itive poop operated out '. of his tr aespilit tops the poop pre- office. or the following year. he **se) 4041 felt Met this will be thL tenth grade 4..w *ii advisor **it ****ttiott4Mi, 100.4 where he feels he can bemore of a $11044414,0 WO 414 *01 1,0000,11 from counseling aid. The SAC instructor 140; tt i w t to*, spooster showed wilt also advise the Student Forum iitlip iisterspostet MA tom. They (the high school studentgovern- Ceps leossiefrep diet limed this ment), and thatbodywilloperate ta8etiowd ties, fee 408241. out of the SAC offices. lnets44, tone ttwatets will be 4440*0.4 1* Off,* its pert facilite. At the middle school SSC, less toes in t40 'voter *C dewbose time will be spent on theone-to-one bonding, and the groups will be * cott4016110 ttioPO in the high started earlier in the semester. te4seol i* -*titli** of a taco' The program's student evaluation 1041 401 Meet* **4 *tog, The $$C instrument, the Student Behavior clovristis* porteivied a weed is work Evaluation (888) has been simplified itch 14.0 firma', of iiv deaths of for 19112-83. The instrument was slow Meade and Motives,se a 4eaigtied for use at the high school diettssimaat these lassos will level, and the middle school staff saittard &setts 14p 4911243semi- feel that it is too detailedand test:s complicated for middle schooluse. Data from an evaluation ofthe SSE in lie high sp4me4 Stud/eat Attie- supports their assessment.2 tostet 9040 emphasle will be ileee4 an the attivittes composest. Ste* ettlyittee will be pleased, and leiessestele willhe orgamiaed 1 4 $444* 044 fill and oprimg aessesters one trilllosolvie este parlitiptatit. iii* Floutth hove (the ',mach retied) ei wits* of l..formal theft is psaitm4d as topics each as eismitoolloo, 'lammed ptrosehood, and 2 Si. Report on utility for diagno- OelPitivo tore Early. Wimp s411 be sis of student Behavioral Evaluation dosSgmed te ostructere more ofthe Scales, April, 1982. Available from author.

341 Plymouth

References

Bachman, J. G.,O'Malley, P. M., & Johnston, J. Adolescence to adulthood: Change and stability in the lives ofyoung men. Ann Arbor, Mich.: Insti- tute for Social Research, 1978.

Carlton, R. The Plymouth-CantonAlternative EducationProject: Interim Report. In G.D. Gottfredson (ed.), The School Action Effectiveness Study: First Interim Report (Report No. 325). Baltimore: The Johns Hopkins Univ- ersity, Center for Social Organization of Schools, 1982.

Generel Accounting Office. The Office of Juvenile Justice and Delinquency Prevention's Special Emphasis Program Has Not Realized its Full Potential (GGD-82-42). Washington, D.C.: Author, 1982.

Gottfredson, D. C. Personality and persistence in education. Journal of Per- sonality and So0a1 Psychology, 1982, 43, 532-545.

Kandel, D.B., Kessler, R. C.,& Margulies, R.Z. Antecedents of adolescent initiation into stages of drug use: A developmental analysis. In D. B. Kandel (ed.), Longitudinal research on drug use: Empirical findings and methodological issues. Washington, D.C.: Hemisphere, 1978.

7456- 342 Table 1

Outcomes on Measures of Plymouth High School Student Services Center Goals

Treatment Control Goals M SD N M SD

Reduce alienation

Alienation .44 .27 50 .43 .25 42 .88 Improve academic skills

Self-reported school .33 .29 52 .30 .27 42 .45 effort

Self-reported grades 1.79 .76 43 1.91 .65 43 .41 9 8182 G.P.A. 1.66 1.53''45 1.82 1.57 45 .62

Classes attempted 5.34 .77 44 4.67 1.30 45 .004*

Classes passed 4.24. 2.06 46 3.91 1.99 46 .44

Percentage of classes .77 .35 42 .76 .31 44 .93 passed Reduce delinquent behavior

Self-reported delinquency .42 .20 46 .37 .21 38 .27

Self-reported serious .22 .23 46 .20 .23 39 .63 delinquency

Drug use .89 .17 42 .78 .30 38 .05*

Truancy petitions 0.00 0.00 57 .02 .13 60 .33 1 Improve coping behaviors

Interpersonal competency .73 .30 50 .84 .20 41 .05*

Rebellious autonomy .82 .27 51 .72 .35 42 .11

Practical knowledge 1.35 .40 51 1.34 .49 42 .91 Increase parental involvement in education

Parental emphasis on .32 .28 46 .42 .49 42 .007* education

* Statistically significant. 1 The Expression Scale was completed by too few control students for analysis. 2 A very weak measure of achievement.

-457- 343 Table 2

Outcomes on Measures of PlymouthHigh School Student Service Center'' Objectives

Treatment Control Objectives .P. SD N M SD

Increase attendance SAES non-attendance index 1.52 .57 52 1.64 .57 44 Classes missed second .32 102.86 45.70 43 73.89 46.74 semester 45 .004** Classes missed per 19.80 9.22 44 15.95 10.21 class attempted 43 .07* Reduce official voluntary withdrawals Voluntary withdrawals 9% -- 57 21% Voluntary and invol- 61 .10* 11% 57 21% 61 .18 untary withdrawals ..

Increase sense of belong- ,, ing and purpose Attachment to parents .41 .24 52 .41 .29 44 .98 Attachment to school .46 .25 50 .47 .27 43 .86 Alienation See Table 1. Involvement .07 .10 52 .13 .13 42 .02** Decrease expulsions Expulsions 0.00 0.00 54 0:00- -0:06 Administrative with- -00-1:60 .56 1.31 55 .70 1.74 54 drawals .64 Increase self-esteem Self-esteem .58 .18 49 .62 .18 41 .35 Increase participants' ability to own responsibility for their own behavior No measure Reduce out-of-school suspensions Out-of-school suspensions .08 .27 .53 .29 .89 52 .10* Increase parental involvement in schools Parental emphasis on See Table 1. education Develop more positive parental attitudes toward school and alternative education No measure Other theoretically relevant measures Belief .64 .25 51 .66 .24 42 .74 Internal control .48 .29 51 .49 .27 42 .88 Self-reported suspensions .31 .47 51 .28 .45 43 .72 In-school suspensions .34 1.00 53 .25 .58 53 .55

* Strong trend. ** Statistically significant. yu -458--. 344 Table 3

Plymouth High School Student Services Center Has Negative Effects on First Semester 1981-82 Students

Treatment Control .E Goals M SD N M SD N

Interpersonal Competency .76 .27 21 .91 .14 17 .05*

Self-esteem .52 .19 20 .73 .12 17 .0005*

t Rebellious autonomy .92 .14 21 .73 .36 - 17 .03*

Involvement .06 .06 23 .12 .12 17 .05*

Attachment to school .41 .27 22 .62 .25 17 .02*

Parental emphasis on .31 .31 21 .54 .32 17 .03* education

Invalidityl .10 .17 21 .11 .14 17 .98

1 A measure of truthfulness in reporting.

*Statistically significant.

459= 345 Table 4

Parental Contact Data, Plymouth

Second Semester, 1981-82

Parent Phone Calls Parent Visits Parent Meetings

M SD M SD M SD

High School SSC 2.26 2.59 .48 .72 .87 1.108

Middle School SSC 5.32 4.11 .81 1.16 .41 .99

High School SAC 1.76 1.72 .19 .50 .33 .12

1 P. Growth Works 9.68 7.91 4.46 5.52 .89 1.15 a, 0

346 Table 5

Pretest Measures on Plymouth Program Participants and Controls

High School SSC High School SAC Middle School SSC Growth Work T C T

Attachment to parents 6.6 5.8 .14 6.8 6.7 .92 6.6 6.8 .49 6.7

Affection from parents 6.0 6.2 .77 5.9 6.2 .69 6.1 6.2 .81 6.5

Alienation 1.7 1.9 .55 1.7 1.6 .94 2.0 1.4 .001** 1.4

Attachment to school 4.7 4.0 .26 5.9 5.04 .46 3.03 4.4 .01** 5.0

Belief 4.1 4.0 .95 3.6 2.8 .17 2.7 2.3 .07* 2.8

.; Expression 2.0 1.9 .92 2.7 3.6 .12 2.2 2.8 .04** 2.9

Interpersonal compe- 3.3 3.3 .91 2.8 2.5 .61 2.2 1.7 ..01** 1.8 tency

Self-esteem 7.5 6.5 .07* 6.6 5.0 .03** 4.9 5.0 .72 5.2

Internal control 2.9 2.7 .63 2.' 2.6 .63 2.2 2.2 1.00 2.5

* Strong trend. '* Statistically significant.

3 Sic Table 6

Correlations Between Time in SSC Class and Selected Client Characteristics: Plymouth High School SSC Class

Time in SSC Class

Client characteristic r p

Parental meetings -.11 .47

Parent phone calls -.13 .37

Parent visits -.09 .53

-462- 343 Table 7

Percentage Completing Student Questionnaire Plymouth, Spring, 1981-82

Treatment Control

High School SSC 81% 71%

Middle School SSC 91% 79%

High School SAC 81% 80%

Growth Works 62%

High School English Classes 97%

Middle School Random Sample 99%

463 350 Table 8

Results on Measures of Plymouth High School Student Activities Center Goals

Treatment Control Goals M SD N M SD

Reduce alienation

Alienation .36 .23 32 .56 .31 17 .02** Improve academic skills

Capitalization) 1.44 3.90 34 1.08 3.57 13 .77 1 Punctuation 4.91 3.15 34 -.38 3.15 13 .0000 * *. 1 Words .4.29 3.54 34 .54 .3.27 13 .002**

., 1 Sentences 3.74 4.70 34 1.15 4.86 13 .10* 1 Sentence structure 2.09 5.22 34 1.62 5.33 13 .78 1 Study skills 5.29 4.19 34 .62 4.72 13 .002** 12 Es ay ' 7.06 11.41 36 -12.50 10.62 6 .0004**

Self-reported grades 2.08 .55 36 _ 1-61_ _.78__18__.01"_ 8182 G.P.A. 1.62 .89 34 1.52 .63 21 .66

Self-reported school .29 .28 36 .29 .21 17 .94 effort

Classes attempted 5.41 .82 34 4.73 1.28 22 .02**

Classes passed 4.27 1.50 34 3.64 1.43 22 .13

Percentage of classes .80 .27 -; .65 .43 22 .12 passed Reduce delinquent behaviors

Self-reported delinquency .28 .20 32 .34 .34 14 .35

Self-reported serious .14 .19 32 .21 .22 14 .28 delinquency Drug use .59 .39 34 .63 .35 14 .77

Truancy petitions 0.00 0.00 34 0.00 0.00 25 1.00 Improve coping behaviors3

Interpersonal competency .81 .21 33 .75 .27 17 .44

Rebellious autonomy .85 .24 33 .88 .27 16 .72

Practical knowledge 1.48 .47 32 1.13 .64 17 .03** Increase parental involvement I, education

Parental emphasis on ed. .32 .29 34 .47 .30 17 .10* 1 This is the average change from pre- to post-test. 2 Very low n's make this analysis inconclusive. 3 The Expression scale.was not completed by any control students. * Strong trend. ** Statistically significant. -464- 351 Table 9

Results of Measures of Plymouth High School Student Activities Center Objectives

Treatment Control p Objectives M SD N M SD

Increase attendance

SAES non-attendance index 1.53 .61 36 1.63 .50<16 .58 Classes missed second 91.41 55.79 34 79.82 50.69 2 .43 semester --..., Classes missed per 17.05 10.14 34 18.55 13.71 22 .64, class attempted Decrease official voluntary withdrawals

Voluntary withdrawals 16% 44 8% 25 .57 Voluntary and involuntary withdrawals 16% -- 44 8% 25 .57 Increase sense of belong- ing and purpose

Attachment to parents .45 .27 36 .32 .26 18 .12 Attachment to school .49 .30 35 .40 .31 17 ,36 Alienation .36 .23 32 .56 .31 17 .02* Involvement .06 .09 34 .03 .08 16 .25 Reduce expulsions

Expulsions 0.00 0.00 44 0.00 0.00 25 1.00 Administrative with- .52 1.35 44 .76 1.33 25 .48 drawals Increase self-esteem

Self-esteem .64 .17 30 .59 .24 13 .45 Increase ability to own responsibility No measure Reduce out-of-school suspensions

Out-of school suspensions .11 .49 44 .08 .28 25 .75 Increase parental involvement in schools No measure Develop more positive parental attitude toward school and alternative education No measure Other theoretically relevant measures

Belief .59 .24 33 .54 .27 17 .48 Internal control .58 .23 33 .56 .28 17 .81 Self-reported suspensions .23 .43 35 .29 .47 17 .62 In-school suspensions .20 .55 44 .44 1.08 25 .24 *Statistically significant -465- .., Table 10

Correlations of Two Implementation Measures (Parent Contacts and Time in SAC Class) with Selected Client Characteristics for the Plymouth Middle School SAC Random Treatment

Implementation measure and client characteristic

Total parent contacts

Administrative removals .40 .003

Outofschool suspensions .48 .001

Total suspensions .43 .004

Classes missed .60 .001 Classes passed .32 .06 81-82 G.P.A. .32 .06

Expression Pretest .71 .004

Time in SAC class

Parental meetings .09 .58

Parent phone calls .00 .98

Parent visits .03 .85

466 353 Table 11

Outcomes on Measures of Plymouth Middle School Student Services Center Goals

Treatment Non-equivalent Control n Goals M SD .N PS SD

Reduce alienation

Alienation .46 .25 38 .50 .25 76 .57 Increase academic skills

Self-reported grades 1.53 .94 36 1.93 .77 81 .02*

Self-reported school .15 .18 35 .37 .32 83 .00002* effort

Classes attempted 7.32 1.13 43 7.37 1.17 98 .83 Classes passed 5.72 2.19 43 6.65 1.49 98 .004*

Percentage of classes .76 .25 43 .91 .14 98 .0000* passed Reduce delinquent behavior _ _

Self-reported delin- .60 .29 32 .40 .27 74 .0009* quency

Self-reported serious .53 .31 32 .33 .28 75 .002* delinquency

Drug use .80 .30 49 .52 .39 77 .0000*

Truancy petitions .02 .15 45 0.00 0.00 105 .13

Detentions 1.60 2.17 20 1.31 1.71 52 .55 Improve coping behaviors

Interpersonal competency .63 .24 36 .70 .24 75 .18

Rebellious autonomy .65 .31 35 .70 .34 76 .43

Practical knowledge 1.33 .37 34 1.33 .51 78 .97 Increase parental involvement in education

Parental emphasis on .50 .29 36 .49 .27 80 .87

*Statistically significant. ?Ate 12

'.4t04410* 4014.miuro,of Plymouth Middle School itimelest Services Center Objectives

4 ibmilleOr Troot000t Non-equivalent Control 0411.410,r4, X SO SD N Is et, wpm .tt orm

11,.,q1",111i A44144/1444. 1.54 .73 37 1.11 .88 84 .01** Afro-mad *mac 5.02 .99 43 5.02 .70 99 .61 1tmomov4944 si,ofmrspopu 6.4$ 11.07 43 5.36 5.60 99 .03** tufAttalaixitLeditata vtmotonta ; onrirp It% 45 62 109 .51 6104motorf mfil towlomp. (*f m4664cow416 It% .. 451 62 109 .51

44(001* .17 .27 )7 .42 .29 85 .41 440000,44. *04 .29 .21 36 .36 .27 82 .09* 44440mlop1 t.#41 .51 .24 34) .46 .24 79 .25 limmtvoftofts .09 .14) 36 .15 .19 80 .10*

1-44LtellettAtit".-Ate, f4041004* 0.00 45 0.00 0.00 106 1.00 Ailloot,tfottlio 040- 5.20 6.09 44 2.00 4.40 102 .0005** 4rows$4 tivatz., tan .49 .14 12 .58 .15 75 .004** Ilmockettamtv,"

4iwirio foxtbutt

.441/4004114n4 1. 91 2.29 44 .66 1.23 106 .0000** W-gtOttf.441MtiALOVALY4'tan* fibbf**14. 4040i1144444i 00. Tadao 5. **woo-04os

4014M41440440,14.1101MA

.44 .29 11 .55 .25 78 .10* Im4ofma psomotroll .11 .24 5$ .46 .27 78 .22 46011twOopotto6 6m*poototon6 $66 .49 11 .49 .50 80 .09* log064600i 406Opoo61006 $65 1.21 20 1.08 1.34 32 .28 1. 08*

4001 140140.011161410+81,18$18844. .468 355 Table 13

Correlations of Two Implementation Measures (Parent Contacts and Time in SSC Class) with Selected Client Characteristics for the Plymouth Middle School SSC Treatment

Implementation measure and client characteristic r p

tft Total parent contacts

Excused absences .35 .02

Administrative removals .26 .09

Truancy petitions .47 .002

Court referrals .44 .004

Time in SSC class

Parental meetings .11 .69

Parent phone calls .09 .73

Parent visits -.05 .84

-469- 356 Table 13

Correlations of Two Implemei (Parent Contacts and Time with Selected Client Chz for the Plymouth Mid( SSC Treatment

Implementation measure and client characteristic

tft Total parent contacts Excused absences Administrative removals Truancy petitions Court referrals

Time in SSC class Parpntn1 mpptinpq Educational Improvement Center--South Alternative Education Project: Second Interim Report

D. E. Rickert, Jr.

The Education Improvement Center- Rickert, Gottfredson & Advani, South (EIC-South) is a technical 1983). Rather, the project is based assistance and resource center. The on a flexible set of ideas or primary clients are educators, but hypotheses. In the EIC-South Alter- many community groups and individu- native Education Project,staff do als use the resources of the Center. what seems most appropriate at the The catchment area for the Center is time. southern New Jersey, which is com- prised of Atlantic, Camden, Cape The basic guiding ideas of EIC- May, Cumberland, Gloucester, and South's Alternative Education Pro- Salem Counties. ject are best presented in the project staff's own words; they appear The Alternative Education Project as an appendix to this report (see is one of many programs sponsored by Appendix A). EIC-South. It is the only project within the larger agency that pro- Objectives videsdirect services on a large scale. The details of EIC-South and In the first Interim Report many its, organizational arrangements are objectives and suggested measures described more fully elsewhere (Ric- were described. Measurement was kert, 1982). only im-plemenied--for -sa-meof fhese- (see Table 1). Project staff mem- During the 1981-82 schoolyear, bers were uncomfortable with the the Alternative Education Project's structure elicited using the PDE delinquency prevention efforts were model, andfelt a morerealistic targeted at four schools: Pleasant- description of their project was ville Middle School in Pleasantville preferable. At the end of the year, and Cleary Middle Schoolin Buena, th refore, project administrators both in Atlantic County, D'Ippolito revised the explicitstatement of Elementary Schoolin Vineland, Cum- the problems their program was berland County, and Pennsauken Jun- addressing and the corresponding ior High in Pennsauken, Camden goals and objectives (see Table 2). County. Interventions During the 1981-82 Problems, Goals and Guiding Princi- School Year ples In the first Interim Report, five During the 1981-82school year general components of 'the EIC-South the explicitly identifiedproblems program were described: (a) a and goals (i.e., those identified in school climate improvement compo- the Program Development Evaluation nent, (b) a community process compo- plan) were the same as described in nent, (c) a youth participation com- the first Interim Report, but dis- ponent, (d) a public relations cussion with -program staff made it component, and (e) a leadership and apparent that the project does. not training resources (technical assis- operate on a "theory" in the sense tance) component. that it is used in Program Develop- mentEvaluation(PDE; Gottfredson,

-471- 358 New Jersey

In reality,the major thrust of project director explained that it the program, in terms of direct ser- was important for Teacher/Counselors vices provided, were alternative (those responsible for carrying out education classes, called School/ this intervention) to gain experi- Community Relations classes, held in ence in their assigned schools and each of the four target districts; a then execute the program as seemed "self-contained" class in Penn- appropriate. sauken;) and "Community-Process" in two ofthe target districts,Vine- In short, the Teacher/Counselors' land and Pennsauken. Many of the activities are not guided by a sin- target students also received indi- gle model, but rather by a casework vidual counseling from EIC-South approachwhere what seems intui- personnel. 'A small number of stu- tively correct becomes the interven- dents participated in "special tion. clubs" (e.g. dance club, etc.). Some tutored students in lower The general aims ofthe School/ gradesand some receivedtutorial Community Relations Classes were the 'services. (See Tables 3 and 4.) A following: (a) teach students cora- student referred to the Alternative "munication skills, (b) help students Education Project could receive as improvetheir self-image, (c) work few as one treatment ora combina- at resolving students' family prob- tion of as many as six of the listed lems, (d) improve study skills, and treatments. This depended upon what (e) aid students in career explora- project personnel perceived the stu- tions. dent's needsto be,as wellas on scheduling convenience. The follow- One Teacher/Counselor was ing sections describe the project's assigned to each of the four target major interventions. schools. Students chosen to participate met several times a week with School/Community Relations Classes their Teacher/Counselor in a classroom setting. Teacher/Counselors The wayin which this interven- seemed to have a lot of flexibility tion was executed evolved during the to do whatever they thought best 1981-82 school year. There was not, with particulAr students, subject to at the outset, a clearly articulated guidance bythe Educational Direc- set of implementationblueprints, tor. They were responsible for although training in preferred keeping the Educational Director and approaches took place during the Project Director informed oftheir preceding springandsummer. The treatment plans,attendance, paren- fact that there were no blueprints tal contacts,and schedules. Ser- for implementation is related to an vices provided included counseling, incremental approach to planning home visits, tutoring in basic followed by this project. The skills,and arranged meetings with community leaders.

In additionto classtime with 1. The "Self-Contained" class in students, Teacher/Counselors devoted Pennsauken existed prior to the a considerableamount of time to EIC-South Alternative Education Pro- providing advice and consultation to gram as a special class for students other teachers in the school, parti- with behavior problems. When the cularly teachers of participant stu- EIC-South program became opera- dents. They reportedly alsomet tional, the self-contained class was frequently with vice-principals with adopted as part of the program. regard to discipline policy.

-472-

359 New Jersey

Comments by principals (1982 Process Consultant, thenconducted Principal Questionnaire) at the four student-family interviews to deter- target schools indicate that the mine whichyouths would volunteer project was perceived asa program for the project. Teacher/Counselors to improve students' attendance, then met with school guidance grades, discipline,and self-image counselors to schedule volunteers' through individual counseling and project participation into their basic academic instruction. One school schedules. principal of a project target school stated that he would like the pro- The project did not receive ject to develop more of a blueprint enough referrals, to constitute for operation such asa curriculum treatment and control groups as guide. The 1982-83 program is based planned. on a more explicit curriculum guide. Self-Contained Class in Pennsauken How students wereselected for the School/Community Relations At Pennsauken Junior High School, Classes. As early as the proposal- the EIC-South Alternative Education writing stage, nine selection crite- Project alsooperates a Self-Con- ria were agreed upon by Alternative tained Class. Students assigned to Education Project staff. Theyare this class stayin the same class- the following: (a) poor attendance room for the entire day. The Penn- record,(b) high number of suspen- sani,en Self-Contained Class was in sions, (c) low academic performance, operation several years prior to the (d) perceived "drop-out potential," EIC-South Alternative Education Pro- (e) history of delinquency, ject and was incorporated -into- the (f) history of discipline problems Project when it began. in school, (g) court involvement for status offenses, and (h) high degree An alternative-Education Project of parentalcommitment to partici- staff member is assigned full time pate. Also listed as a criterion to the self-contained class. He was that students should be referred conducts the School/Community Rela- by school administrators, teachers, tions curriculum daily (i.e., commu- school counselors, or parents. nication skills,self-image, family Early discussions with the national probems, study skills, andcareer evaluation staff ledto a plan to exploration), teaches history, and develop pools of at least 60 stu- supervises the students' study per- dents meetingproject criteria at iod. In addition to the EIC-staff each school;30 wereto have been member,three staff teachers,spe- randomly assigned to the treatment, cially selected for their ability to and 30 to the control group. deal with "problem" students, visit each day to give instruction in At the end of the 1980-81 school mathematics, science, and English. year, copies of the criteria for studentselection were distributed The same criteria as for the to all administrators, guidance School/Community Relations classes counselors, teachers, and principals were used to select students for the who hadcontact with sixthgrade Self-Contained Class except that students. Meetings with the the certain students were judged to be potential referring agents were held more likely to benefit from the in each district to determine refer- Self-Contained Class. The criteria rals for the project. Each team of underlying these judgments are not project staff, consisting of a specified. Teacher/Counselor and a Community

-473-

360 New Jersey

Community Process Component althoughthe project director per- ceives that awareness was increased. ThreeCommunity ProcessConsul- tants were employed by EIC-South Project Process during the 1981-82 school year. One worker was, assigned to Vineland, one We do not know as much about the to Pennsauken, and one to both Buena implementation of this project as we and Pleasantville, since they are in would like. The ad hoc nature of the same county. Apparently, only program operations makes it diffi- in Vineland and Pennsauken was the cult to monitor implementation. Our component fully implemented to the early attempts to get project manag- extent that students actually parti- ers to be more explicit about what cipated (see Table 3). they planned to do and why may have served toalienate them. Much of The Community Process Consultants the early numeric feedback relayed were charged with primary responsi- to project personnel was seen as bility for coordinatingin-service abstract and, therefore useless. trainingfor all teachers in the Put another way, both the spontane- target districts, covering such top- ous nature ofthe project and dif- ics as violence and vandalism, dis- ferences in the language-nd per- pute resolution, parenting, teacher spectives of implementers and effectiveness,counseling services, evaluators havesometimes made it the criminal justice system and dis- difficult for us to engage in useful cipline. Another major function of process evaluation. the Community Process Consultants was tocall the homes of project Interviews with key project staff target students who were absent. indicatethat the, project experi- Theyalso provided counseling ser- enced no substantial opposition dur- vices for a smallnumber of stu- i- its firstyear of operation. dents. taft r t that key school system personnel have become more receptivc, The Community Process Consultants to the project over time.' This had key responsibility for organiz- accords withthe comments made by inglarge committees of community principals in the 1981-82 Principal figures and school personnel. Some Questioillaires. But byand large, students served on these committees the EIC-louth Alternative Education (See Table 3). The purpose of these Project is a "black box." At the meetings was to attempt to put into end o the 1981-82 program year it action the Project Director's theory was evident that thePDE about how institutional change Jcess was not being used by pl.:-:- occurs (See Appendix A). ject personnel in the spirit that. it was intended, and therefore was cot Generally, large meetings were providing usefulinformation ah,:.?ut called where key community members project plans and implementation. would discuss community problems and their causes. This process,in and Fortunately,there is ri,,,s(%1 to of itself, was considered to be believe that by the end of the therapeutic. A hoped for outcome 1982-83 school year, the pioject was that people with power in the will be less of a black box be.:ause communities would become more aware the project developeda' newplan of the problems that youth experi- according to its ownmodel, and enceand then do somethingabout motivation todocumentimplementa- them. The actual outcomes of these tion is now higher. A clear meetings are, by and large, unknown,

-474- 361 . u New Jersey workplan with milestones has been the OJJDP project officer and a mem worked out. Also,the project has ber of the Polaris (Technical Assis reworked its theory into a guide for tance) staff, a meeting was held at action that personnelsee as wan the OJJDP officein Washingtonto ingful, and reduced its list of discuss the importance of the goalsand objectives. Finally, a evaluation and obstacles to proceed fulltime Evaluation Coordinator was ing. The project director did not hired by the project. teel that the evaluation would be useful andhad reservations about Interim Outcome Evaluation the appropriateness of collecting information aboutindividualsin a Design Evolution governmentsponsored demonstration project. The meeting did not result After earlier plans to conduct a in an immediate solution, but in true experiment were abandoned (see January,1982, a full time Evalua the first Interim Report), new plans tion Coordinator was hired bythe were made in August, 1981 for a qda project. In the weeks following the siexperiment. The new design meeting and during a site visit made called for identification of none 'to EIC`youth in January1982, new quivalent control groups in each of arrangements for evaluating the pro the target schools. Also, at the ject were made. The idea of "con suggestion ofthe Project Director, trol" schools was abandoned as control schools were to bechosen /Inf0asible. Again, the agreed upon for each of the four target schools, design was a nonequivalent control matched on demographic characteris group comparison. Students who were tics (i.e., SES, Race, school size, in treatment as of -January-30-, -1982 average achievement test scores and were to comprise the treatment school grade structure). A college group. Allotherstudents on the student was hired parttime todo waiting list and new referrals would the workinvolved with collecting be inthecomparison group. This the data for matching schools. design, ofcourse, depended on the Within each control school referrals program having reached a stable were to have been solicited for sev stage whereby new referrals and enth and eighth graders based on the waiting list students would not same nine criteria as in the target enter treatment during the year. In schools. lieu of a survey pretest, the pro ject wastocollect pretreatment Because of the nonequivalent data from records from the 1980-81 nature of comparison groups the schoolyear forall treatmentand design depended on the administra control students. tion of a pretest survey in the fall of '81 toall treatmentand The expectation that treatment control students, the controls in students would remain. in treatment the target schools as well as in the and control students wouldremain control schools (see Table 5). untreated was not met. By March, 1982, virtually all students in the By late October,1981,the pro control group had received some type ject was not ready to administer a of service (see Table 6). pretest. With the assistance of

475

362 PO. New Jersey

Relationship Between Known Back- Several additional measures of ground Characteristics of Target background characteristics were Students and Amount of Various availablefrom school records for Treatments Received the previous (1980-81) school year. These were: number of absences dur- Because Fall, 1981, pre-tests ing the 1980-81 school year and num- were not administeredas initially ber of grade retentions in the stu- planned,there were few pre-treat- dent's history. Data on 1980-81 ment data available fora thorough suspensions were provided but were exploration of the relationship bet- missing for most students. Indivi- ween background characteristics and dual subject grades for the 1980-81 treatment. school year were provided but were not used because, for two schools, Several individual items and sca- many of the correlations between les from the 1982 Spring Student individual 1980-81 grades were Questionnaire can be viewed as mea- either peculiarly lowor negative sures of background characteristics, (see Table 7) and correlations bet- for instance, the Parental Education ween 1980-81 and 1981-82 overall 4 scale. Correlations, between this GPA'swere low for Vineland (r = variable and the amount of treatment .02) and negative for Pennsauken (r received were non-significant. The = -.19). This pattern of correla- correlation between student self-re- tions suggests that the data may ported sex (coded 0=female, 1=male) have been incorrectly transcribed. and the number of quarters in a special club wassignificant(r=-.21; Number of absences during the p<.05). The correlations between 1980-81 school year was signifi- students' self-reported age and cantly positively correlatedwith number of quarters in community pro- community process participation cess (r=.22; p<.01), counseling (r=.18; p<.05),counseling (r=.16; (r=.18; p<.05), and self-contained p<.05), participation in special class (r=.48; p<.001) were signifi- clubs (r=.23; p<.01), and participa- cant, indicating that the older tar- tion in the self-contained class get students tended to receive these (r=.28; p<.01). treatments. Self-reported grade level wassignificantlypositively The number of retentions in stu- correlated with the number of quar- dents'histories was significantly ters in the self-contained class positively correlated with community (r=.40; p<.001) and tutoring (r=.19; process participation (r=.24; p<.05). Self-reported race (coded p<.01), participation in special 1=white, 0=a11 others) was corre- clubs (r=.18; p<.05), and participa- lated significantly with the number tion in the self-contained class of quarters in tutoring (r=-.22; (r=.34; p<.001). These correlations p<.05), school/community relations are summarized in Table 8. class (r=.19; p<.05), community process (r=.47; p<.001), counseling Implications of these correla- (r=.28; p<.01), and self-contained tions. The correlations just class (r=.35; p<.001). The implica- described imply that students parti- tion isthat the students partici- cipating in the various program com- patinglongest in school/community ponents are different inimportant relationsclasscounseling andthe ways from students who did not par- self-contained class tended to be ticipate--i.e., the comparison group white, whereas students receiving is indeed non-equivalent. It will tutoring tended to be non-white. therefore be necessary to attempt to adjust statistically for these

-476-

363 New/ Jersey

pre-existing differences in search- both statistically significant. ing for the effects of participation These results should be interpreted in the various project components. withcaution becausepre-treatment GPA data were not available in the Analytic Methods schools where these interventions wereimplemented. Weredependable Because, by the end of the pie-treatmentdata available, they 1981-82 school year, therewere vir- would beused asstatisticalcon- tually no control students, a trols to probe the plausible regression approach to data analysis hypothesis that this correlation is was applied, using number of quar- due to pre-existing differences bet- ters in various treatments as pred- weenparticipants and non-partici- ictors in separate regressionequa- pants. tions. All target students in schools where treatments were imple- Measures from the Spring 1982 Admin- mented were included in each analy- istration of the SAES Student Survey sis, but if a treatment was riot implemented in a school, that school The Student Questionnaire was was not included in the analysis. administered anonymously to 7th and The project provided data on the 8thgraders in two of thetarget number of quarters that students in schools (Pleasantville and Buena), the target group spent in each type 7th graders in Vineland, and aran- of treatment. Number of quarters in dom sample of 7th, 8th, and9th each treatment was coded from 0 to 4 graders in Pennsauken. Response in increments of .25 quarters. With rates and school level results are this coding the unstandardized reported in-the oirerVieircif-the-SAES regression coefficient (slope) in second interim report. the regression equation can be interpreted as the value on the out- Questionnaire booklets for target come variable associated (i.e. students were identified with a con- added or subtracted) with one school fidential survey numbernecessary quarter in treatment. for merging these survey data with information aboutparticipation in Measures from School Records project components so that regression analysescould beperformed. Results for a variety of measures Unfortunately, students who were no obtained from school records are longer in treatment at the time of shown in Tables 9 through 12. survey administration were not given Because students participating in theassigned booklets; therewere various project components differ many missing data. Fifty (31%) of from non-participants ona variety the target students were not admin- of pre-intervention characterittics, istered their assigned booklets. In it maybe misleading to interpret addition, item non-response (a fai- simple correlational results as lure ofstudents to answer many of effects of participation. The the questions in the booklets) made results shown in Tables 9 through 12 it possible to score only about half are results of simple correlational the scales, for each target student analyses of this kind. (samplemean=.49). Biases of an unknown nature may have resulted The association between number of from this attrition and non-re- quartersin cross-age tutoring and sponse, so the survey data are of GPA and the association between much diminished value for 1982. self-contained class and GPA are

-477-

1P4 New Jersey

Becausethere were few students the Rewards Index, and the associa- participating in either special tion betweenparticipation in the clubs or cross -age, tutoring and School-Community Relations Class and because ofthe previously mentioned Involvement) remained. See Table missing data problem, student ques- 13. tionnaire outcomes were not analyzed separately for these groups. The An obvious interpretation ofthe same goesforthe Pennsauken Self- analysis of Student Questionnaire Contained Class. outcomes would bethat. the program did nothave negative effects but The same regression approach that neither did it help. The problem was described for the data from with such a conclusion isthat the records outcomes was applied here. students who had left treatment at In cases where the association bet- thetime of survey administration ween a type of treatment and an out- were not administered an identifia- come was significant, a multiple ble survey. This could mean that we regression approach, utilizing step- essentiallyhadno measurement of wise regression was applied.2 This the project's successes. (Alterna- , procedure yielded an estimated tively it could mean we have no mea- regression coefficient (labeled "b" sure of the project's failures!)We in the tables) that is adjusted for will have to wait until next year to the contribution of other variables. be ableto say anything definitive Of the adjusted associations between about programeffects on outcomes treatment and outcome, only two measuredby theStudent Question- apparently negative effects (the naire. association between number of quarters in counseling and the score on The Evaluation Design for 1982-83

Finally, a non-equivalent control group design with appropriate num- 2. TheSPSS (Nie, Hull, Jenkins, bers oftreated and untreated stu- Steinbrenner, & Bent, 1975) stepwise dents has been agreed upon and regression procedure withlistwise implemented. The following number deletion of cases with missing data of treatment andcontrol students was used. Stepwise regression were chosen at the beginning of the essentially allowsthe computer to 1982-83 school year: build a parsimonious statistical Treatment Control model of the criterion variable. In Vineland 35 46 all cases, the computer run was set Pennsauken 30 36 up so that it was possible for the Pleasantville 38 39 following variables to enter into Buena 34 35 the model: race, grade level, age, school, number of quarters in coun- A Student Questionnaire pre-test was seling, community process, tutoring, administered to all in October, 1982 special club and self-contained and the response rate was virtually class. In the currentsituation, 100%. A variety of pre-treatment the purpose for performing the step- data from records have been provided wise regression was to determine the for twopre-treatment years (i.e. value of an estimated "adjusted" 1980-81; 1981-82). These data regression coefficient and totest include standardized achievement its significance. Parameters in the test scores, the Tennessee Self-Con- regression procedure were chosen to cept Scale scores, number oftimes set the tolerance at .5 and the p to suspended, GPA, attendance, promo- enter at .05 (see Nie et al., 1975).

-478- 365 New Jersey tion record, age, race, grade level, ner. and sex. The data appear on initial inspection to be of high quality and The marginal results of the eval- contain few missing values. uation of 1981-82 outcomes drove home the point about the need for a The on-site Evaluation Coordina- more intense level of intervention tor, now a veteran of the difficult and a much stronger evaluation position she took on, has made plans design. Students in the 1982-83 to administer the Spring 1383 Stu- cohort are staying in treatment much dent Survey in a more careful man- longer (most for the entire year).

-479-

366 New Jersey

References

Gottfredson, G. D., Rickert, D. Gottfredson,'D. C., & Advani,' N. Stan- dards for program development evaluation (Report No. 341). Baltimore: The Johns Hopkins'University, Center for Social Organization of Schools, 1983.

Rickert, D. E., Jr. The Educational Improvement Center-lUuth's alternative education project. In G. D. Gottfredson (ed.), The School Action Effec- tiveness Study: First interim report (Report No. 325). Baltimore: The Johns Hopkins Univeristy, Center for Social Organization of Schools, 1982.

-480- 367 Table 1

The Original PDE Objectives of the EIC-South Alternative Education Project and Actual Measures of Achievement Employed

Objective Measures Employed

Goal 1: Reduce (Prevent) Delinquency

1. Raised self-esteem of students in Program classes SAES Student Questionnaire Postive Self Concept scale

2. Decreased negative peer influence SAES Student Questionnaire Negative Peer Influence scale

3. More mature attitude toward drugs and persons

4. More respect for public property (schools) and persons

5. Awareness and "buying into" school rules by students and Awareness possibly measured by Clarity of Rules scale teachers

Goal 2: Academic Success

1. Improved teacher attitudes with regard to student's academic success

2. Improved physical environment

368 .144* 1 ftW0414 ! .1. Inc 32 XIS ... 151s 1.44.0,.44 4-1.4044 relp 4 ifirt*0

#r OW ja 4 44-.4., 4 .4,4 *.t* 14.vAt* ditilmootIkok4 fitomw:t.11$11 ow!'

, A? NI 04.

p 44 9.4 4 400 14 440.40 *44

4 ,,f.jt 0.. ot,i4r14 ,.44, 444«. +40.1,.44

0444.!.,'44 4+40444 ..11,40 16,0* t44f4i0 44911I.41t1t."0 mA11,104 4+44 0Am#611m,

0444-w0,0. 40 '4,4+ .4 '44t444!,

tsvoto 1444.1* 44 t 101 ,- :t '1 " it r I' t 4,11%1[1,11T ..... % ,44 tt47141 ,4,/, 4 44- 0.4441

4 V+ 0.440 $4-1 44.11.4} 0,4 44 lt,..04 0.

.4.40444 4040. po 4 ,t,444,44,

*AU .4 ROA,- **V.. 1..414 .4 yowl 4411v, i040wilk mi1momou040 mol 14011,9evi ems WI* 0,40, 40.000P1*,044.1 pi iteopp.14 1011 14614 1141,1 '111+414941

4,1140* 0404 541.441 44.11111104 *.k sovto t1,10 er4V4 mweseil444 poittomplatpes solimmist I

, . ; tt , =1./177.17. ,r, Yllrlf 44 11 "2114 es

9C r Table 1(Cont.)

Coal 5: Reduce (Prevent) Suspensions, Expulsions, Dropouts

- --- I. School districtopenness to re-evaluation and changing Judgment based on examination of policies volicies about suspensions and expulciens

School provide creative and diversified additional resources for dealing with behaviors that would normally result in suspensions (outside of the school resources). :lomeone in the school will coordinate the requests for and delivery of these services.

3. Change in teacher and administratorattitudes toward using suspension as a method of reducing disruption

Goal 5: With Regard to Dropout Specifically

4. Increase academic success - School grades - Standard achievement test scores - Self-report grades from SAES Student Questionnaire - Self-report reading ability from SAES Student Ques noire

5. Increase self-esteem SAES Student Questionnaire Positive Self-Concept scale

Coal 6: Effect Institutional Change l. Interagency cooperation Documentation that representativesfrom various age! attend community process meetings and with what freque

2, Comrunity input into planning Documentation that representativesfrom various age' attend community process meetings and with what freque

3. Youth input into planning and implementation

4. Standardization across agenciPR of Arnndards ef conduct

5. Reduce youth alienation and apathy SAES Student Questionnaire Alienation scale

6. Positive communityattitudes towards theAlternative Educatiun Project

3 0 Table 2 Revised Goals and Objectives of the EIC-South Alternative Education Project (Summer, 1982)

Goal 1: To reduce/prevent delinquent acts by 10% fortreatment group.

1. To limit disciplinary referrals for classroom disruption byteachers.

2. Improve/maintain attendance.

3. Decrease out-of-school suspensions in Pleasantville by 10%; maintain levels in the other three districts.

4. Increase positive peer influence.

5. Establish an on-going process/structure todevelop community standards, resource use, problem solving, involving youth and otherseg- ments of the community.

6. Establish/increase opportunities forsummer youth employment.

Goal 2: To contribute to incxaased academic success.

1. Improve/maintain attendance.

2. Improve the study skills of project students.

3. Maintain high level of promoteion to next grade level.

Goal 3: To bring about the adoption of proven alternative educationprograms and policies.

1. To simplify school discipline policies to ensure comprehension by youth and parents.

2. To expand and install concept of peer tutoring.

3. To achieve adoption of one or more units of the Alternative Education Curriculum Guide by the participant districts.

4. To ensure effective involvement of youth.

5. To increase visibility of the New jersey Alternative Education Proj- ect.

6. To increase the TA capacity for the State Board of Education in the area of alternative education and school climate.

-484- 37. Table 3

Number of Students Receiving Each Type of Treatment by the EIC--South

Alternative Education Project -- Broken Down by School and Quarter

School

Vineland Pennsauken Pleasantville Buena

QtrQtr QtrQtr Entire QtrQtr Qtr Qtr Entire Qtr Qtr Qtr QtrEntire Qtr QtrQtr Qtr Entire Treatment 1 2 3 4 year 1 2 3 4 year 1 2 3 4 year 1 2 3 4 year

Alternative ed class 8 11 22 15 23 20 19 30 31 37 15 15 33 26 39 7 8 12 11 14

Community process 0 7 0 0 7 5 0 34 30 31) 0 0 0 0 0 0 0 0 0

Individual counseling 5 9 18 12 25 In 14 13 34 31 37 0 0 18 12 23 8 9 24 22 26

' I Special club 0 0 0 0 0 4 5 2 5 8 0 0 4 1 1 1 1 0 2 3 3

Cross-age tutoring 3 4 2 4 6 0 0 5 6 6 0 0 0 1 1 0 0 0 0 0

Self-contained class 16 16 14 14 18 --

Tutoring 7 7 0 0 0 0 0 0 1 1 0 0 0 7 7 0 0 0 0 0

Any treatment 34 42 51 :1.6

No treatment 0 1 2 1

372 373 Table 4

Average Number of Quarters that EIC-SouthAlternative Education Project Students Spent In EachType of Treatment-- 1981 -82 School Year -- Broken Down bySchool

Entire Project Vineland Penns:.:: Pleasantville Treatment SD N M SD N AI SD SD,

Alternative education 1.68 1.40 159 1.43 1.46 34 2.31 1.37 43 class !..57 1.16 53 1.7.P- 1.56

Community process .47 .82 159 .20 .41 34 1.58 .83 43 ., 0 0 53 0 0 Individual counseling 1.39 1.28 159 1.18 1.14 34 2.10 1.28 43 .57 .72 53 2.07 1.32 Special club .14 .50 159 0 0 34 .37 .82 43 .03 .21 .53 .14 .42 Cross-age tutoring .16 .58 159 .38 .95 34 .26 I .66 43 .02 .14 53 0 0 Self-contained class .36 1.08 159 0 0 .34 1.34 :1.75 43 0 0 53 0 0 Tutoring .09 .29 159 .20 .41 34 .02 1 .15 43 .13 .34 53 0 0

374 Table 5

Representation of Non-Equivalent Control Groups Design Agreed to In.ugust, 1981

Treatment November April Population Starts Pretest Posttest

4 target schools' September Yes Yes 7th graders

4 target schools' No Yes Yes 8th graders

4 4 Control Schools' No Yes Yes 7th graders

4 Control Schools' No Yes Yes 8th graders Table 6

Numbers of Treatment and Untreated Comparison Students at Two Points in the 1981-82 School Year

Total Number Who Number of Had Received Any Comparison Students Treatment

Through Through

School Jan'82 Mar'82 Jan '82 Mar '82

Vineland 18 33 16 1

Pennsauken 38 38 4 5

Pleasantville 19 50 34 3

4 Buena 16 26 10 0

376 Table 7 Zero Order Correlations Between Grades in Individual Courses for th. 1980 - 81 School Year - Vineland

English Math Reading Social Studies

English 1.00 .53 .76 .12 .01

Math 1.00 .06 .09 .10

Reading 1.00 -.26 -.22

Science 1.00 .49

Social Studies 1.00

377 ..

Table 8

Zero Order Correlations Between Background Characteristics and Amount of Various Treatments

Received - EIC South Alternative Education Projects

Grade Parental Days Absent Retentions In Age Race Treatment Level Education 1980 - 81 Students History

-.22 .19 -.21 Tutoring -.10 .04 -.01 (95) (40) (131) (151) (109) (90)

.19 -.00 -.30 School/Comm .12 -.03 .09 (90) (45) (40) Relations (131) (151) (109)

.47 .13 -.22 Community .18 .24 .23 (109), (90) (95) (40) Process (131) (151)

.28' -.10 -.04 0 Counseling .16 .00 (95) (40) (131) (151) (1089)1 (90)

-.02 .05 ,-.26 Special .23 .18 (40) (1049)1 (90) (95) Club (131) (151)

.13 .00 -.05 X-age tutoring -.02 .05 (95) (40) (131) (151) (10079)11 (90)

-.33 .481 .35 .40 Self-contained .28 .34 (109) (90) (95) 1(40) Class (131) (151)

differential degrees ofmissing dat . n's. The n's vary duel to Note: Numbers in parentheses are

376 379 Table 9

Regression of Outcome Measures on Degree of Participation in Treatment Components Operating in All Four EICSchools

Days of .non GPA on a Number of times attendance 1-5 scale suspended

2 2 2 Quarters in b r r r

Schoolcommunity 1.36 .01 143 .07 .02 154 .32 .02 142 relations class

Counseling 1.27 .01 143 .04 -.00 - 154 --.02 .00- 142

Note. Each b is the unstandardized regression coefficient ina separate equation regressing the outcome var:able on the treatment variable. None of these regres sions is significant. Table 10

Regression of Outcome Measures on Degree of Participation' in Treatment Components Operating Only in Vineland and Pennsauken

Days of non- GPA on a Number of times attendance 1-5 scale suspended

,.., 2 2 2 Quarters in b r r r r

Community process -1.49 .01 71 .08 .01 75 -.04 .01 73

Cross-age tutoring -4.20 .04 71 .23* .06 75 .00 .00 73

Note. Each b is the unstandardized regression coefficient in a separate equation regressing the outcome variable on the treatment variable.

* p< .05. Table 11

Regression of Outcome Measures on Degree of Participation in Self-Contained Class Operating Only in Pennsauken

Days ofnon- GPA on a Number of times attendance 1-5 scale suspended

Quarters in b r2 N b r2 N b r2

Self-contained -1.63 .02 39 .20** .19 41 -.04 .03 39 class

. Note. Each b is the unstandardized regression coefficient in a separate equation regressing the outcome variable on the treatment variable.

**p<.01.

382 -493- Table 12

Regression of Outcome Measures on Deg_ce of Participation in Special Clubs Operating Only in Pennsauken, Pleasantville and Buena

Days of non- GPA on a Number of times attendance 1-5 scale suspended

2 2 2 Quarters in b r r r

Clubs .91 .00 111 .04 .00 120 -.48 .01 108 1.8

Note. Each b is the unstandardized regression coefficient in a separate equation regressing the outcome variable on the treatment variable.

383

-494- Table 13

Regression of 1982 Student Questionnaire Scale and Selected Item Scores on Degree of Part- icipation in Various Treatment Activities-- Ney Jersey

Quarters in Alt. Quarters in Com- Quarters in Education Class munitv Process Counseling

raw :1Tdiusted raw adjusted raw adjusted

b N b N b N' b N b N b

Parental emphasis on education -.08 40 -.10 11 -.01 40 -- 20 Attachment to parents -.06* 104 -.05 86 -.22** 47 -.05 86 -.10** 104 -.30 Negative peer influence .00 97 .18** 46 .04 53 .n3 97--

Attachment to school .00 65 -.09 21 -.04 SO --

Belief scale scored . positively -.03 34 -- -.16 21 -.04 34 -- Interpersonal competency .01 34 -- -.33** 9 -.16 21 -.01 34 --

Positive self-cbncept .03 28 -- .02 6 -- -.00 28 -- Self- reported delinquency .01 88 -- ?20** 42 -.03 71 -.03 88 Self-reported drug use .04 88 .34** 45 -.02 62 .06 88 -- Serious delinquency -.01 91 .13* 45 .03 74 .01 91 -- Punishment index -.03 75 -.03 27 -.03 75 --

.rewards index -.07* 75 -.01 54 -.04 27 -- -.10** 75 -.10** 54 Victimization -.04 69 -.09 24 -.06* 69 -.03 54 -.05 -.06 33 -- Invalidity scale -.04 34 8 _ _ School effort -.02 102 -.01 48 -.01 102 -- Practical Rnowledge .11 31 -.25 8 -.05 31 -- Internal- external control .06 34 .:08 8 .02 34 -- Alienation -.01 34 -.07 8 .00 34 -- Self- reported grades -.03 107 -.02 48 -.01 107 -- Reading ability self- rating .11 99 -- .09 45 .04 99-- Days of school cut last four weeks -.06 102 -.01 47 .05 102 -- How often cut one or more classes -.06 108 .47 48 .18 108 -- Educational expectations .14 105 .43 46 -- .06 105 -- Did student work for pay last week? -.01 106 -- .02 48 -.03 106 -- Regular part-time or full-time job -.06 104 -- .07 46 -- -.03 104 -- Suspended from school this term -.02 56 -- .10 19 .:\ -.03 56 --

School nonattendance index -.05 102 -- .28 47 -- .08 102 -- -- Rebellious autonomy .02 32 -- .34* 8 .18 31 .09 32 Involvement -.07** 98-.03** 78 -.15** 47 -.05 78 -.06** 98 -.01 78

Note. Figures in the columns entitled "raw" are unstandardized regression coefficient,: for simple regressions of each outcome variable on each of the Independent variables (i.e. Quar- ters in Alt. Ed. class, Quarters in Community Process, and Quarters in Counseling). Numbers in "adjusted" columns are unstandardized regression coefficients from multiple regressions. where, in addition to the independent variable of interest (i.e. the treatment) control variables were introduced into the equation. The purpose of such a procecI.-!. Is to attempt CO control for the effects of pre-existing characteristics of the students.

The control variables utilized were the following: grade level, race, age, school of attendance, participation in other treatments (i.e. counseling, class, tutoring, clubs, community process). Of these, only those whose contribution to a regression equation was statisticaliy significant (i.e. p .05) with a tolerance of .5 were utilized.

<.05 -495- **p <.01 384 Appendix A: Guiding Principles of the EIC-South Alternative Education Project

Working Theory of Delinquency

Delinquency is a means of expression both in a protesting sense and as a means of getting attention (youths don't know how to get around in the system in a conventional way). Many things contribute to it.

1. Self-esteem--"Delinquent acts are the only thing I can successfully accomplish. This is the only thing I can be proud of."

2. Family relations--no communication, guidance, supervision, and youths raise themselves. Family does not view itself as a means for providing ,urnort for the total growth of the youths (provide food, shelter, and clothes, that's all) and in some cases is not a cohesive unit. Negative contacts with community institutions ---> powerlessness to do anything about things other than food, shelter, and cloth- ing.

3. Boredom--Youths aren't challenged in school. Curricula is irrelevant. Youth activities provided by the community are irrelevant and they are "provided" instead of decided and managed by the youths. This is a great and easy cop-out for the youths. They don't know how to get around in the conventional system and don't find out how to; thus they cop-out. Lack of programs to participate in. Transporta- tion is an obstacle. Parents are usually uncreative in thinking up activities to sustain the family unit.

4. Negative peer and adult contacts--peers and some adults positively responding to delinquency which creates more delinquency.

5. Poverty--no money which leads to delinquent acts which are gratify- ing. Youths don't know other ways of getting money.

6. Drugs and alcohol--to get these youths need money which means commit- ing delinquent acts to get it. Once they have the drugs and alcohol, youths do crazy (delinquent) things. Drugs and alcohol are delinquent, and the youths have immature attitudes toward their use. These contribute to academic failure because youths are spaced out in school and not responsive to any learning which may take place.

7. Depersonalized victims

This appendix is drawn from material prepared by project personnel in completing the PDE plan. It is reproduced here with only very minor editing.

-496- Youths don't have a sense of ownership of public property and do not feel a need for preserving the property

don't know their "neighbors" so hurting them or their property is not personalized

8. Lack of jobs leads to no money which youths want (they don't know how to participate in a conventional sense). Transportation is sometimes an obstacle to employment.

9. Lack of uniform standards of conduct and the application of those standards (vague school rules, without community input).

inconsistency in rules and the consequences for breaking a rule between institutions.

double standards (youths can't smoke but adults can)

youths as consumers of these institutions, do not have input in the decisionmaking which governs them

Youths experience academic failure-because:

1. Youths enter school with low selfesteem

2. Schools are not responsive to individual needs:

a. Youths are unable to fit into the system due to measurement (used as a means of tracking, labeling, not as diagnostic tools) and cultural differences which lead to dissaffected youth not "buying into" the system.

b. Youths don't feel cared for because of teachers, principals and school policies.

3. Students have no input, particularly in curriculum, (thus it is bor ing), discipline, and policy making. Youths end up not caring

4. Teacher attitudes: (hopelessness, cynicism, etc.)

a. Teachers don't expect youths to achieve which leads to youths not being challenged. In many cases, the potential is there, but no motivation to achieve.

b. Lack of communication between teachers and adminstration directly affects teacher ownership (they do not feel they have ownership).

5. Parental contacts with school are for negative reasons and "they don't want to hear it Parents aren't supported by the school or encouraged to support their children.

-497- 386 6. Youths not feeling slit. by teachers and parents leads to teach ers not feeling support,. Principals, superintendents, board, com munity, and student's pa leads to principals not feeling sup ported by superiors, boart, munity, teachers (depending on the school), and parents (deper on the prinicpal) leads to superin tendents not feeling suppol by: board, community leads to boards not feeling supported by community (superiors) leads to community not getting involved.

The above problems die a slow and painful death and there is no resolution.

Principals

a. Are responsible for insuring that youths come to school.

b. Carry out the tracking, and labeling procedures because they impact scheduling.

c. Approve the curriculum/development implementation and can change the whole curriculum.

d. Strongly influence budgets which don't end up buying needed books, sponsoring field trips, and providing custodial care for afterschool activities.

e. Are responsible for teacher placement. ("Good ones" end up in "bad" classes); consequently, teachers don't want to perform well because there are limited rewards, if any.

f. Carry out disciplinary responses sometimes inconsistently due to others' influences (board, superintendent, politicians).

g. Aie responsible for insuring that youths come to school, but the various responsibilities deem that they leave coming to school up to youths and parents.

h. Designate responsibility to the guidance departments and the local court (15 dollar fines for not sending youths to school). Guidance department are busy with "successful" youths and pass the problems on to truant officers.

7. Boards of Education superintendents (and community) are not planning educational goals and plans, budgeting for the whole district, and the small minority of "bad youths" get lost in the plans.

8. Classified students

Youths who are legally classified present another significant prob lem in the theory of student academic failures. These youths are very often scapegoats for an education system which uses labeling and tracking techniques to rid itself of these students who have not conformed to the schools social and educational "norms."

498 387 Teachers are not trained to adequately deal with the multifaceted handicaps of the classified student. Insufficient teacher train ing and limited experiences with this type of student lends itself to low expectations from teachers and no motivation on the students' part.

Most frequently used levels of classification for the special education student- -neurologically impaired perceptually impaired emotionally disturbed socially maladjusted

Theory: Violence and Vandalism

1. Selfesteem--"Delinquent acts are the only thing I can successfully accomplish. This is the only thing I can be proud of."

2. Boredom--Youths aren't challenged in school. Curricula is irrelevant. Youth activities provided by the community are irrelevant and they are "provided" insteac of decided and managed by the youths. This is a great and easy copout for the youths. They don't know how to get around in the conventional system and don't find out how to; thus-they copout. Lack of programs to participate in. Transportation obstacles. Parents not being creative in thinking up things to do or making youths responsible in the home.

3. Drugs and alcohol--to get these-youths need money which leads to del inquent acts to get it. Once they have the drugs and alcohol, they do crazy (I1elinquent) things. Drugs and alcohol are delinquent, and the youthikhave immature attitudes toward their use. These contri bute to academic failure because youths are spaced out in school.

4. Depersonalized victims

Youths do not have a sense of ownership of public property and no stake in preserving the property

5. Negative peer and adult contacts--peers and some adults positively respond to delinquency which leads to more delinquenct acts.

6. Lack of uniform standards of conduct and the application of those standards (vague school rules, without community input). Inconsis tency in rules and the response to breaking a rule between institu tions and double standards (youths can't smoke but adults can). Youths, as consumers of these institutions, do not have input into the rules which govern them.

7. Gap not bridged: transferring the idea of private ownership to public property so that youths feel as if they own it too. This leads to a

-499- need for youths to become involved in and responsible for ownership as a constituency of the community in partnership with others, specifically peer and adults.

8. Rage--"I'm not worth anything!" These are often expressions of anger at not being recognized. These expressions may lead to attacks on teachers (and others) who become personalized victims of the rage. Many demeaning things have accumulated over time and culminate with the expression of rage, hostility.

Theory: Absenteeism

1. School becomes a negative place to be.

a. The differences between youths (skin coloi, dress & other) are apparent to .youths/teachers. and some youths get a lot of negative responses.

b. Language barriers between: Hispanics & English, Black dialect & English.

c. Problem with curriculum and achievement leads to poor testing skills which leads to being "labeled."

d. Parents transfer their negative feelings about-the school- (see- academic failure, No. 5) to their youths.

e. Boredom--Youths are not challenged in school. Curricula is irrelevant. Youth activities provided by the community are irrelevant and they are "provided" instead of decided and managed by the youths. This is a great and easy cop-out for the youths. They don't know how to get around in the conventional system and don't find out how to; thus they cop-out. Lack of programs to participate in. Transportation obstacles. Parents not being creative in thinking up things to do.

f. Students have no input, particularly in curriculum (thus it is boring), discipline, and policy making which leads to youths not caring.

g. Teacher attitudes: (hopelessness, cynicism, etc.)

1) Teachers do not expect youths to achieve which leads to youths not being challenged and boredom becomes commonplace!

2) Lack of communication between teachers and administration directly affects teacher ownership (they don't feel they have ownership too).

2. Due to family structure (responsibility for babysitting while parents work), youths have to work (migrant workers). Migration of the family interferes with going to school. Education is not valued.

3. Social Success

5007 389 a. Youths who go to school and are not able to communicate with stu dents, teachers, parents (no interaction between them) but who want social success, often find it with others who aren't in school.

b. Can't handle all that's thrown on them (not adults yet). Moon lighting in jobs and can't be succesful in both school and work. The job wins out (its more rewarding). They want immediate grati fication so they get jobs and school is not an immediate gratifi cation.

Theory: Suspensions, Expulsions, Dropouts:

Expulsions don't happen very often because it is a difficult process (due process requirements). Expulsions only happen in very clearcut and serious ci.rcumstances. Intervening at an earlier stage prevents expul sions.

Suspensions occur because they are a very easy way of coping (getting rid of) with disruptive students. Teachers pressure administrators to suspend disruptive students. They don't know what to do with youths. Suspensions, as a consequence (punishment), are based on the assumption that the youths want to be in school in the first place. In reality, being suspended can be a reward to some youths.

Guidance counselors work loads don't allow the time to deliver counsel ing services that disruptive youths need.

Dropouts (pushouts?)

Youths drop out because:

1. They are not academically successful

2. They have low selfesteem and no sense of worthiness

3. They want to earn money

4. Of boredom

5. Of family relatives, who appear to be "making it."

6. Of drugs and alcohol/problems.

OR disruptive youths are counseled out the door. Which means that some students are not encouraged to perform at any level.

They dont' fit in any program and therefore, in subtle ways; are pushed out the door.

501 Theory: Effect Institutional Change In view of perceptions of fact and the stated problem definition, the following hypotheses have been drawn:

I. Academic success (positive experiences at various levels of ability), attendance rate, suspension, expulsion, dropouts, truancy are not subject to impact by schools alone.

2. The school system in any given community, is so reflective of the attitudinal make-up of that community that the schools' effectiveness in implementing change in areas such as violence, vandalism, suspensions and expulsions is limited to remediation at best, without a change in the community attitudes that the school system reflects.

3. Schools, municipal governments, juvenile service providers, and independent service groups are generally operating within their own spheres to alleviate causal factors such as permissiveness in society, lack of respect for authority and property, boredom, and drug and alcohol abuse. This is primarily dealt with on a case by case basis; this approach results in a haphazard individual remediation program, with little or no long range preventative effect.

4. Youth alienation and sense of powerlessness on the part of the community; what we commonly refer to as "apathy" -in terms. of vot--- _ ers, community groups, PTA and school councils, etc., is more likely a result of the sense of powerlesness than a demonstration of the attitude of the community in general.

5. This "apathy" is a direct result of a feeling of alienation from the very institutions that have most direct impact on the community: schools, local government, social service providers. The implementation of a variety of programmatic solutions has created a confusing set of standards of behavior for children, parents, and the community. These standards are not reflective of what the community desires, although the community feels powerless to change them.

6. A problem-solving process must be designed and implemented which will allow the community direct input into analyzing the causes of juvenile problems in schools, home and community. These causes must be sufficiently defined to allow for programmatic application and solution; the process must involve all agencies charged with effecting the solutions along with the community; the solutions must be a cooperative effort so that schools, government, service providers, and the community are not working at cross-purposes, as antagonists, or in competition with each other.

a) The process in each community must be credible, and subject to an intensive public relations campaign.

b) All resources, public and private, should be brought to bear upon the causes through systematic youth, parent, community and

391 educational programs, vocational training and placement, counseling, and individual service programs.

7. At a minimum, the process should result in:

a) a standard of conduct that the community deems appropriate in its schools and throughout the community;

b) widespread dissemination of that standard and its community basis, together with a rewards/punishment system;

c) fair and uniform application by schools, police courts, and other agencies charr,ed with these duties by the community;

d) positive attitude development throughoUt the entire school curriculum; and

e) on-going assessment by the community of its standards.

8. Community Process

A. Advisory councils are not necessarily what the project is hoping to formulate. Advisory councils are usually composed of:

1) Most times the same people who always get involved.

2) This means only 35-40% of any given community is directly involved.

B. Expectations of community process

-Create climate for change (working with formal/informal leadership)

-Don't ask for big committee or time

-Ask for suggestions/recommendations

-Hopefully they will start thinking process for what the community wants to happen

-Inter-agency cooperation

-Want change to occur from the inside without threatening anyone's territory

-Communication extremely important

-Agree with the "Rock-Throwers"

-Ask them to do something about their complaints

C. Kinds of people/organizations to invite to meetings

-503- 392 -Schools Municipal Governments (Elected and appointed representatives) -Church Youth ("good" and "bad") J -Business/Industry (Merchants Asso., Chamber of Commerce) -Kiwanis/Lions/Jay Cees PTO/PTA's Social service delivers paid professionals delivering services for youth -Ordinary People Police Courts (municipal, courts) Senior Citizens Families -Social/Recreational Juvenile. Conference Committee (citizens, meetings, as needed, to handle 1st infraction for youths: punishment, etc., to be erased from youth's record) Knights of Columbus Firefighters (volunteer) JVS

Jewish Vocational Services Alternative Education Project: Evaluation Report

M. S. Cook

Abstract.

TheJewish Vocational Services AlternativeEducationProject developed and implemented three interventions. The Milwaukee Youth Employment Center (MYEC) counseled dropout youth and attempted to place them in employment. The Return Center, operated in cooperation with the Milwaukee Public schools,assessed and referred to alternativeeducationalprogramming youth who were contemplating dropping out, or who had already dropped out and wished to re-enroll in formal education. The Job Score class was a regular high school course developed by MYEC staff to teach employment skills to youth at risk for dropout.

An experimental evaluation of the MYEC program indicates that it was not successful in increasing the employment opportunities of its clients. Subsidiary analyses suggest that in general, clients did notreceive many services, although remedial education provided through the project probably had positive effects. Evaluations of two project components- -the Return Center and Job Score classes--were not completed due to the early termination of the project.

In contrast to some recent reports, regression analyses suggest that employment causes little,if any, pychological harm to youth who have already dropped outof school. Returning such youth to school may result in some negative outcomes.

The Job Score curriculum may be appropriate for use and evaluation byothers interestedin providing employment counseling to youth. It is recommended that prevention projects be allowed to follow a Program Development Evaluation cycle that provides more time for program refinement and testing than was available to this project.

Overview In thetwo year historyof the alternative education project, three The Jewish Vocational Services distinct interventions were devel- Alternative Educational Project was oped and implemented by JVS: an attempt to provide alternative (a) theMilwaukee Youth Employment services to students who had dropped Center, a job counseling and place- out of the Milwaukee Public Schools, ment center for dropout youth; or were atrisk of droppingout. (b) the Job Score class, an employ- Jewish Vocational Services (JVS) is ment counseling curriculum for use a large agency with a 40-year his- in the regular public school curri- tory of operation in Milwaukee, and culum; (c) and the Return Center, an it has particular expertise in assessment and referralcenter for employment problems faced by special youth seriously at risk of dropout. populations such as immigrants, youth, and the handicapped.

-505- 394 Shownwslitau-slLakiessAits LESUSIILAIL12111

Iko tollowitrg4* me abbrovinted PS staff identified as problem torowoo Oro 0111 prOject'a to be addressed youth unemployment, 44004$44044 ,V44**03e. 44*60410. youth es welfare. delinquency, lack 000is. 00404044w*. more enwtemm Of teleran curricula. in the Mit- eoitio 4.0004.400 44004 %Mfrs soy osis's., Fnblic Schools, and lack of a lire 0004 41* teneoeti end method for youth seeking educational 04.40,4sitio Auts*. ***oily from service to be referred to spprupri- 01,0 eso4**4*. 41410 programming. Consequently, their #0410 were to: 13004: I. Seduce the uneeploynent rate #14 04; tbet dam dropouts. wow* Moe fivet**0004*tin Aims- A4***0 Liu meltfteetnees inch st per- 2. Serrealut the proportion of 44004 .10404 4(togeemol teesrel/. the drepoots es welfare. mitevossone.ii*e moo ot *ad Ihd 01WNW** ifotOn drer on* et oohs*, 3. Decrease the incidence of dol- *woof*Joey ne 404 joicet*o tradi- ing...et acts neon; dropouts. tions** *440&osimg 4410woot to 4044 444404.441 Owy *44404011 to 4. tsetitutioealise the MYEC job 0044,o 44*** nor4.**4eme by wow comeseliag model within the Milwau- .40f 411101014miwt tot, 644, MO/. kee Public School Syetem by incorpo- For 444 ** oppertoai4ies for ratisgreties a competeery -based, work -re - wooktopown*4 tbe *woo** et helitiog a lato4 curriculum into the existing 0100 owt Wimee 41 provides program. **Lidwo*** Adiamoomry tcf. 4breammitiorg4 Otter 'WOO soy 5. Provide a referral and assess- Me rcovided emeetwieveer educe- meet center where dropouts or poten- thomel ompotimmoei ow tbet Owl sae tial dropouts could receive guidance and tom. or tm arse mod oleo* mot eheni appropriate education ow. 3L4001toottel* Ftr each Weer choice. rmotb,, *vow* is * dittereot **oot efte remmemt mot **Witt*, for **s- ailLUZIS *oertsO 4000 provemtteg -004Mintienkor to COPAN O), Objectives for the JV$ slterna- live education project include: mmomS MismoreticelOseie for if 'armies MYSC clients' "job dm* e lime 0* tioeffective reedioess" skillsthe skills nacos- 404 iiNtdr m eisel imodemoste 004441 0400404 440 tro iy MOO. *DP eery to find, secure, and keep a ends eso Miemolo4 ft mo oesiety,noel job. *moo me food for otweferoily. 'bey rMallowhod to etleificsat othu. 2. Increasing WIC clients' or*.444*Ohiml 1004 100 oaring role work nelited academic skills. frafFitexaleounestiore mod ter b autono - erruminhow441 del oeniewento tlr to 3. Deere/Wog rebellious false independence flehrffol emmlopmemt, and sr.-feeling of del is- and f net seeding anyone or any - 10 flIMI*4c.4* 04Wageleig 1011*Ffs 0164.

4011.. 395 JVS

4. Increase feelings of personal JVS had a history of poor rela- power (internal control). tions with local CBOs,social service agencies, and the Milwaukee 5. Provide youth with income Public Schools. All of these agen- through employment. cies were eventually required to serve as referral sources and inter- 6. Decreaseyouth's time spent vention sites. Referrals therefore nonproductively. were slow to develop,and negotiations for cooperative work on parti- 7. Returndropout or potential cularinterventionswerelong and dropout youth to more adequate arduous. alternative education programming. An earlyerror in staffing the 8. Develop andtesta work-re- project occcurred whenJVS hired a lated, competency-based high school non-JVS person to directthe pro- curriculum. ject. This did notturn out well becausethe project director never History and Organizational Context felt himself to be part of the of the Project organization, or particularly accountable to it. The project The JVS Alternative Education director, being an outsider, was not Project was plagued with implementa- invested in the project as a whole, tion problems from its very incep- but only in those parts that he felt tion. Many of these will be he was hired specifically to attend detailed below in conjunction with a to. Thus, for example, the school description of the particular inter- related parts-of -th-e- Froje-ct (th-at ventions. Presented here are gen- OJJDP was especially interested in) eral problems that affected the pro- were given short-shrift because the ject as a whole. project director felt that their management was beyond his purview. Just as the project was funded in December of1980,JVS underwent a JVSproposal writers, too, may severe organizational shake-up. The havepromised more thancouldbe organization was undergoing federal, expected of the project. For exam- state, and local audits as a result ple, in the original grant proposal, of allegedembezzlement of funds. JVS promised to serve 750-800 youth At the same time, federal cut-backs in the MYEC job counseling program in social service funding had during the first year--it actually resulted in layoffs of JVS union served fewer than 350, and only 79 staff. A new executive director was of these received the entire treat- hired to overseethe stabilization ment. Similarly, the proposal said of the organization during its fis- it would serve 40-100 youth in the cal crisis. Thusthe project was ReturnCenter during the initial born in an organizational climate of year. As it turned out, the Return crisis and distrust. Center did not begin until the second year of the project. The layoffs in staff continually subjected the project to union JVS and the Sponsor demands for priority in hiring, regardless of qualifications. Line An additional organizational con- staff still on the job did not trust flict lay in JVS's relationship with theorganization, andresorted to its federal sponsors. JVS adminis- strikes or strike threats to press trators failed to quickly or seri- their demands.

-507- 396 JVS ously dealwiththeconcernsthat The Interventions OJJDP had on budget, management, and program implementation. Two events Milwaukee Youth Employment Center that ultimately may have contributed The Milwaukee Youth Employment Cen- to a decision notto continuethe ter (MYEC) was the core intervention project for the third year will of the JVS project. MYEC was a "job illustrate the point. First, in readiness" training program, modeled August, 1982, the project was still afterthe Jobs For Youth,Boston, unaware ofits continuation status program. Clientsweretrained in for the thirdproject year. JVS the skills necessary to find, management chose to address this secure, and keep employment, and ambiguity by not sending anyone to were referred to jobs identified by the national conference--an OJJDP the project. Youth ,served were 16- requirement. Inexplicably nobody at and 17-year-old,unemployed, offi- OJJDP orat Johns Hopkins was even cial high school dropouts from the notified of the JVS decision not to greater Milwaukee area. appear at the conference. They simply did not show up. Hurried phone The MYECprogram consisted of calls, and the reminder that appear- three units: (a) a counseling unit ance at the conference was an that trained youth in the job readi- already-incurred contractual obliga- ness skills; (b) an Educational Ser- tion, produced a JVS delegation a vices unit that provided competency- day late. based individualized instruction to thoseclientsneeding or desiring Second, after the JVS staff it; and (c) anEmployer Services appeared, they were unable to suffi- unitthat locatedjobs available, ciently engagein program develop- and acted as the employers' advocate ment efforts--the purpose of the within the program. conference--because they were still trying to prepare a budget for the The MYEC project went through two third year proposal. Despite a phases: March to December 1981, and clear requirement that all projects January 1982 to the end of the pro- submit a third year budget that was ject in November, 1982. In the 30% smaller than the previous year, first phase of the project, job the .JVSthird-year proposed budget readiness skills were identified was 20% larger. Project staff spent (based largely on the Jobs For a great deal of the time at the con- Youth, Boston, model), and youth ference deciding whom they were were assigned to individual counse- going to lay off when they got back. lors for training in the skills. In Such thoughts are not conducive to the second phase of the the project, program development efforts. moredetailedskills were identified,criteria forsuccessful mas- These JVS organizational problems tery were outlined, and a structured subjected the project to continuing series of workshops were begun which upheaval over interpersonal rela- all clients had to attend. tions, interorganizational relations, labor relations, and bugetary There were three "core" job read- management. Little organizational iness skills that had to be mastered effort remained to focus on the before one was referred to a job: interventions themselves. jobapplication,job interviewing, and job retention. In addition to these core skills, clients were also judged forjob readiness by their

-508- 397 JVS

counselor based on their appearance, responsibility was ensuring that punctuality, dependability, and employers were satisfied with the attitude. quality of MYEC referrals. A criti- calpart of the program (particu- In phase one of the project, larlyduring a recession) was the clients were assigned to an indivi- ability of Employer Services to have dual counselor at intake, who a.job bank available from which MYEC trained them in the skills, and youth could choose appropriate referredthem to a jobwhen the employment. While the number of counselor felt that the client was available positions fluctuated dur- job ready. In phase two, the skills ing the time the project operated, were taught in a structured workshop there wereusuallyjobs available format, and attendance at the work- forthe youth to be referred to. shops was generally regarded as suf- The more common problem was the ficient evidence ofjob readiness. quality of the available jobs. Many Each client was also assigned a were parttime, and virtuallyall counselor. An individual could were very low level entry jobs such retake a workshop, or receive indi- as counter clerks, dishwashers, res- vidual help from the counselor if it (.taurant workers, and maintenance was apparent that he or she had not personnel. mastered a requisite skill. Youth who were referred to jobs, In both phases clients also but did not land them, were continu- received general counseling "as ally referred until they found needed." Also, those youth who employment or droppedout of the decided to upgrade theiracademic program; Youth who -1-inde-d -a- job skills (either by their own wish, or were followed up by their counselors at the suggestion of the counselor) atirregular intervals to see how werereferred toEducational Ser- they were doing (although this did vices. not happen as often as project plans called for). Youth havingdiffi- Educational Services provided culty on the job were encouraged to competency-based individualized comeback to MYEC for additional instruction in basic skills (reading counseling or training. Youths who and math), advanced instruction in lost jobsthey had been placed in, G.E.D. subjects, and functional eitherbecause they quit orwere vocational training includingtyp- laid off,could be re-referred if ing, cashiering, and driver's educa- the circumstances warranted it. tion. EducationalServices was an Employer services representatives optional part ofthe MYEC program. checked with the employers of youth Youth who did not need further edu- who lost jobs in order to assess the cational assistance did not receive difficulty the youth had with its services. employment. This often led to additional counseling for the youth. The Employer Services unitwas For example,a youth laid off for responsible for developing jobs tardiness would not be referred to (i.e., locating job openings and another job until some judgment had creating good will among employers) been made that tardiness would not so that positions to whichMYEC again be a problem. clients could be referred would be available. As alluded to earlier,the program began with an emphasis on one- An important element of their

-509- 398 JVS

to-cne counseling provided to the where a post-test job application clients by theirassignedcounse- had to be completed with 80% accu- lors. In the fall of 1981, the pro- racy in order to be certified), the ject director became dissatisfied workshop format was an improvement with this arrangement, believing over the previous procedures because that the counseling provided to it insured that every clientwas clients varied too greatly depending exposed to the fundamentals of each upon the counselor. Inaddition, skill believed to be necessary for the EmployerServicesdivision of employment success. the MYEC program complained about the"job readiness" of the gradu- The switch to the workshop format ates. Employer Services was receiv- also improved the program for a sec- ing complaints from employers about ond reason: it forced project per- the quality of referralsfromthe sonnel to commit topaper exactly program,and was frustrated by the the skills and information necessary number of clients who accepted jobs to meet the criterion of "job and then quickly quit them. ready." In order to write a workshop that would be presented to all In order to insure that all clients, and could be facilitated by clients received training in the any of the counselors, project staff three identified job readiness were forced to come up with a set of skills, and to gain some quality curriculum materials that specified control over program delivery, a theinformation to betaught, and workshop format for training was the methods and techniques to be developed and implemented in January utilized. The technical assistance of 1982. This change initiated contractor (Polaris) provided train- phasetwoof the project. In the ing in group facilitation skills to workshop format,MYEC clients were the staff specifically for the pur- still assigned to a personal counse- pose of strengthening the workshops. lor, but that personal counselor was Such training does not insure that no longer solely responsible for the staff acquired the necessary providing the job readiness train- skills, butit at least forced the ing, and certifyingthe client as project to engineer into the work- job ready. Instead, each client shop specificactivities aimed.at went through a structured series of the job readiness criterion, and workshops. Workshops were held at givesus more confidencethat all set times eachweek, andclients clients received instruction in the progressed through the workshops in job readiness skills. serial fashion. MYEC clients first attended an orientationworkshop, At the same time thatthe work- then one on completing job applica- shop format was developed and imple- tions, followed by a workshopon mented, Educational Services was interviewing, and one on job reten- more strongly integratedintothe tion. program. In the first year, clients were referred toEducational Ser- Satisfactory completion of the vices at the discretion of their four workshops resulted in certifi- counselor. In phase two MYEC cation asjob ready. Although de- clients were automatically referred termination of "satisfactory perfor- to Educational Services if they mance" was still largely a judgment showed low reading or math skills. on the part of the counselors con- A written recommendation of instruc- ducting the workshops (exceptfor tional objectives was prepared and the job application skills workshop discussed with the client. Referred

510- 399 JVS clients were not required to receive often to avoid rather than confront training from Educational Services, problems. Three events involving but it was emphasized to them that the evaluation illustrate this avoi- their chances of success in the dance of confrontation in uncomfor- employment market were remote with- table situations. The project had out basic skills remediation. promised to secure court contact data on treatment and control The switch to a structured work- youths. Despite the obvious inter- shop format, and to more formal uti- est of both the evaluators and OJJDP lization of Educational Services in these important data in a delin- changed the MYEC program from one of quency prevention project, and des- providing ad hoc one-on-one counsel- pite assistance in preparing to per- ing in vaguely defined job readiness form this task, the court data were skills, to one in which all clients not collected. Although the request were provided with training in much was made in September, 1981, 15 more tightly defined skills.We can months before the project was termi- therefore be much more confident nated, the project director did not about thejob readinessskillsof negotiate with the clerk of court to the 1982 graduates of the MYEC pro- get the required data. gramthan wecan about the 1981 graduates. As a second example, the project director did not inform the evalua- Problems in implementing MYEC. tion staff of an upcoming strike in The MYEC program suffered from man- the spring of 1982. The result was agement difficulties and strained that a follow-up of clients ground labor relations from its beginning. to an iaexpliCable (ta ES) Hart,- and It was a large project. The project we were in the dark asto how to director was responsible for hiring provideassistance. We speculated staff and overseeing the design and that nomention was made of the implementation of three complex labor disputes in order to avoid the interventions. The projecthad a appearance of difficulties. full-time staff of about 20 persons and involved negotiations with the A third example involved the evaluator, OJJDP, the Milwaukee development of a strategy to pay Public Schools, the juvenile court, experimental control subjects for local CBOs, upper level JVS manage- their participation in a MYEC fol- ment, and the JVS employees' union. low-up. Theprojectdirector had The newly hired project director had led evaluation staff to believe that little experience in management, and the system for payment was complete his personal inclinations ran toward and approved. It was not until the providing direct services to evaluation was over a month overdue clients, not to running a large pro- in starting that the director ject. The project director did not informed evaluation staffthat, in have the management experience,or fact, he had not gotten the agree- the exposure to program development ment of his comptroller to pay con- to adequately oversee this complex trol subjects. task. The project director adopted a laissez-faire management style, A poor relationship between JVS which was in keeping with his pre- and its employees' union made man- ference for service rather than management of the projectdifficult. agement. The director, who had not been previously associated with JVS was The project director appeared

-511- 400 JVS immediately faced with the task of Staff of the Employer Services hiring staff in an atmosphere of and Educational Services unit appear distrust between JVS management and to havehad morepreparation for its employees. In the two years of their work. Most of the Employer project operation, there was one Servicesstaffhad backgrounds in union strike, and a second was nar- business, orhad previously worked rowly averted. as job developers on other JVS projects. All Educational services Union regulationsseverely con- teachers were certified teachers: strained the director's efforts to the requirement that they be certi- select qualified staff. Immediately fied allowed the supervisor of Edu- priorto project start-up,JVS had cational Services to hireoutside been forced to lay off many ofits the JVS union. employees. When the Alternative Education Project began, the project High turnover and strikes by JVS director was forced tohire staff union (which included nearly all from a pool of union employees avai- MYEC employees) created other prob- lable for recall. He had to hire lems for MYEC. By the end ofthe from thelist basedon seniority, project,in November, 1982,11 per- with little attention to qualifica- sons had terminated out of the26 tions. differentpersons whoever worked full-time on the project. Two The problem of forced hiring extensive staff re-organizations based on union seniority resulted in were carried out by the project particularly acute problems in director in the space of a year and, staffing MYEC's counseling unit. a half. The average length of stay The union felt that virtually anyone on the two-yearprojectwas 13.6 was qualified to be a counselor, and months (andthis does notinclude pressed the issue. The result was several JVS personnel who spent con- that the backgrounds of the counse- siderable time on the project, but lors who were charged withimple- were not paid out of grant funds). menting MYEC's primary intervention Project staff did not get along well were in diverse fields, but not in with one another. As already men- counseling. Two of the persons who tioned, there was one strike in served as counselors did not attend October of1981,and a second was college; others had B.A.'s in avoidedin April 1982ina "final sociology, business, and physical hour"settlement: MYEC staff had education. Another counselor had a fully expected the employees to B.A. ineducation, but had never strike. MYEC sought help in staff taught, and her prior work experi- relations from the technical assis- ence was as a buyer for department tance contractor (who suggested that stores. The original supervisor of even more assistance inthis area the counseling unit did hold a Mas- might be helpful). ters of Education witha major in Counseling, but she was replaced by Poor staff relations were not a counselor who held a B.A. in Cri- confined to problems betweenline minal Justice. Although degrees do and supervisory staff: the project not guarantee successful counseling, director and his supervisors in JVS the lack ofcounseling training in didnot work well together. The the MYEC counseling staff undermines project director was hired from out- confidence that sound counseling side the organization. He had pre- practices were implemented. viously worked in a local CB O. The director disliked the JVS organiza-

-512- 4 0.1 JVS

tion, and never considered himself a tle training or experienceto work part of it; he felt he was hired to as counselors,(d) poor JVS manage run MYEC, and that MYEC should be an mentlabor relations put the project independent entity from the organi in continual conflict with its own zation at large. In turn,JVS did staff, - even to the point of a nottrust the project director to strike, and (e) the project was manage the project,and placed him begun without first establishing an under theimmediate supervision of extensive client referral network. the Director of Youth Services. At Thus the project careened from one one point during the summer we were crisis situation to the next, with

told thatthe project director had . little time for planning, or been or was about to be dismissed; thoughtful program development. shortly thereafter we learned that the project director had filed suit The Job Score Class againstJVS fordiscrimination in pay. The project director was still The Job ScoreClasses werean holding thatjobin November when offshoot of the MYEC job counseling the project ended. 1,program, designedand directed by the supervisor of MYEC's educational Another problem forthe project services unit. They were an attempt was the number of clients being to provide to students still in served by the project. The pro school the job readinesstraining iect's proposal promised to serve a available to dropout youth referred large number of persons, and the to MYEC. sponsor pressed the project to meet its client load target. The program One of the prime motivations was begun before developing strong behind the instigation of the referral links to local CBO's, classes was the desire by OJJDP to Juvenile Court, Milwaukee Public see the JVS project more involved in Schools, or the county Department of "alternative education"--the basis Social Services. The result was of the federal initiative. OJJDP that despite a fulltime staff aver did not regard MYEC,which wasan aging over 17 persons,the program alternative toformal schooling as served fewerthan 350 clients its "alternative education."The Office first year of operation, and most of insisted that the JVS project these came in a burst at the end of develop programs to serve youth that the year; by December, 1981, only 79 had not already dropped out of youth had completed the entire MYEC school, and theJob Score Classes job readiness program. were an attempt to meet this requirement. In summary, MYEC's implementation difficulties fall into four catego A tentative curriculum was ries: (a) the project director had designed in the summer and fall of littlemanagement experience, and 1981, and a contract between JVS and hispersonalmanagement stylewas MPS providing for the implementation incompatible with the demands ofa of the class in six area high large, multifacited project, (b) the schools was signed in October, 1981. project director did not get along Classes actually began infour of with hissupervisors,anddid not the high schools in January,1982. consider himself a real part of the Four more classes were held in the JVSorganization, (c) union hiring Fall of 1982. demands resulted in placing workers in the counseling unit who had lit-

-513 402 JVS

The Job Score class was a regular The Return Center high school class that could be taken for credit toward the high The Return Center was the focus school diploma in the Milwaukee of the originalgrantproposal to Public Schools. The curriculum OJJDP. It was envisioned as one of revolved around the MYEC job readi- a number ofcentersthat would be ness skills. Topics included how to set upstatewide as a cooperative fill cuta job application, how to venture of all Wisconsin agencies interview for a job, what to expect responsible for providing social on the job, how to locate jobs, and services to youth. how to get along with employers and fellow employees. Also included in These service delivery centers the curriculum wasinstruction in were supposed to ,be centralized basic life survivalskills such as diagnostic, referral, -and record- how to balance a checkbook, shop for keeping centers. Their purpose was insurance, and figure interest to serve as sites where youth with charges. problems could be assessed and then referred to an appropriate service Several weeksinto the semester delivery provider in the area. Cen- (optimally, around six weeks) those tralized record-keeping would pro- students thatwerejudged by the vide a method of follow-up and eval- class teacher to be "job ready" uation, and would allow agencies to (using the same criteria as clients better plan for the needs of youth. in theMYECprogram, above)were JVS would focus on developing a referred to MYEC for job referral. Return Center that would serve the Those students that MYEC also judged needs of Milwaukee area youth who to be job ready were then referred had dropped out of school, or who to parttimejobs. Studentsthat were contemplating dropping out, but landedjobs worked halftime, and who sought some form ofstructured attended school half time. They education. The Return Center would received academic credit toward assess their skills and interests, their diploma for the time they and thenrefer them to agencies spent working. The program design (including the Milwaukee Public called for students to maintain School system) that provided appro- their academic standing,butit is priate intervention. not known if this requirement was adhered to. Students who did not It is perhaps symptomatic of the find jobs continued in the regular difficulties JVS had in implementing curriculum. its project that, despite the intent of the originalgrant proposal to The curriculum package as devel- focus on the Return Center concept, oped by the supervisor of Education the JVS-sponsored Return Center did Services appears promising. She not begin operation until January of wrote detailed day-by-day lesson 1982--more than a yearafter the plans with clearly specified project was funded. instructional objectives, and included methods of assessing suc- Three major problems contributed cess. All of the MYEC job readiness to theslow implementation of the skills were covered and reinforced. Return Center: The curriculum appears to be highly relevant to students, and was favo- 1. The project director believed rably viewed by thefourteachers that he had been hired primarily to and principals involved in the ini- oversee the development and manage- tial application of the course.

-514-- 403 JVS ment of the MYEC job counseling tions. JVS feared thatMPS could component of the program. It was not be held accountable for the hisbeliefthat the Return Center funds, and would insist on control- was primarily the responsibility of ling the design and staffing ofthe upper level JVS staff, and indeed, Center. MPS suspected that JVS atthe March 1981 planning confer- would really like to keep the funds ence in Washington, D.C. a JVS staff "in- house" and would be reluctant to person not funded by theproject refer "JVS" youth to the Center, and accepted responsibility for negoti- back to MPS. Since neither organi- ating the Return Center contract zation trusted the other,negotia- with MPS. At the conference the tions to set up a Return Center, project director was assigned res- staffed by MPS personnel, but paid ponsibility for developing contacts for outof JVS/OJJDP funds, became with MPS in-school personnel so that long and extremely complicated. referral sites would be available to accept clients from the Return Cen- 3. Thethirdreasonfor slow ter. implementation ofthe Return Center wasthatthe negotiations forthe 1,1 The project director had little establishment ofthe Return Center experience in the negotiation of were tiedto thosefor developing contractsand agreements, andwas and implementing MYEC-sponsored Job not happy with working with the Mil- Score classes in six Milwaukee waukee Public Schools. In lact, the public schools--for which curriculum project director (and MYEC staff in did not exist in 1981. Educational general) were reluctant to refer Services personnel within MYEC were clients in the MYEC program to MPS. assigned respOnsibiliiifoF deiign- Also, the implementation of the MYEC ing the curriculum, and did so, but component itself was not progressing in the meantime,negotiations for rapidly, largely due to labor diffi- the Return Center could progress no culties. The result of all of these more rapidly than those for the Job forces was that the project director Score classes (and vice-versa). assigned top priority for his attention tothe MYEC job training pro- Theresult of the three major gram, and put the Return Center on obstacles to the implementation of the back burner. Negotiations for the Return Center was that a draft the establishment of the center were of the contract with MPS wasnot left to JVS staff outside of MYEC. completed until September, 1981. By The project director felt that if the time a site was found and staff JVS upper management thought the were assigned, it was January, 1982. Return Center was so important, then theycould oversee it. In short, To summarize the primary causes the project director feltthat he of the slow implementationof the was director of the MYEC portion of Return Center: (a) the JVS project theproject, and that theReturn directorconcentrated hisenergies Center was an additional (relatively on developing the MYEC component unimportant) duty that took time while paying little attention to the away from what he saw as his central Return Center (and felt justified in task. so doing since other JVS personel took responsibility for negotiating 2. The second major reason for the Return Center contract with the slow implementation of the MPS);(b) the contract negotiations Return Center was that JVS and MPS progressed slowly due to poor rela- had poor inter-organizational relations between JVS and MPS; and

-515- 404 JVS

(c) theReturn Center negotiations Evaluation Results were tied to negotiations for establishing Job Score classes in the Results for the Milwaukee Youth Milwaukee public high schools. Employment Center

As finally implemented, the Two types of research were per- Return Center staff consisted of a formed to examine the effects of the program coordinator, a psychologist, MYEC component of this project. a social worker, a vocational First, straightforward analyses of counselor, three teachers and a sec- the experimental results were per- retary. formedto describethe effectsof the project. Then, more complex The intervention had three (some- analyses were performed to unearth times four) components: as much information -as possible about the relative effectiveness of 1. A reentry conference was held various elements of the overall MYEC in which the youth (sometimes with activities. his or her parents) met with Return Center staff to initially assess and Primary analyses of MYEC effects. interest the youth in participating JVS successfully implemented a true in school activities. experimental evaluation of the MYEC job finding intervention. Beginning 2. An assessment phase followed in August, 1981 and continuing until in which formalassessment of the February,1982, 252 intake youth were youth's basic academic level, intel- randomly assigned to treatment, and ligence, and vocational interests 154to control. Randomization was was made. accomplishedon-site by using an intake blank on which 62.5% of the 3. A referral conferencewas lines contained a "treatment" desig- held (sometimes with the youth's nation and 37.5% of the lines con- parents in attendance), in which the tained a "control" designation (a child was referred to either an MPS five-to-three ratio of treatment to or Community-Based Organization control clients). These blanks were (CB0) forinstruction, or was re- prepared bythe national evaluator ferred in-house to receive individu- using computer generated pseudo-ran- alized instruction at the Return dom numbers. Center itself. Applicants' names were entered on 4. Youth remaining at the Return this list in the orderthey pre- Center for instruction received sented themselves at MYEC. Control individual tutoring, usually youth were referredto other pro- directed at completing theG.E.D. grams for youth in the Milwaukee Most youths who stayed at the Center area. At intake, both treatment and did so because they wished to return control youths were administered a to an MPS site, and could not do so pretest that measured several of the immediately because it was too late objectives of the MYEC program. in the academic semester. They were placed inMPS programming at the Seven months after intake, treat- beginning of the next semester. ment andcontrol youths were contacted by MYEC and JVS outreach staff. Each was individually administered a questionnaire thatass- essed several personal characteris-

-516- 405 JVS

tics, past andcurrentemployment post-test. This scale shows lower status, and involvement in educa- test-retest reliability than the tional activities. Control youth other two scales (.19). were paid $10.00 for their participation. A total of 203 treatment Table 1 gives the results of youth were surveyed, for a 81% fol- post-randomization checks on the low-up rate; 118 of the control equivalency of theMYEC treatment youth were administered the questi- and control groups. Chi-square onnaire for a 77% rate. Thetwo tests on the sex of the subjects and follow-up rates do not differ signi- the percentwho were of minority ficantly (p = .41 by the chi-square ethnicity (mostly Black and His- test). panic), show no differences between the groups on these.measures. Ana- The pre- and post-test measures lyses of variance indicate that the of personal characteristics were groups were similar in age,and in subjected to principal factor analy- the time of intake (there is a sis with iterated communalities. trend, p = .10, toward the treatment Three factors had eigenvalues youth being slightly older than the greater than 1.0 and were subjected 'controls, but the absolute size of to VARIMAX rotation. Separate ana- thedifference is very small--see lyses ofthe pretest and post-test Table 1). The groups did not differ items yielded similar factors. The on the time between intake and fol- three factors were labeled Psycholo- low-up. gicalHealth, Interpersonal Compe- tency, and Rebellious Autonomy. Analyses of the pretest psychosocial measures show nb-diffeielices-on Scales were constructed from the PsychologicalHealth, or Interper- items loading most highly on each of sonal Competency. The control group the three factors; a particular item reported significantly greater was only included on the scale on Rebellious Autonomy (p=.006) than which it had its greatest loading. thetreatment group. This result Although the three factors were means that this pretreatment differ- similar for the two testing periods, ence will have to be accounted for there were differences in a few in any analysis of outcomes, parti- items. Various combinations of cularlyon Rebellious Autonomy as items were subjected to internal measured atfollow-up. Given the consistency item analysis to develop procedure usedto assign youths to a set of items that worked well for treatmentand controlgroups, and each of the threescalesat both the quite close equivalency ofthe times. Appendix A gives the item groups, we are confident that a true content for thethree psychosocial experiment was implemented. measures developed through these procedures. Psychological Health Changes in the psychosocial mea- had a reliability coefficient sures were assessed bysubtracting (Chronbach's alpha) of.61 at pre- the follow-up score from the pretest test, and.59 at post-test, and a score. Mean changes, and results of test-retest correlation of .40. analyses of variance carried out on Rebellious Autonomy had alpha coef- the scores may be found in Table 2. ficients of .49 atboth pre- and No differenceswere statistically post-test and a test-retestreli- significant. The treatment and con- ability of .39. Thethirdscale, trol youth showed nodifferential Interpersonal Competency had an change in Psychological Health, alpha of.45 at pretest, and .46 at Interpersonal Competency, or Rebel-

-517- 406 JVS lious Autonomy; An additional both groups were notworking at attitudinal measure not available at follow-up. There is a tendency for pretest was developed for the post- the control group to report more test. It was called Attitude Toward involvement in school (either re-en- Education. This was a scale com- rolled in high school, or attending posed of five items designed to mea-. a G.E.D.program). Thistendency, sure the degree to which the respon- whentaken together withthe very dent perceived formaleducation as small difference in work status, usefuland relevantto hisor her produces atrend (p = .07) for a life. Thismeasure was developed higher percentage of controls to because EducationalServicesstaff report a "favorable outcome" (either felt that one of tneir primary working or it school) than treatment objectives was convincing youths youth. that was indeed relevant to their lives and livelihood. The difference in reported school Table 2 shows that there were no attendance is difficult to inter- differences between the experimental pret. Project staff have repeatedly and control groups on this scale. noted that at intake clients invari- ably report that they are working on The analyses ofthe measures of their G.E.D., even if they have MYEC's intermediate objectives gives never enrolled, or have never no reason to believe that they were attended a G.E.D. class. Many young met. Nothing in these results sug- dropouts believe that they will gests that any additional job exper- graduate, "some day." MYEC clients ienceprovided toclient youthby are faced with the fact that if they theMYECprogramresulted in any truly want to earn their G.E.D., the positive psychosocial change. training is available at MYEC through Educational Services. It is More critical for the MYEC evalu- therefore moredifficultfor MYEC ation is an analysis of the current clients to claim that they are work- and past employment and educational ing ona G.E.D., if they are not, status of MYECclients and their than for the control youthto make controls. Table 3 gives the results such a claim. Although we have no of chi-square analyses comparing way of verifyingthe respondents', treatment and control youth on three reports it is doubtful that the measures of program success: cur- experimental and control groups dif- rent employment status, current edu- fered in participation in a G.E.D. cational status, and a combined edu- program. cational and employment status measure. For thiscomposite mea- Further information on employment sure, a "positive outcome" was and educational status can be found regarded as a report ofbeing in in Table 4. This tabulates 'the schoolor employed. These outcome results of chi-squares comparing measures were derived from a ques- responses to various categories tion asking "What are youdoing available in response to a question mostly?" The respondent could only asking "Whatare youdoing now?" check one of severalcategories of Allowing multiple responses enabled educational involvement oremploy- students who were working part-time ment. Analyses indicate that the and going to schoolto record both MYEC clients are no more likely to activities. reportthemselves to be currently working on a job than are the con- As the table shows,the results trol youth. Approximately80%of of the analysis of the questionnaire

-518- 407 JVS item measure mirrorthosefor the None of the nine measures reveal previous item. About equal percen- any differences between the treat- tages of treatment and control ment and control groups. The groups youths report they are currently do not differ on the current number working for pay, but more control of hours working per week,current thantreatmentyouthsreport that hourly wages, current weekly wages, they are attending a G.E.D. program ortotal wages earned in thelast and more report that they are six months. Thegroups are also enrolled and attending high school. similar in the the number of weeks More treatment than controls acknow- that they had worked full time and ledge that they are unemployed parttime. Finally,a logarithmic (which also connotes not attending transformation of the current hourly school). A higher proportion of the wages and total wages earned (a control youths also report that they transformation commonly carried out are "On temporary layoff from work." on wage data to decrease the influ- enceof extreme valuesin what is Tables 3 and 4agree in implying usually a markedly skewed distribu- that (a) the MYECclients are no tion) did not produce any evidence more likely to be employed at fol- of differences between the treatment low-up than arethecontrols (and and control groups. veryfew of eitherareemployed), and (b) a higher proportion of con- There is noevidence in these trol group members report themselves results that MYEC improved the tobe re-involved insome kind of employment situation of its clients. formal education. Of particular They are no more likely to be interest is the difference inthe employed at follow -up, to have proportion ofrespondents from the worked more in the recent past, or two groups who report they are to be earning or have earned any "Enrolled in and attending high more money. There istherefore no school." We cannot be sure that the evidence in these results that the controls were simply"yeasaying" MYEC program met its primary goal of more than the treatment stu- increasing the employment opportuni- dents--although the lack of differ- ties of its clients. ences in responding to work-related questions would argue against that The MYEC project was funded pri- hypothesis. But both Tables 3 and 4 marily as a delinquency prevention suggestthat participation in MYEC effort. Employed youth were theor- may have discouraged students from ized to have less of a need to com- returning to formal schooling. The mit crime out of a need for money, results offer no evidence that MYEC for status, or for ,excitement; was successful in aiding clients to Because the treatment group did not find more stable employment than differfrom the control group in they could have found on their own. employment status, one would not expect (on the basis of this theory) Other follow-up questions con- delinquency to be differentially cerned hourly wages, current hours affected. Yet, personal counseling worked per week,and hours worked was a largz part of the MYEC pro- full and part-time "in the last six gram, and might be expected to have months." This insured that all some positive effect on delinquent repondents would be reporting only behaviorindependent of employment on employment experienceacquired effects. There is no evidence that since MYEC intake. Table 5 gives it did. The final outcome listed on the results for analyses of variance Table 5 shows results for a question for the wage and hour measures.

-519- 408

J. m

$40* 441-41y it*** Oat the last tech of the leer possible reason* os Otto tho teseottkeit Cad ette tot falling to demonstrate program +44e4 up 4)* che $tip. %et* ere effectiveness is important, and a pie tiftwo**** **,*40o0 tioo it**troti prtgrat orits evaluation may fail sue oostto4 IP#44 00 WO 01044444.1 for any of these reasons.

044 040010 fit sstetisottal seracafteot easily assess the ade- es-J*4*ft** of tlie Vi i« it townael. quacy of a program** theoretical .1440, 044140 400 4.401144+ MI, to premises, or its relation to its folke.wis*, OW, to* he 44414401444 44 theory,unless one knowsthatthe fleitos*i rhost is so ovideame that program was reasonably wellimple- the Whe sepses Nod owp affect as oonted. Table 6 offers strong evi- 4110441p4101 trilt44. psythatosial dent' that the HYIC job counseling liese4almos44 oat detteguett beksist. program vas not implementedwith Uwe ;* tome woe, tettdeset Om the enough fidelity to reach strong con- #014404V6 edif hem Umelearaged poeth clueless aboutits ,gaggligl of *t Monies to taloa.

AN4A4MUL.ANCWWL...g....fing Table 6 gives simple descAptive illAtAp4 thete ate fosspesethle statistics of four implementation aises* w4p, au evalwatioa may ukow measures; the number of counseling 44414.104* 4ivitoot treattoott eel contacts, the number of educational ivattel groupti servicescontacts, the number of jobsto which the client was re- i* 'lbw tilimety atebtod the progtam ferred, and the number of jobs that Is 4**04--110 0614* st *1* rearms the client was actually placed in by tout* ust **to toot tot mom it its WIC. Meet impressive is the last obleciiee* Beare. columnin Table 6. It showsthe percentages of clients receiving Po Mc eosmsom tide( address Aims of that particular part of the the theorpithe pees,. woe tart- intervention. The bottom line, of tots* te the the tows wbtds it Mr.,. is job placementsplacing wet mappeeedly baseed. youth is jobs is what MYEC was all about. Yet, 72Z of the MYEC clients 2* fat program wee oet tople- between Moot 1981, and February, 00ated pempetlyn.,the tatervestieme 062 were not placed in a job. In Mat occurredmote eat the met other words, only a little over a Weeded* quarter of KYICclients received MTEC's main service. Only half were 444 tovelea44., desists was ever even Wang to a job. For iOneosillive 4. the tree effects of each of the four implementation mea- 4Aom progrom. sures, the modal (most common) amount of service provided to a par - al;WWwir.14402,41.114WW0,41,446M2.10.41.44 titular client was aero.

I* Est lees plao. called for the Apparently the MYEC project was a project to immure of records job placement project that did not sof olive 08044itts. Partly because place many youths in jobs. In other at the tarty temitatios of thie words, the job counselineplacenent peolect, sod pertly doe to the low program was not implemented as petteity pawed ma evaluates maav Weeded. We cannot clearly address Otos hp she project director (see the question of the theoretical ade- its 110.0 ,), Oficial police records quacy of the MYEC program, or tore erne obtsimool. relation to its theory(the first

409 JVS and second possible reasons for pro- allowed to enter atthisstage if gram failure listed above) because they contributed to the explanation the evaluated MYEC program was not of the outcome of interest. the Intended MYEC program. As to whether the evaluation designwas Inthefirstset of regression sensitive to possible program analyses, MYEC treatment effects effects, a well implemented true were studied by entering at step two experiment is a strong design (Cook an overallindex of MYEC treatment and Campbell, 1979). The high res- received. Any additionalvariance ponse ratefrom alarge number of in the individual employment, experimental treatmentandcontrol schooling, and delinquency scores participants implies that this that was captured bythis stepin aspect of the experiment was faith- the analyses might be attributable fully implemented. Although ques- to MYEC treatment, netof measured tions could be raised about the mea- background characteristics. sures examined, the job- and education-related measures appear The MYEC treatment intensity straightforward and highly relevant score was constructed by dichotomiz- 4. to the evaluation of this project. in the four MYEC implementation measures atthe median. Each MYEC A further question might be, "If client received a score of1 on each MYEC had systematically implemented measure for which he or she received the program, would it have been able more than the median amount of to placemore of its clients in treatment, and 0 if he or she jobs?" In order to investigate this received less than the median amount a question,a series of regression of treatment. These four scores analyses were undertaken. Statisti- were then summed to yield a single cal models were built to account for index of treatment intensity (in separate analyses) individual received. A client's treatment variation in the five primary intensity score could range from employment measures, an indicator of zero to four. enrollment in school,and self-reported police contacts. Table 7 reportsthe results of the MYEC treatment intensity regres- Two regression models were sion analyses on the primary employ- employed toinvestigate effects of ment variables: hours currently differential exposure to MYEC treat- working per week, weeks worked ment. In the first model, for each full-time inthelastsix months, MYEC client a singleindicatorof weeks worked full-time inthe last MYEC treatment intensity was entered six months, log-transformed current which reflected the total amount of wages, and log-transformed total MYEC treatment received. In the earnings in the last six months. second model,each separate compo- Listed are the effects ofa) those nent of the MYEC program was indivi- predictor variables entered at step dually examined. In both models, a one that significantly increased the stepwise procedure was employedin variance explained, and b) the MYEC which background and personality treatment intensity variable measures at intake to MYECwere (labeled "MYEC participation"). The entered first; sex, ethnicity, age, column headed "beta" shows the con- month of enrollment in the program, tribution ofeach variable, netof Psychological Health at pretest, any previously entered variables. Interpersonal Competency at pretest, The beta given for the MYEC partici- Rebellious Autonomy at pretest, and pation score is for the direct basic skills proficiency were

-521- 410 JVS effectof MYEC treatment,indepen- Table 8 gives the results for dent of all examined background mea- regression analyses investigating sures. That is, the beta represents the degree to which MYEC treatment the MYEC treatment's contribution to participation can predictthe psy- explanation that cannot be attri- chosocial outcomes of Psychological buted to any other variable in the Health, Interpersonal Competency, model. The third column headed and Rebellious Autonomy as measured "Increment to R2" shows the propor- at post-test, re-enrollment in for- tionof variance in thecriterion malschooling,self-reported police accounted for by adding that predic- contacts, and Attitude Towards Edu- tor to the model, net of previously cation as measured at post-test. entered predictors. Thus, the "MYEC Again, background variables were participation" entry gives the added entered first, and the MYEC partici- variance in the predictor explaina- pationvariable last;- the effect ble by increased MYEC participation, given.forthe MYEC variable under holding constant the background mea- the heading "beta" indicates the sures. Finally, the last colutin, directeffects of MYECparticipa- headed "p for increment" provides a tion, independent ofany variables statistical test for the added con- entered previously. tribution (incremental validity) of each successive variable. The p's Table 8shows that greater MYEC for MYEC participation are of parti- treatment has no statistically sig- cular interest--significant incre- nificant net contribution to psycho- ments in explained variance (marked social development (although there by an asterisk) suggest that parti- aretrends towardtreatment being cipation in the MYEC program proba- negatiVely associated WithRebelli- bly contributed to an increased ous Autonomy, p=.08, and positively score on the criterion measure associated with Attitude Toward Edu- investigated. cation,p=.10). Table 8 also suggeststhat increased participation Table 7indicates thatthe MYEC in MYEC activities is associated intensity variable was a significant with an increased likelihood of predictorof employment status for being re-enrolled insomeform of four ofthe five employment depen- formal schooling, i.e., MYEC clients dent variables: receiving more MYEC who persisted in the program were treatment is associated with more more likely to enroll in school. hours per week currently worked, Note that this result differs from more part time work engaged in dur- that found in analyses of the treating the last six months, higher cur- ment and control comparisons. rent wages earned, and greater total Results ofthose analyses suggested wages earnedduring thelast six that the MYEC programdiscouraged months. re-enrollment in school. This apparent discrepancy can be reconciled Table 7also indicates that age when itis recalled that most MYEC and ethnicity are consistently asso- clients received little or no MYEC ciated with employment outcomes. treatment. It would appear that for Older youth show better employment the majority of MYECclientswho and earnings than blacks. There is receivedonly superficialexposure also some evidence that greater to the program, MYEC discouraged Interpersonal Competency as measured them from becoming reinvolved in at pretest predicts less part-time formal schooling--yet the program employment, and, therefore, less had the opposite effect on youth total earnings over the last six thatpersisted intheprogram and months. received the entire treatment.

-522- 411 JVS

These regression analyses show tus on the other, and some that there is a mild negative pred- suggestion of a weak effect on del- ictive relationship (beta=-.12, inquency. p=.13) between MYEC participation and self-report arrests ("picked up In order to more closely examine bythe police during the lastsix the MYEC effects,a second set of months"),i.e.,there is weak evi- regression analyses were carried dence in these data of a direct out. In this set of analyses indi- effect of MYEC participationupon vidual measures of each of the four the reduction of delinquency. This MYEC treatment variables were used result is slightly muddled in that to predict employment status (Table police contacts are measured somew- 9) and psychosocial, schooling, and hat concurrently with the MYEC vari- delinquency status(Table 10). In able--the youth are reporting on these analyses, the individual MYEC their last six monthsexperience, treatment indices weresubstituted part of which would have been spent for the overall MYEC treatment par- in MYEC. Some clarity is lent to ticipation score in the second step this analysis by the fact that most of the analyses, following the entry clients werein MYEC fora short of the background variables. Counts period oftime,and even long-term of counseling contacts, educational clients would seldom have been in services contacts, job referrals, the program for a month. Since the and job placements were allowed to average follow-up period was about enter the statistical modelin the seven months,the majority of MYEC order of the amount of variance each participation would have occurred explained. In all cases, statisti- prior to the period forreporting cally significant effects listed are arrests. There is some legitimacy, for the direct contribution ofthe therefore, in believing that the significant MYEC treatment compo- MYEC effect found (such as it is) is nent(s)toward the explanation of a true "predictive" relationship: variance ofthe outcome measure in the longer a youth persisted in MYEC question.2 treatment the less likely he or she was later to report being picked up Table 9 shows that the number of by the police. educational services contacts and the number of job placements are Table 8 also indicates thatthe generally associated with more favo- strongest predictor of future psy- rable employment outcomes. Counsel- chosocial development is present ing contacts show little relation- psychosocial development--personali- ship to employment outcomes, except tyis fairly stable across the time for a slight tendency for those who period covered by this study. Most received more counseling to be less of the variance in Psychological involved in in part-time employment Health, Interpersonal Competency, (p=.09). This tendency contrasts and Rebellious Autonomy at post-test with the result for number of educa- is accounted for by the same person- tional services contacts which show ality variables measured at pretest. a moderately strong positive

To summarize the effects of increased exposure to MYEC treatment, Tables7 and 8 show positive 2. Effects for the other variables relationships between the amount of in the model--Lhe backgroundvaria- MYEC treatment on the one hand and tion--are not given in Tables 9or employment and formal schooling sta- 10 because they areidentical to those given in Tables 7 and 8.

-523- 412 JVS relationship with part-time employ- greater Psychological Health, and ment(p<.001). MYEC records indi- with more positive Attitude Toward cate that most ofthe jobsin the Education. MYEC job bank during the time of the evaluation were part time jobs. Theoveralltreatment intensity These resultssuggest that clients analysis indicated that increased availing themselves of the help MYEC treatment was associated with offered by Educational Services were increased re-enrollment in formal more successful atobtaining work schooling; Table 10 shows that this than were other clients. The nega- effect is largely the result of the tive correlation between counseling efforts of educational services. It contacts and part-time employment is the only individual treatment suggests either that the counseling component that is. a significant staff were actively discouraging predictor of re-enrollment in clients from seeking part time work, school. or that youths who received counseling also experienced more difficulty Finally, Table 10 suggests a in getting jobs. There was no small positive effect for job place- organizational policy of either ments on police contacts (p < .10). encouraging or discouraging part- Clients who were placed in jobs were time work, and different parts of slightly less likely to report them- the MYEC project may havetreated selves to have been picked up by the their clients differentlyin their police. Thus,the majority ofthe approach to part-time work. trend toward decreased self-reported arrests associated with persistence Overall, Table 9 provides no evi- in MYEC (see Table 8) is attributa- dence that counseling in the coun- bleto actually being placed ina seling unit was successfulin pre- job. This is important for a theor- paring clients for employment. etical analysis of the relationship Controlling for backgroundvaria- between employment anddelinquency bles, there is no significant rela- (see Appendix B). tionship between counseling and employment. Recall that counseling A final observation from this wasconsidered to be the primary analysis of MYEC treatment component intervention: educationalservices effects is that the counseling com- contacts were supposed to be an ponent is essentially unrelated to additional treatment available for employment (although somewhat nega- clients desiring upgrading of basic tively related to part-time work), skills. The present analyses sug- and is associated with significantly gest, however, that training in less Interpersonal Competency at basicskillshad more effect then follow-up, i.e., the few effects the prescribed counseling. noted are in a 'direction opposite to program objectives. Table 10 shows the effects of the individual treatment componentson In summary, the results of the psychosocial development, re-enroll- foregoing regression analyses sug- ment in school, and self-reported gest that: arrests. Only the implementation measure ofjob placements explains 1. More exposure to theMYEC any additional variance in the psy- treatment is associated with better chosocial measures at post-test over employment status at follow-up, but that explained bythe pretestand the primary evaluation--the true background variables. Being placed e.hw ex-per-i-ment---in-di-cates no in a job by MYEC is associated with

-524- 413 JVS overall positivebenefit for the tional services offered in-house MYEC program. It is difficult, even G.E.D. instruction,it also contri- with the variety of pretest and buted tothe reinvolvement of MYEC background information available on clients in formal education. the clients, to entirely control for pre-existing motivational differ- The positive educational services ences among MYEC participants; youth results are important for the over- that stuck to the MYEC program, and all evaluationof the Delinquency received more treatment, might show Prevention Through Alterative Educa- enhanced employability simply tion initiative; this alternative because they are harder workers, or education interventionappears to are more persistent in a job search. result in some positive outcomes.

Nevertheless, because we do have 4. Three other consistent pred- a number of statistical control ictors, of employment status are the variables, the results suggest that racial status of the clients, their had clients received more systematic age, andthe monththe youth was and thorough exposure to the treat- enrolled in the MYEC program. ment, the program would have been Whites are more likely to be successful. employed; olderyouth havebetter employment outcomes;and the later 2. Receiving assistance through the in the year that a youth was refer- EducationalServices componentand red to MYEC the lessthey worked actually being placedin a job by full- or part-time in the following MYEC is associated with some posi- seven months. tive outcomes. No evidence suggests that counseling, MYEC's primary The results for race imply that intervention, was successful in aid- discrimination in hiring exists in ing MYEC clients to find employment the cityof Milwaukee. Even when or to improve their psychosocial controlling for backgrounddiffer- skills. In fact, to the extent that ences, including basic skills profi- pre-existing client differences were ciency, there is still ,aresidual successfully controlled for by pre- net benefit to being white. In test measures,the results suggest fact, in most analyses; the beta for that counseling produced a decrement racial status is largerthan that in InterpersonalCompetency. This for receiving MYEC treatment. result is not necessarily an indict- ment of employment counseling in One might argue that some motiva- general. Recall that the MYEC tionaldifference accounts for the counselors hadlittletraining or differential employability of whites experience in counseling. and minorities. This hypothesis can beexplored bylooking atpersis- 3. The educational services compo- tence in treatment. Table 11 shows nent of the program appears to have the simple correlations between the contributed to the employment and number of MYEC component contacts educational status ofits partici- and white/minority status. Minori- pants. Again, to the degreethat ties stayed in treatment longer as motivationaldifferences are cont- measured by the number of counseling rolled for by the pretest and back- and educationalservices contacts. ground measures, basic skills educa- There is no evidence here to suggest tion in an alternative out-of-school that minority youth were unmotivated formatincreasestheemployability to attend program sessions. of dropout youth. Because educa-

-525- 414 JVS

The lack ofcorrelation between deepened, and jobs may have become minority status and job referrals in increasingly hard to come by. The Table 11 indicates that MYEC person- effect is not due to any increase in nel referred proportionately as many time betweenentryand follow-up. minority as non-minority clients for The time lag, which was measured for jobs. Yet the significant correla- most of the follow-up population, is tion between job placement and uncorrelated with any of the outcome white/minority status demonstrates measures. that MYEC was unable to overcome the basic white/minority difference in Whatever is the source of the employability. Although MYEC spent decrease in full time employment for more time with minority clients, and the MYEC clients over the course of referred both whites and minorities the evaluation, there is nothing in equally. to jobs, white youths were the results to suggest that the pro- still more successful in securing gram improved during the courseof employment. the evaluation. Clients served in the later months of the project were The findings of enhanced employ- less likely to work full-time than ability with increased age, and were clients served in the project's decreased employability as the pro- early months. Project personnel gram progressed are also important argued that the recession made their results. The age of MYEC's service task almost impossible;employment population was a continual source of was so high in general, and in their contention both within the project targetpopulation especially, that itself, and between the project and there were simply no jobs available the federal sponsor. MYEC personnel in which to place the clients. feltthey had been saddled byJVS Research indicates that youth unem- with an impossible task: finding ployment is verysensitive to the employment for 16- and 17-year-old state of the economy--a one percent dropout mostly minority youth during increase in the official adult male a recession. Particularly disturb- unemployment rate translates into a ing to the staff was the constraint 5-6% increase in the number of teen- they felt in being required to serve age males looking for work(Clark only 16- and 17- year -old, youth. and Summers, 1980). In fact, Clark They felt that they could have been and Summers (1980)arguethat the much more effectiveserving older foremostreason for high teenage dropout youth.3 unemployment is a lack of jobs, and that it is fruitless to set up The finding of poorer employment "employability" programs (such as status as the evaluation progressed MYEC). Such programs (even when is difficult to interpret. One pos- they work) may merely substitute one sibility is that the general Milwau- youth for another in the employment keeemploymentsituation mayhave queue with no net positive result on been deteriorating over the course unemployment in the target popula- of the evaluation asthe recession tion. This problem would be particularly acute during a recession as severe as the one that occurred during MYEC's operation. 3. The Office for Juvenile Justice and Delinquency Prevention's legis- Additional observations on MYEC. lation limits its purview to juven- As discussed earlier, the MYEC pro- iles. Because the project was spon- gram underwent substantial changes sored by OJJDP, it had to serve during the winter of 1981-82. juveniles.

-526- 415 JVS

Emphasis was shifted from a one-on- preliminary experiment was attempted one personal counseling orientation, in all four schools, but the designs to a structured seriesof defined brokedown in three of the four workshopsaccompanied byincreased schools: control students were not input from Educational Services. pretested (or identified), and ran- Since the evaluation covered the domization was not actually imple- period of August 1981 to mid-Febru- mented because randomization took ary 1982, the follow-up essentially place before students in the pool of evaluated the early version ofthe eligibles were asked whether they MYEC program that the program per- wished to be in the class; many did sonnel themselves had found unsatis- not. In one of four schools random- factory. ization did take place, but in that school the collection offollow-up A second experimental evaluation data at the end of the-semester was of the MYEC project had been begun. incomplete. Of the 21 students A job readiness pretest covering the assigned to treatment and 18 to con- skills thought necessary for emplOy- trol, complete data were only avai- ment success had been developed by lable on 12 (57%) ofthe treatment project staff. This instrument 'students and 7 (39%) of the control wouldhaveserved not only as a students. Results of such a follow- experimental pretest, but as a check up are virtually uninterpretable. on the program's success in provid- The data were analyzed nonetheless, ing the skills it deemed important. and the treatment and controls did This second experiment was to begin not differ on any outcome measure: in October, 1982,but was delayed Internal Control, Self-esteem, _ _ _ _ _ several times. It hadbegun on Interpersonal Competency, Rebellious November 30,1982, butthe discon- Autonomy, Alienation, Attitude tinuation of the project by the Off- Toward Education, Employment Status, ice for Juvenile Justice and Delin- or Drop-out. quency Prevention aborted the second evaluation. Having learned from their first experience, the experimental evalua- The loss of the second evaluation tion of the Job Score class in four is unfortunate because of the major high schools in the fall of 1982 was changes that theMYEC project had much better implemented. Randomiza- undergone. Accordingly, the evaluation was carried out properly,and tion presented here is an evaluation pretests on personality and job ofthe program in development, not readinessskills were administered as developed. We know that a MYEC toall treatmentand controlstu- program based on one-to-one counsel- dents. Unfortunately, theexperi- ing was notsuccessful; we do not ment was terminated along with the know if the project modifications project when OJJDP decided not to produceda more successful setof continue funding. The Milwaukee interventions. Public Schools were unwilling to complete the follow-up withoutthe Evaluation Results for the Job Score funding provided by OJJDP. Class In short,we know little about In the spring of1982, the Job the effectiveness ofthe Job Score Score class was begun in four Mil- class. This is particularly unfor- waukee high schools. Together with tunate because an enormous amount of the project staff, we attempted to careful work went into the develop- implement two successive evaluations ment of its curriculum, and because of the Job Score Classes. A

-527- 416 JVS of the interest the project's referred to a program close to where sponsor had shown in developing this they live--the design provided a aspect ofthe project. One ofthe provocative comparison of "official" goals ofthe Alternative Education alternative education versus CB0- Initiative was to develop innovative sponsored alternativeeducation. alternative educational approaches for youth; the Job Score curriculum The evaluationwas never com- seems tohave been aimed atthat pleted. The evaluation began well goal. and was progressing nicely when the project's funding was discontinued. Evaluation Results for the Return Pretestand backgroundinformation Center on the first 125 referrals had been collected,and follow-up forms had The Return Center did not begin been designed and approved by JVS, operation untilJanuary,1982. At MPS, and the CB0referral sites. that time MYEC personnel were begin- All information was being provided ing the follow-up of the MYEC exper- as requested. Because ofthe pro- imentaltreatment and controlsub- '4ject's termination, we know nothing jects. This follow-up was not about the efficacy of the Return progressing well, and we recommended Center. that MYEC focus its energies on completing theevaluation ofthe job Recommendations counseling programaleady in progress. One good evaluation was The evaluation of the JVS Alter- _ judged tobe more worthwhile than native -Edudation-Pfoje-dt,--affd- our twobad ones. In addition, since interpretation of it, lead to the the Return Center was'a brand new following recommendations: intervention, we felt that it should be allowed to operate for some time 1. The Job Score class appears and work the kinks out before being to be a carefully developed curricu- subjected to formal evaluation. lar innovation worthy of dissemina- Thus, no evaluation forthe Return tion and testing. Procedures should Center was designed for the spring be established to examine its effec- of 1982. tiveness.

During the August,1982 confer- 2. Delinquency prevention pro- ence, an evaluation ofthe Return jects involving individual counsel- Center was designed for implementa- ing should be staffed with profes- tion in the fall of 1982. The sionallytrained counselors. Such design involved tracking the first counselors should, at a minimum, 125 referrals to the Center for both meet the guidelines or standards for thefirst and second semestersof counselors as laid outby the Ameri- 1982-83 to see how many actually did can Personnel and Guidance Associa- re-enroll in some kind of schooling, tion, the American Psychological and what kind of progress they had Association, the Association of Mar- made by the end of that semester. A riage and Family Counselors, or the comparison would be made between the laws ofthe state within which the success of youth referred to MPS project operates. Paraprofessionals alternative education sites, and providing counseling services should

youth . referred to CBO-sponsored be closely supervised by profession- training (mostly G.E.D. instruc- als. Counseling interventions tion). Since referral was largely should be rigorously evaluated. based on geography--youth were Funding agencies should require that

-528- 417 JVS- such minimum standards be adhered to ingness to subject themselves to by projects they sponsor. rigorous evaluation. Evaluation is risky--it exposes oneself to the 3. Experimental research, should verification offailure. But this be conducted on the consequences of risk is mitigated whenevaluation the employment of dropout and poten- results are used to produce program tial dropout youth. We should not improvement, and an ultimately have to rely on correlational meth- satisfactory evaluation. This is ods to answer policy questions that the essence ofthe Program Develop- would be better answered experimen- ment Evaluation process. It is tally. unfortunate that the present report is a summative evaluation--it would 4. Experimental research should have been better construed forma- be conducted on the outcomes of tively. returning dropout youth to school. We have long assumed that "the more It is much to ask of a newly con- education, the better." Some ceived program thatit demonstrate results presented here question this 'effectiveness in its first or second assumption. Under whatconditions year of operation. Even private is it beneficial to return youth to enterprise ventures routinely oper- anenvironment inwhich theymet ate ata loss early intheir his- with failure and frustration? tory; this is considered normal, and unnoteworthy. Yet social service 5. Experimental research should programs, addressing large problems be conducted on the net effects of in small ways, are expected to job search assistance programs such demonstrate efficacy almost immedi- as MYEC. Are emplOyment slots ately. There can be no place for filled more quickly, or with more sloth or useless projects; there can productive personnel? Or do employ- also be 'no place for pie-in-the-sky ment assistance projects simply expectations for instant success in rearrange the employment-seeker difficult areas. What is needed is queue? What happens to the persons systematic attention to the improve- that would have landed the jobs? Is ment of programsover time. Our thereany netbenefit to society nation musthave the courage and during a recession when there is a commitment to provide the support large labor surplus, and competition that will allow program development for jobs is fierce? to take place in a spiral of improvement over time. Projects to Epilogue design, build, and test ballistic missle systems can take decades--why MYEC and the JVS project in gen- would less time be required to pre- eral were exemplary in their will- vent youth crime?

-529- 418 JVS

References

Alwin, D. F. & Hauser, R. M. The decomposition of effects in path analysis. American Sociological Review, 1975, 40, 37-47.

Clark, K. B. & Summers, L. H. The dynamics of youth unemployment. In Youth Knowledge Development Report 2.9. Research on Youth Unemployment and Employability Development: The Youth Employment Problem--Dimensions, Causes and Consequences. Washington, D.C.: National Bureau of Economic Research, May, 1980. Available from the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C., 20402.

Comer, J. The education of inner-city children.. Grants Magazine, 1980,3, 20-27.

Cook, T. D., & Campbell, D.T. Quasi-experimentation. Chicago: Rand McNally, 1979.

Gold, M. Scholastic experiences, self-esteem, and delinquent behavior: A theory for alternative schools. Crime and Delinquency, 1978, 24, 290-308.

Gottfredson, G. D. Models and Muddles: An ecological examination of high school crime rates. Journal of Research in Crime and Delinquency, 1979, . ______16 307-331.

Gottfredson, G. D., & Cook, M.S. A cognitive theory of person-environment interaction with implications for social control. In Program in delinquency and school environments: Program Plans, 12/1/82 - 11/30/84. Balti- more: The Johns Hopkins University, CSOS, 1982.

Greenberg, D. Crime and the age structure of society. Contemporary Crisis, 1977, 1, 189-223.

Greenberger, E. A researcher in the policy arena: The case of child labor. American Psychologist, January, 1983, 1, 106-111.

Greenberger, E., & Steinberg, L. D. Adolescents who r -ork: Health and behavioral consequences of job stress. Developmental Psychology, 1981, 17, 691-703.

Hirschi, T. Causes of delinquency. Berkeley, Calif.: University of Califor- nia Press, 1969.

Yamasaki, C. The Milwaukee Youth Employment Center. In G. D. Gottfredson (ed.), The School Action Effectiveness Study: First interim report (Report No. 325). Baltimore: The Johns Hopkins University, CSOS, 1982. Table 1

Post-Randomization Checks on the Equivalency of the MYEC Treatment and Control Groups, 1982

MYEC Control SD N M SD N p

Age (years) 16.81 .67 247 16.70 .72 149 .10

Sex (male=1) .67 -- 252 .63 -- 154 .30

Month enrolleda 3.90 1.58 252 3.98 1.52 154 .60 in program

Days from enrollment 219.79 34.94 184 215.63 44.00 98 .39 to follow-up

Interpersonal .89 .14 240 .88 .15 144 .58 Competency

Psychological .73' .21 245 .70 .22 145 .21 Health

Rebellious .61 .26 240 .69 .24 145 .006* Autonomy

a The first month of the experimental evaluation (August, 1981) is coded "1," the second month (September) is coded "2," etc.

* Statistically significant. Table 2

Mean Post-Test Psychosocial Development Scores for '1YEC Treatment and Control Groups, 1982

MYEC Contol

Pi SD N M SD. N

Interpersonal Competencya -.01 .19 176 -.02 .19 101 .65

Psychological Healtha .02 .43 180 .05 .26 101 .26

Rebellious Autonomya -.02 .30 176 -.06 .28 102 .27

Attitude towrd Education pc 2.79 .70 185 2.75 .71 108 .95

a Average change from pretest to posttest. b Average rating for each item; scale for items ran from 1 - 5. c This iis an after-only score, ie., no pretest is available.

-532- 421 Table 3

Percentage of MYEC Clients and Randomly Equivalent Controls Reporting Favorable Employment and Education Outcomes

Outcome MYEC Controls

Employed 18 21 .68

In school 17 26 .10

Favorable outcome 36 47 .07

(N) (172) (106)

Note. Source is response to the question, "What are you doing mostly?" Response rates to this item are nearly equal for the two groups: 68% for }NEC, 69% for the control group. Favorable outcome means the respondent was employed or enrolled in school (or both). Table- 4

Iktrcontoge of KYZC Clients and Randomly Equivalent Coolest' Lporting Various Current Activities at follow-Up

10.011WW,WWIDW MMMMM ktivity MTECControls p 21.0.WWMUM MM W MM MIIMIPWOW

% kiPG fir psy 13 30 .27 (166) (90)

TakCag vocatioast 8 11 .57 courses (159) (85)

Gradested free high school 9 .9 .99 or coopleted C.C.D. (196) (114)

Astsodina a CEO program 25 39 .04 (163) (88)

Worollod is or attooding 5 14 .02 high *shoot (157) (81) told off Iron work 1 10 .002 (155) (81)

Geld 69 57 .05 (175) (97)

WOWW0WWWWWWW.WWWWWW1

Ipm. Source of most percentages are responses to a **Rills-option quostion, "What are youdoing?" ilsopeodoits mad mark as many alternatives as oppiiod. In a 'sprat* question respondents were asked about high school graduation. O's differ free 'with* to quipstloe because not all respondents fellosted as instruction to *ark one res- pease for oath lt** in the multi-optionquestion. pa eostbesos *We tiro number of respondents to each Start totst X for MEC 0 252, total M for control gywp* 154.

-534 423 r Table 5

Treatment and Control Group Comparisons on Employment Outcomesand Self-Reported Arrests, 1982

MYEC Control' M SD N N SD -11

Current hours worked 3.00 9.89 239 3.84 12.24 146 .46 per week

Current weekly wage 11.54 33.72 187 17.00 43.78 109 .23

Total earnings last 174.58592.38 189 160.76 509.05 112 .84 six months

Hours worked full-time 1.31 4.73 202 _1.27_ 4.12_117..__.95_ last six months

Hours worked part-time 1.92 6.34 202 1.77 5.18 116 .83 last six months

Current hourly wage .58 1.50 192 .67 1.39 114 .57

Transformed currenta .58 1.49 187 .72 1.69 109 .47 hourly wage

Transformed wagesa .75 2.22 189 .92 2.36 112 .55 last six months

Number of times picked .27 .69 185 .39 .89 109 .18 up by police, last six months

aBecause of extreme skewness in the earnings data, the hourly and total wage variables were subjected to a logarithmic transformation.

-535-

424 Table 6

NYEC Program Implementation Measures, August, 1982 to February, 1982

Received no treatment Implementation measure Mean MedianMode SD Range

Counseling contactsa 4.47 3.54 0,0 4.59 0-31 48 19

Educational services contact 3.43 0.20 0.0 8.96 0-60 171 68

Job referrals 1.58 0.43 0.0 2.69 0-21 129 51

Job placements 0.30 0.16 0.0 0.59 0-3 182 72

Note. N=252 for all measures. aIncludes personal meetings, and phone calls

-536-425 Table 7

Direct Contributions and Incremental Validity of a MYEC Implementation Measure to the Explanation of Work Outcomes According to Parsimonious Causal Models

Incremnt p for Predictor beta to R increment

Hours currently working per week (N = 198)

Ethnicity (white = 1) .19 .03 .009* Age .14 .02 .05* MYEC participation .16 .03 .02*

Weeks worked full-time in last six months (N = 169)

Ethnicity (white = 1) .24 .06 .001* Month enrolled in -.21 .04 .006* program Age .03 .01* MYEC participation .02 .00 .75

Weeks worked part-time in last six months (N = 169)

Interpersonal Competency -.31 .10 .001* Month enrolled in program -.15 .03 .04* MYEC participation .15 .02 .04*

Log-transformed current wages (N = 155)

Ethnicity (white = 1) .17 .03 .03* MYEC participation .23 .05 .03*

Log-transformed total earnings in last six months (N = 157)

Interpersonal Competency -.23 .05 .003* Ethnicity (white = 1) .21 .04 .008* MYEC participation .16 .02 .04*

Note. The components of the overall MYEC intervention were implemented to different degrees for different members of the treatment group. The increment to R2for MYEC participation shows the proportion of the variance in each outcome accounted for by a composite measure of participation in various MYEC treatment components. MYEC was more fully implemented for students participating more fully in the various interventions. A positive beta for MYEC participation implies that persons who participated more fully in MYEC had higher scores on the particular outcome measure in question, net of measured pre-existing personal characteristics. Variables listed in each set above MYEC participation are those pre-test measures that significantly contributed to the prediction of each outcome.

* Statistically significant.

-537- 426 Table 8

Direct Contributions and Incremental Validity of a MYEC Implementation Measure to the Explanation of Psychosocial Outcomes and Arrests According to Parsimonious Causal Models

Incremnt p for Predictor beta. to R increment

Psychological Health (N = 157)

Psychological health .47 .22 .001** Basic skills .20 .04 .006** MYEC participation .10 .01 .14

Interpersonal Competency (N = 156)

Interpersonal, Competency .18 .03 .02** MYEC participation -.04 .00 .58

.., Rebellious Autonomy(N = 155)

Rebellious Autonomy .40 .16 .001** MYEC participation -.13 .01 .08*

Re-enrollment in formal schooling (N = 141)

Psychological Health .24 .06 .005** MYEC participation. .16 .03 .003**

Times arrested last six months (N = 158)

Gender (female = 1) -.20 .04 .01** MYEC participation -.12 .01 .13

Attitude towards Education (N = 156)

Psychological health .20 .04 .01** Ethnicity (white = 1) -.17 .03 .03** Basic skills .16 .03 .04** MYEC participation .13 .01 .10*

Note. The increment to R2for MYEC participation shows the proportion of the variance in each outcome accounted for by a composite measure of participation in various MYEC treatment components. A positive beta for MYEC participation implies that persons who participated more fully in MYEC had higher scores on the particular outcome measure in question, net of measured pre-existing personal characteristics. Variables listed in each set above MYEC participation are those pre-test measures that significantly contributed to the prediction of each outcome.

* Strong trend. ** Statistically significant.

-538- 427 Table 9

Direct Contributions and Incremental Validity of Participation in MYEC Treatment Components to the Explanation of Work Outcomes Controlling for Client Background and Pretest Measures

Incremnt p for Predictor beta to R increment

Hours currently working per week (N = 198)

Counseling .06 .00 .44 Educational services .08 .00 .25 Job referral .00 .00 .99 Job placement .24 .05 .001**

Weeks worked full-time in last six months (N = 169)

Counseling .04 .00 .91 Educational services -.04 .00 .61 Job referral .0j .00 .65 Job placement .09 .00 .23

Weeks worked part-time in last six months (N = 169)

Counseling -.13 .01 .09* Educational services .34 .12 .001** Job referral .01 .00 .94 Job placement .11 .01 .13

Log-transformed current wages (N = 155)

Counseling -.06 .00 .48 Educational services .20 .04 .01** Job referral -.02 .00 .83 Job placement .28 .08 .001**

Log-transformed total earnings in last six months (N =157)

Counseling -.09 .00 .31 Educational services .22 .05 .004** Job referral .13 .01 .08* Job placement .15 .02 .07*

* Strong trend. ** Statistically significant.

-539- 428 Table 10

Direct Contributions and Incremental Validity of Participation in MYEC Treatment Components to the Explanation of Psychosocial Outcomes and Arrest Controlling for Background and Pretest Measures

Incremant p for Predictor beta to R increment

Psychological Health (N = 157)

Counseling -.05 .00 .51 Educational Services .05 .00 .49 Job referral -.01 .00 ..93 Job placement .16 .03 .02*

Interpersonal Competency (N = 156)

Counseling -J.§ .03 .02* Educational Services .02 .00 .83 Job referral .09 .00' .31 Job placement .05 .00 .55

Rebellious Autonomy (N =155)

Counseling -.07 .00 .38 Educational Services -.06 .00 .44 Job referral -.02 .00 .80 Job placement -.10 .00 .18

Re-enrollment in formal schooling (N = 141)

Counseling .12 .01 .18 Educational Services .23 .05 .001* Job referral .10 .01 .22

Job placement I .11 .01 .19

Times arrested last six months (N = 158)

Counseling .01 .00 .90 Educational Services -.08 .00 .33 Job referral -.03 .00 .74 Job placement -.13 .01 .10*

Attitude towards Education (N = 15(i)

Counseling -.13 .01 .12 Educational Services .07 .00 .39 Job referral -.09 .00 .34 Job placement .17 .03 .03*

2 Note. The increment to Rfor participation in a MYEC component shows the proportion of the variance in an outcome accounted for byparticipation in that component. A positive beta implies that persons who participatedin that component had higher scores on theparticular outcome measure in question, net of measured pre-existing personal characteristics. The results for Attitude toward Education must be interpreted withspecial caution because this attitude was not measured at pretest. * Statistically significant. -540- 429 _ Table 11

Correlations between Extent of Participation in MYEC Project Components and Ethnicity (White = 1) (N = 238)

Treatment component !ft r p

Counseling contacts -.12 .04*

Educational services contacts 7.11 .04*

Job referrals .01 _.43

Job placr.ments .16 .006*

* Statistically significant.

-541- 430 Table 12

Direct Contributions and Incremental Validity of Explanatory Variables in a Model of Psychological Health at Follow-Up

Incrment p for Predictor r beta to R increment

Statistically significant prddictors

Psychological Health (pre) .41 .31 .17 .001*

Basic Skills Pretest .26 .23 .04 .01*

Rebellious Autonomy (pre) -.21 -.20 .03 .02*

Participation in formal -.23 -.17 .03 .04* schooling (concurrent)

Concurrent variables not significant net of pre-test measures

Hours per week currently .10 .07 .00 .37 working

Weeks worked full-time .05 .04 .00 .61 last six months

Weeks worked part-time -.01 .01 .00 .85 last six months

Logged current wages .08 .08 .01 .28

Logged total earnings -.03 -.04 .00 .58 last six months

Note. The model was constructed by allowing significant pre-test or background predictors of Psychological Health at follow-up to enter the equation first, and then entering each of the concurrent work and schooling variables into the equation. Only participation in formal schooling significantly adds to the explanation of Psychological Health when pre-test measures are controlled for. Due to differential patterns of missing data, the models used to estimate the regression parameters for the non-significant regressors vary slightly from those for which the parameters are shown here. The differences are trivial. N's range from 132 to 156.

* Statistically significant.

-542- 431 Table 13

Direct Contributions and Incremental Validity of Explanatory Variables in a Model of Interpersonal Competency at Follow-Up

Incrgment p for Predictor r beta to R increment

Statistically significant predictors

Interpersonal Competency .21 .21 :05 .01* (Pre-test)

Participation in formal -.25 -.24 .06 .005* schooling (concurrent)

Concurrent variables not significant net of pre-test measures

Hours per week currently .09 .09 .01 .27 working

Weeks worked full-time .01 .03 .00 .75 last six months

Weeks worked part-time -.07 -.01 .00 .92 last six months

Logged current wages .04 .07 .01 .38

Logged total earnings .06 .11 .01 .19 last six months

Note. The model was constructed by allowing significant pre-test or background predictors of Interpersonal Competency at follow-up to enter the equation first, and then entering each of the concurrent work and schooling variables into the equation. Only participation in formal schooling significantly adds to the explanation of Interpersonal Compe- tency when pre-test measures are controlled for. Due to differential patterns of missing data, the models used to estimate the regression parameters for the non-significant regressors vary slightly from those for which the parameters are shown here. The differences are trivial. N's range from 131 to 155.

* Statistically significant.

-543- 432

,1.' I Table 14

Direct Contributions and Incremental Validity of Explanatory Variables in a Model of Rebellious Autonomy at Follow-Up

Incrqment p for Predictor beta to R increment

Statistically significant predictors

Rebellious Autonomy (pre) .40 .40 .16 .001*

Weeks worked full-time .15 .15 .02 .04* last six months

Concurrent variables not significant net of pre-test measures

Hours per week currently .14 .12 .01 .17 working

Weeks worked part-time .01 .01 .00 .85 last six months

Logged current wages .09 .08 .01 .33

Logged total earnings .16 .12 .01 .12 last six months

Participation in formal .08 .12 .01 .15 schooling

Note. The model was constructed by allowing significant pre-test or background predictors of Rebellious Autonomy at follow-up to enter the equation first, and then entering each of the concurrent work and schooling variables into the equation. Only participation in formal schooling significantly adds to the explanation of Rebellious Autonomy when pre-test measures are controlled for. Due to differential patterns of missing data, the models used to estimate the regression parameters for the non-significant regressors vary slightly from those for which the parameters are shown here. The differences are trivial. N's range from 130 to 154.

* Statistically significant.

-544- 433 Table 15

Correlations between Personal Characteristics Measured at Pre-test and Follow-up with Number of Self-Reported Arrests

Personal characteristic r N

Measured at pretest

Age -.13* 285.

Sex (female = 1) -.22** 288

Ethnicity (white = 1) .03 291

Month enrolled in program .00 294

Basic skills test -.01 179

Psychological Health .00 283

Rebellious Autonomy -.02 284

Interpersonal Competency -.04 280

Extent of participation in MYEC -.12* 287

Measured at follow-up

Hours worked in past week .11 273

Full-time employment .04 292

Part-time employment -.01 291

Log-transformed current wages .00 269

Log-transformed total .01 274 earnings last six months Re-enrollment in school -.08 259

Psychological health -.10 280

Rebellious Autonomy -.04 277

Interpersonal Competency -.14* 278

-545- 434 JVS

Appendix A

Items in Scales Used to Measure Psychosocial Development in the MYEC Evaluation

Psychological Health

Scoring Item Others see me as a loser. It's pretty tough to be me. No matter what I do, it's not going to make any difference. I feel I do not have much to be proud of. These days I feel I'm just not a part of things. I don't know whom I can count on these days. I have little influence over what happens to me. Good luck is more important than hard work.

Pretest alpha = .61 It Posttest alpha = .59 Retest reliability = .40

Interpersonal Competency

ScoringItem I know how to get along with adults. I am the kind of person who can make it if I try. If I want to, I can explain things well. Others see me as successful. I like myself. My friends regard me as a person with good sense. I feel no one really cares what happens to me.

Pretest alpha = .45 Posttest alpha = .46 Retest reliability = .19

Rebellious Autonomy

Scoring Item What I do with my time is my own business. I don't like anybody telling me what to do. I can get along just fine on my own. What happens to me is my own doing. Nobody has the right to tell me how to spend my money.

Pretest alpha = .49 Posttest alpha = .49 Retest reliability = .39

546 435