R EPORTRESUMES ED 011 169 SP 001 388 TEACHER SELECTION METHODS. BY- GILBERT, HARRY B. LANG, GERHARD NEW YORK CITY BOARD OF EDUCATION,BROOKLYN, N.Y. JUN 6T REPORT NUMBER BR-6 -1665 PUB DATE GRANT OEG- 1- 6- 061665 -1624 EDRS PRICE MF....$1.00 HC...$10.16 252P.

DESCRIPTORS- *CONFERENCES,*GOVERNMENT EMPLOYEES,INDUSTRY, PERSONNEL SELECTION, RESEARCH,TEACHER RECRUITMENT, *TEACHER SELECTION, TEACHER SHORTAGE,TEACHER SUPPLY AND DEMAND, *URBAN SCHOOLS,

THIS REPORT OF A TWO -DAYCONFERENCE ON TEACHER SELECTION METHODS, ATTENDED BY 45EXPERTS IN THE FIELD, CONTAINS13' POSITION PAPERS DEALING WITH(1) PERSONNEL SELECTION IN NON-TEACHING FIELDS,(2) PROBLEMS IN TEACHERSELECTION, RECRUITMENT AND IN VALIDATION OFSELECTION PROCEDURES, AND (3) NEEDED RESEARCH IN TEACHERSELECTION --ALSO CONFERENCE PROCEEDINGS ON (A) TEACHER SELECTIONMETHODS,(B) PERSONNEL SELECTION IN INDUSTRY, (C) PERSONNELIN CIVIL SERVICE, (D) DESIRABLE POLICIES ANDPROCEDURES FOR SELECTION OFTEACHERS, (E) PROBLEMS IN VALIDATINGTEACHER SELECTION POLICIESAND PROCEDURES,(F) NEEDED RESEARCH IN THEAREA OF TEACHER SELECTION, AND (G) SUMMATION OF THECONFERENCE. RECOMMENDATIONS INCLUDED (1) NEWAND IMPROVED SELECTION PROCEDURES,(2) BETTER UTILIZATION OfEDUCATIONAL MANPOWER, (3) SYSTEMATIC PLANNED REGIONALSEMINARS AND ANNUAL CONFERENCES. (AW) FINAL REPORT Project No. 6-1665 Grant No. 010 1-6-061665-1624

TEACHER SELECTION METHODS

June 1967

U.S. DEPARTMENT OF HEALTH, EDUCATION AND WELFARE

Office of Education Suntan of Research

U.S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE

OFFICE OF EDUCATION

THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS MOVED FROM THE PERSON OR ORGANIZATION ORIGINATING IT.POINTS OF VIEW OR OPINIONS

STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION POSITION OR POLICY. TEACHER SELECTION METHODS1

Project No 6-1665 Grant No. OEG 1-6-061665-1624

Harry B. Gilbert

(PennsylvaniaState University

Gerhard Lang

Montclair State College

June 1967

The research reported herein was performed pursuant to a grant with the Office of Education, U.S. De- partment of Health, Education, and Welfare. Con- tractors undertaking such projects under Government sponsorship are encouraged to express freely their professional judgment in the conduct of the project.

. Points of view or opinions stated do not, therefore, necessarily represent official Office of Education position or policy.

BOARD OF EXAMINERS Board of Education of The City of New York

New York, New York Board of Education of The City of New York

BOARD OF EXAMINERS*

* Isidore Bogen, Chairman

Paul Denn, Vice-Chairman

** Harry B. Gilbert

Jay E. Greene

Arthur Klein

Samuel Streicher

Murray Rockowitz

Gertrude E. Unser

*** Bernard E. Donovan Superintendent of Schools, ex-officio

Serving as acting examiner: Irving J. Gold

* deceased 41* on terminal leave ***represented by TheodoreH. Lang, Deputy Superintendent of Schools, Office of Personnel Acknowledgments

The preparation for a conference of this magnitude called for considerable cooperation and generous expenditure of energy. Many individuals are to be thanked, and it is hoped that oversights will be forgiven.

Grateful acknowledgment is expressed for the contributions of the following:

Louise C. Klein, who served as conference secretary and ably attended to the many details of organizing the conference

Jerrold Eisenberg, who conscientiously tape recorded the proceedings

Paul Fitzgerald, who performed many administrative and statistical tasks

Barbara Cassel and Shirley Robbins for transcribing the tapes, a difficult feat well done

Mary Lou Woods, for competently typing the final report

Nathan Engelberg and Samuel Cooperman, who handled many routines and photocopied strenuously.

Most of all, we are grateful to the United States Office of Education for making the conference and publication of proceedings possible. TABLE OF CONTENTS

Page

BOARD OF EXAMINERS ii

ACKNOWLEDGMENTS iii

SECTION

I. INTRODUCTION 1

II. POSITION PAPERS 4

Selection Procedures at Standard Oil Company(New Jersey) by Paul C. Baker, Standard Oil Company of New Jersey. 4

Personnel Selection Practices in Industry by Douglas W. Bray, American Telephone and TelegraphCo. 17

Personnel Selection Practices in New York CityCivil Service by Bernard Berger, Department of Personnel, TheCity of New York 26

Lifhe Relevance to Teacher Selection of Civil ServicePersonnel Selection Practices in New York State by Thomas L. Bransford, New York StateCivil Service Department 314

Selection Methods in the Federal Merit System by Raymond Jacobson, U.S. Civil ServiceCommission . . . 41

/Impact of Critical Shortages onTeacher Recruitment and X/ Selection Policies by William B. Brown, Los Angeles CitySchools

Desirable Policies and Procedures forTeacher Selection by George B. Redfern, American Associationof School Administrators 61

The Problem of Recruitment . . 69 by Albert Schiff, Detroit PublicSchools .

iCriteria: Problems in Validating Teacher Selection 1fPolicies and Procedures by David G. Ryans, University of Hawaii 81

-iv- SECTION Page

II. (continued)

VSome Notes on Validating TeacherSelection Procedures by Donald M. Medley, EducationalTesting Service . . . . 95

Criterion Problems in ValidatingTeacher Selection ,Policies and Procedures by Harold E. Mitzel, PennsylvaniaState University . . 99

Needed Research in Teacher Selection by John C. Flanagan, AmericanInstitutes for Research in the Behavioral Sciences 103

Needed Research in the Area ofTeacher Selection by Harry B. Gilbert, PennsylvaniaState University . .

III. CONFERENCE PROCEEDINGS

Greetings: Paul Denn, Board of Examiners

Background and Objectives ofConference: Harry B. Gilbert, Pennsylvania State University 111

Session No. 1: Teacher Selection Methods in theUnited States 114

. . 111 Chairman: Gertrude E. Unser, Board of Examiners . .

Teacher Selection Methods in LargePublic School Systems in the U.S.:Gerhard Lang, Montclair State College 115

Teacher Selection Methods in Sixty-twoSchool Systems Affiliated with the MetropolitanSchool Study Council: Perry M. Kalick, Hunter College ofthe City University of New York 117

Discussion 118

Session No. 2: Personnel Selection Practices inIndustry . 125 -procedures currently utilized -validity of selection procedures - relevancyof industrial selection practices to selection of teachers

Chairman: Jay E. Greene, Board of Examiners 125

-v- I

Page SECTION

III. (continued

Paul Baker, Standard OilCompany of New Jersey 125 Douglas W. Bray, AmericanTelephone and Telegraph Co. 129

131 Discussion

Session No. 3: Personnel Selection Practicesin Civil Service 140 -procedures currently. utilized -validity of selection procedures -relevancy of civil serviceselection procedures to selection of teachers

. . . 140 Chairman: Samuel Streicher, Board ofExaminers

Bernard Berger, Departmentof Personnel, The City of New York 141 Thomas L. Bransford, New YorkState Civil Service Department 142 Raymond Jacobson, U.S. CivilService Commission 145

Discussion 149

Sesdon No.4: Desirable Policies andProcedures for Selection of Teachers 159

159 Chairman:Arthur Klein, Board ofExaminers

William B. Brown, LosAngeles City George Redfern, AmericanAssociation of School Administrators Albert Sciaiff, DetroitPublic Schools .

173 Discussion

Session No. 5: Criteria: Problems in ValidatingTeacher Selection Policies andProcedures 180

180 Chairman: Paul Denn, Board of Examiners

David G. Ryans, Universityof Hawaii 180 Donald Medley, EducationalTesting Service . 187 Harold Mitzel, PennsylvaniaState University.187

188 Discussion

-vi- SECTION Page

III. (convinued)

Session No. 6:Open Discussion 193

Chairman: Paul Denn, Board of Examiners 193

Session No. 7: Needed Research in the Area of Teacher Selection 209

Chairman: Murray Rockowitz, Board of Examiners . . . 210

John C. Flanagan, American Institutes for

Research in the Behavioral Sciences . 211 Harry B. Gilbert, Pennsylvania State University 217

Discussion 221

Session No. 8:Summation of Conference 229

Chairman: Harry B. Gilbert, Pennsylvania State University 229

IV. REVIEW OF CONFERENCE

Synthesis of Discussions 236

Blueprint for Action

Evaluation of Conference

LIST OF PARTICIPANTS 2223441

-vii- SECTION I: INTRODUCTION

The current national shortage posesserious problems to those responsible for selectingteachers in local school districts. On the one hand, there isalways the danger that selectiontech- niques may screen out applicantswho would have proved to be com- petent teachers. False negatives, in the technicallanguage of the personnel man. But, on the other hand, the veryshortage serves to attract marginal prospects,and indeed, applicants who areserious- ly lacking in basic educationthemselves, let alone the ability to teach.

It becomes essential,therefore, to examine procedures in the selection of teachers, to makecertain that the techniques are reliable and valid, as well asunderstandable and acceptable to the community.

In this context, the Boardof Examiners of the public schools of the City of New York,the largest teacher selection agency in thecountry, has undertaken a seriesof steps to study the field of teacher selection forits relevance locally as well as nationally. The first step in this process was a surveyof teacher selection policies and proceduresin large public school systems in the United States. Aided by a grant from the UnitedStates Office of Education, this study has beencompleted and has been available in booklet form.

The next step is theconference, which is summarized in these proceedings. A group of experts in thefield of teacher se- lection was invited to meet for atwo-day conference to explore and make specific recommendationswith respect to needed researchand action in teacher selection. This group of 45 met from November16 to November 18,1966, at Grossinger's, New York. Again the United States Office of Education hasmade possible the conference and publication of the proceedingsthrough a grant.

The purposes of theconference were as follows:

to suggest needed researchin the area of teacherse- lection,

to offer advice regardingdesigns and implementations of needed relevant studies,

to review and discuss theresults of the survey of poli- cies and procedures ofselecting teachers in large public school systems in theUnited State's,

1 4. to consider various aspects ofexamination procedures, such as the training of raters, typesof rating forms to be used, independent vs.joint ratings, and setting of standards,

to consider the extent to whichpersonnel selection practices in fields other thaneducation can be applied to the selection of teachers, and

6. to formulate desirable policiesand procedures of se- lecting teachers, with somespecial consideration of teachers of children who areculturally different.

A number of the invitedexperts prepared position papers prior to the conference. These are reproduced and thereader is invited to study them beforereading the transcription of the con- ference proceedings. The latter present clearly theproblems as they are faced daily by thosewho are struggling to staffschools in inner-cities - today'smajor educational problem. It is a dif- ficult, probably impossible, taskto summarize fully the delibera- tions of this two-day conference.Hence, a careful reading of the proceedings may well bring the rewardof new ideas or re-inforce- ments.

Certainly one point emerges. Modification of selection procedures will not solve the problemof teacher shortage in inner- cities. The major cities of the country,all represented at the conference, employ a variety ofteacher selection procedures. Nevertheless, no city has solvedthe problem of staffing the schools in the inner-cities. The solution to this problemis part and parcel of the problemof large cities -- housing,employment, transportation, discrimination, poverty,urban social pathology. Here is a major sub-problemwhich remains for intensive study. Given the facts of big cityexistence, how can we attract and se- lect teachers who are most likelyto be successful with slum chil- dren? It is hoped that this problemwill engage the attention of some of those whowill follow the lead of thisconference.

A debt of gratitude isfelt toward the participants in this conference. They came from greatdistances, indeed from Hawaii to Brooklyn and variouspoints in between. Many prepared thoughtful position papers inadvance. All worked hard for two days and their expressions arehere recorded. Their ideas may serve to improveprocedures here or there and toadvance research in teacher selection. This would be the ultimateexpression of gratitude for their service, making acontribution toward the education of our children.

1w4 It is sad to note that Dr.Isidore Bogen, the Chairman

2 Research Committee, of the Board of Examinersand co-member of its of life. He was passed away unexpectedlyand in the vigorous prime of the overall plan a responsi5lespark in the conceptualization for research in teacherselection of which the conference was a confer- Vital part. He was looking forwardwith enthusiasm to the for ence, andindeed met with an accident ashe prepared to leave his conference, anaccident which ultimatelyproved fatal.

The conference proceedingsare dedicated to his memory. SECTION II: POSITION PAPERS

Selection Procedures at Standard Oil Company (New Jersey)

Paul C. Baker

Standard Oil Company of New Jersey

Standard Oil Company (New Jersey) recognizes the crucial importance of well qualified employees at all levels, in all loca- tions, to its continued successful operation around the world. Great emphasis is given to establishing and maintaining sound,con- sistent practices to insure that the company will continue to at- tract and to select the best of possible candidates for all open- ings. This paper examines these selection procedures, their validity and their relevance to the selection of public school teachers.

The comments that follow are not intended as a statement of official company policy nor are they intended to imply that these practices are rigidly followed in every detail in the hun- dreds of employment offices in all the countries of the world where Standard Oil has an interest. Rather they represent the collective aspiration of all our managers which gives direction to their con- tinuing efforts to improve their selection procedures.

To better appreciate the foregoing qualifications perhaps a word about the Standard Oil Company (New Jersey) and six major regional affiliates, Humble Oil and Refining Company, Imperial of Canada, Esso Europe, Esso Inter-America, Esso Standard Eastern and Esso Africa. Each of these companies in turn is composed of affil- iate companies in each of the countries of the region. In addition to these are the major producing companies, Creole of Venezuela and Esso Libya; an international trading company, Esso International; Esso Chemicals, with subsidiary affiliates around the world; anda variety of other special purpose affiliates. In view of the size and complexity of this organization a very remarkable degree of uni- formity in selection procedures can be observed.

To give emphasis and balance, to better relate it to over- all organizational needs, the selection process cannot be properly viewed in isolation, but must be seen in the perspective of a number of related functions:

- Analysis of needs: How many people, with what qualifications, are needed today, next year, five years from now, twenty years from now?

- Recruitment:Attracting the attention of pos- sible candidates to the challenges and oppor- tunities of employment with Esso.

14 -Selection': Choosing from among applicants those most likely to succeed in the company.

-Induction: The act of settling upon thecondi- TrairCremployment and transforming an appli- cant into an employee.

-Orientation:Acquainting the new employee with the organization, its goals, itsstructure, its functions, its relationships withother organi- zations.

- JobAssignment: Getting the new employee into productive work, setting goals andexpectations.

-Performance Evaluation:Judging the degree to which the employee's workperformance satisfies the requirements of the job.

The above outline is carriedto this last step, Perform- ance Evaluation,because upon it hinges thevalidity of all the others, most particularly thevalidity of the selection procedures. It may also be relevant tothe present discussion becauseof the differences that exist between theopportunities for sound evalua- tion programs in business and inschools.

Each of the above functionsmerits detailed discussion to fully appreciate all theinter-relations, but only the selec- tiori process will be dealt withhere.

There is a degree of selectionpresent in the recruiting process. Standard Oil recruits fromthose colleges and universi- ties which offer training inthose skills needed by our employees and, further, from only thosethat experience has shown toproduce a larger proportionof successful employees. There is also selec- tion implied in the choice ofmedia used to announce openingsof various types. However, selection properbegins with the screen- ing of the application blank, or aslabeled at Essc, the Qualifi- cation Record. At this step applicants areaccepted for further consideration on the basic minimaljob requirements such as age, education and training, and priorwork experience.

Having passed this firstscreening step applicants are, invited to take a battery ofqualifying tests.All test batteries have a demonstrated validity,validities ranging from.60 to .75. They all span a wide spectrumof mental measurement apdhave these elements in common:

- Atest of verbal intelligence.

5 411011Momr

- Atest of non-verbal orabstract intelligence.

- Abiographical informationquestionnaire repre- senting an extended,objectiveinterview on paper.

- Atest of job relatedjudgment -- i.e., sales judgment, supervisoryjudgment, etc.

- Atemperament or opinionquestionnaire. i.e., - Atest of job skillswhere appropriate -- typing and dictation forsecretaries.

of test battery It is interestingto note the uniformity salesman in Norway, content and test validitywhether considering a in New York. in Libya, a managerin Peru, or a secretary a mechanic in nearly all Standard Oil's research overthe past twenty years noncommunist countries ofthe world seems toindicate that a battery of tests canbecome a broadly based,carefully researched most effectiveselection aid for anyjob.

Applicants passing thetest battery are theninterviewed the interviewing by three or more managers.In some locations extensive carefully controlled,the managers have had process is validity of training and can beshown to contributeto the overall the selection process. selection pro- For a more detaileddescription of these Procedures Manual of cedures a statementfrom thc Administrative an EssoRefinery is attached. quite generally and The proceduresoutlined above are programs--there are successfully appliedin all Esso selection of implications, obviously variations. But there are a number that need to be ex- differences betweenbusiness and education, to teacher plored to assessthe applicabilityof these methods selection. hired to In all of Esso'sselection programs people are The emphasis,however, is fill a more-or-lessspecific entry job. selection of peoplewith the capacity andthe motivation upon the and to to continue to growand developthroughout their careers responsible jobs. The advance tosuccessively higher level more of qualification implication is that aconsiderably broader range considerably broader rangeof standards can beestablished and a would be true if qualifications in people canbe accepted than unchanging job. people were hiredonly for onespecific relatively job there When a person isviewed as ineffective on one to exists in most cases awide variety ofother possible jobs opportunities which an individual canbe assigned. What similar

6 exist for the ineffectiveteacher? implies that there To develop goodselection procedures be evaluated to exists some standard bywhich the procedures can determine if they aregood. In business thisimplies measuring elaborate programs are de- the quality of thepeople selected, and veloped to provide thiskind of data againstwhich the selection much work to be done program iscompared. Although there is still performance evaluation in the developmentof better more accurate has been programs, it still appearsthat very much more progress exist for made in businessthan in education.What opportunities such criterion researchin the schools?

7 SELECTION PROCEDURES

(From the Administrative Procedures Manual ofa refinerr.)

Introduction

To aid management in its function of recruiting, select- ing and placing the best possible qualified men in existing open- ings, the Employee Relations Division has established a set of pro- cedural guides which call upon the broadest possible resources of the staff and line organizations of the Company.

The eight steps of the complete selection procedure and the departments responsible for them are:

1. Announcement of openings - Employment. 2. Application Screening- Employment. 3. Testing - Training. 4. Medical Examination - Medical. 5. Interview Committee- Employee Relations and Line Management. 6. Security Check - Plant Security. 7. Former Employer and Credit Check- Employment. 8. Summary. Evaluation- Line Management and representa- tives of all the above.

Associated with each of the above steps are detailed pro- cedures and a. set of qualification standards. The procedures are fixed; the standards vary, first, with respect to the specificopen- ings; second, with respect to the status of the applicantas a present employee, former employee or a new employee.

For present employees selection standards may be lower than for new employees.

The reasons for this are:

1. For present employees such factors as work experience, job success, motivation, flexibility, suitability as an employee are already a matter of record.

2. It is the companyts desire to offer present employees continuing opportunities and challenges.

3. When more applications are received than there are openings to be filled it is possible and reasonable to set higher standards for new applicants, standards which will reduce to a minimum the probability of failure of the new employee.

8 in the Following is a detaileddescription of each step for the 1959-1960 se- selection procedure. The standards adopted standards are summarized lection programs andthe reasons for these in an appendix.

Announcement of Openin additional em- After Management hasdetermined a need for department, this needis made ployees for aparticular operation or order. known through aseries of announcementsin the following the several announcements mayvary from amatter The time between the spe- of hours to severalweeks, and theirorder may vary with cific openings. of 1. To Management Personnelthrough normal channels communications and theManagement InformationBul- letin. discussions with appropriate 2. To the Union through members of Management. and the Esso 3. To Employees throughthe Daily Bulletin News.

who have been laidoff within the 4. To Former Employees past year.

The objectives ofthese announcements are: personnel and to enablethem to 1. To inform Management employees and the answer questionsfrom and inform general public. advise 2. To inform all employeesof the need and to the pro- them of the requirementsfor eligibility and cedure to follow to applyfor transfer.

employees with thesituation inso- 3. To acquaint former far as it may applyto their recallrights or re- employment possibilities.

general, state as The announcementsof openings, in clearly and simply aspossible:

1. The nature of theopenings. apply and where toget addi- 2. Instructions on how to tional information. be 3. Specific requirementswhich applicants will

9 expected to meet such as:

a. Have certain prior work experience. b. Have a certain educational level. c. Pass a medical examination. d. Pass a selection test battery. e. Be acceptable to interview committee.

The statement of requirements will encourage only those potential applicants to apply who meet the requirements and will avoid unnecessary confusion and wasted effort by employment person- nel and by large numbers of obviously unqualified applicants.

Application Screening

Screening is the process of separating and eliminating from further current consideration applicants for employment or transfer who do not meet desired standards. It occurs during each of the eight steps in the employment procedure. This section deals with the preliminary screening of applications prior to that accom- plished at other steps in the selection procedure.

Employment maintains an open door policy toward the pub- lic and will accept applications "from anybody at anytime." Most applicants do not request an interview, but one will be arranged for anyone who desires it. The Employment Interviewer reviews the application, explains our procedure and current employmentoutlook and usually makes a note on the application of hisover-all impres- sion of the interviewee. The application is retained for one year for consideration in the event openings should occur.Unless re- newed, it is then destroyed. This also applies to applications of present employees for transfer.

Initial screening of employee-applicants fortransfer or of former employee applicants is usuallyaccomplished by examina- tion of seniority and other personnel recordsand by interview. These people sometimes voluntarily withdrawtheir application after an explanationand discussion of the openings and requirements.

Screening of applicants for "new employment!! todetermine which applicants should be invited to taketests is accomplished largely by inspecting application forms.Desired standards are de- termined first. The criteria used in this preliminaryscreening will vary with the type and number ofopenings and the availability of qualified applicants. They may include such factors as:

1. Age.

2. Education and training.

3. Prior experience.

10 4. Location of residence.

5. Relationship to employees or annuitants.

6. Physical proportions (height and weight).

Testing

The purpose of the testing program is to collect from the applicant by means of group administered paper-and-pencil tests such supplemental information regarding his mental abilities and attitudes as will be useful to management in deciding if the appli- cant is likely to succeed as an employee.

The kinds of test measurements obtained and the questions they help to answer are:

1. Intelligence - can the man learn to do the job? - will he be able to keep up with changes inthe job? - can he beexpected to progress to higher jobs? - will he show soundjudgment in crucial situations?

2. Educational Achievement - does the man have the basic education upon which to build? - does his achievementconform to his years of school- ing and his intelligence? - does hisachievement show that he is highly moti- vated?

3. Attitudes - does he have the attitudes, opinions, and personal history that have been shown to be character- istic of our successful employees? -does his judgment about himself and other people agree with the principles of human relations prac- ticed in the Company?

4. Job Knowledge - does the man know the principles and theory of his craft? does he know the standards of practice of his craft?

The tests comprising a particular battery and the passing scores on these tests are prescribed by management in terms of the needs of a specific selection program.

Passing the test battery qualifies the applicant for con- sideration in succeeding steps in the selection procedure.

To pass the selection test battery means to achieve a cer- tain minimum score on each test in the battery. Passing scores are determined by considering:

11 1. The content of entrance training programs. 2. The norms of present and past incumbent groups. 3. The job requirements, present and future. L. The number of applicants. 5. The status of the applicant: present, former or new employee.

The administration of the testing program is carried out by the Training Department.

1. Testing periods are scheduled both on week days and on Saturdays. Applicants are notified by the Employ- ment Office of the optional testing periods open to them. They are asked to indicate in which period they will take the tests so that preparations can be made for them.

The test battery is administered to small groups of from 15 to 25 applicants in a room. Several such small groups may be tested simultaneously.Every precaution is taken to insure uniformity in test ad- ministration from group to group.

The tests are scored immediately upon completion of the testing. Rosters of the test scores are prepared and distributed to the Head of the Employee Relations Division, the Head of the Employment Department and to members of the Interview Committees. Passing scores and individuals achieving these scores are indicated on the rosters.

Applicants failing to pass the tests are notified by the Employment Department. Present employee appli- cants are notified through their Department Heads. Those passing the tests are notified of the time and place to report for a medical examination and an in- terview by the Interview Committee. Other than an indication of passing or failing the tests, the re- sults are not, in general, discussed further with the applicants. Present employee-applicants may discuss test results with their department head.

In the final Summary Evaluation step of the selection procedure the test results for each individual appli- cant are reviewed together with data collected in the other steps of the selection procedure. A member of the Training Department is present for more detailed analysis and interpretation of tests scores as they are related to other. data..

12 Medical Examination

The purpose of the medical examination is to determine and evaluate the general health status of the applicant as it is related to his ability to perform the job for which he is applying. Because employments are made on a career basis, factors that may affect future health are considered.

1. Appointments for the medical examination are made by the Employment Department. As a convenience to new applicants and former employees, appointments are made for the same day as for the interview.

For present employees applying for transfer, a medi- cal history is available. They are examined only if additional information is needed.

Medical standards for a job are determined by considering the physical, emotional and psychological demands of the job. The degree to which an applicant meets these requirements is reported in termsof:

1. Over-all physical rating on five-point scale. 2. Disabilities or physical limitations. 3. The probable future state of applicant's general health.

The data reported by the Medical Department are evaluated along with data obtained in other steps of the selection procedure in the Summary Evaluation step. A doctor is present at the final summary evaluation to advise those making the decisions.

Interview

Each applicant is interviewed by a three-man committee. The purpose of this interview is to gather data about the applicant that are available only in a face-to-face situation. The types of questions for which answers are sought are

1. How well does he express himself?Listen? 2. Does he show self-control? 3. Is he enthusiasti, highly motivated? 4. How well does he think? 5. Does he have the ability to learn? 6. Will he be able to work wellwith. others? Will he fit into the work situation for which he is being interviewed? 7. Is he able to integrate his education and experience? 8. Will his personal appearance and manner add to or de- tract from his ability to work with others?

13 Procedure

Interviews are usually conducted by three-man committees on the basis that the evaluations made are subjective and three opinions are better than one.More judgments could be obtained by using larger committees, but having the applicant face such a large group might tend to upset him. Two members of the committee are experienced supervisors, department heads, usually, who are familiar with the work successfulap- plicants will do. The third member is usually selected from one of the Employee Relations Departments.This is because of Employee Relations! interest in personnel selection and, at the same time, provides something of an ttoutsidett evaluation.

Thirty to 45 minutes are devoted to each interview.Each interviewer records his impressions of the applicanton a form especially designed for the purpose. In order to get inde- pendent judgments, interviewers record their evaluations with- out benefit of consultation with fellow interviewers. Each interviewer then ranks those applicants he has interviewed, again without consultation. Lists from the several inter- viewers are consolidated.Interviewers then discuss their evaluations among themselves and later with other members of management. (This is discussed in more detail elsewhere in this report.)

Standards

Applicants are compared to each other and are ranked. They are then (1) compared to the best people in jobs for which they are being interviewed, and (2) compared to the interviewer's own standards based on his experienceas a supervisor. Based on these comparisons, interviewers deter- mine which people on the rank order list of applicantsare acceptable, or meet the standards.

Security and Credit Check

Experience indicates that the better employees are law- abiding and meet their financial obligations. In this regard, the best predictor of future conduct of an individual is probably the record of his past conduct. Therefore, it is important to consider past records in these fields. There are a number of private inves- tigating firms that provide this information fora fee. In the past, Dun and Bradstreet and Retail Credit have been used. More recently, however, State Police Department records have been screened and reports from the Baton Rouge Credit Bureau have been obtained. State Police Headquarters maintains on IBM equipment rec. orris of all arrests, by name, in this and a number of other states, The Plant Security Department maintains a close relationship with State Police Headquarters.Names of applicants who have passed tests and are tentatively recommended by interviewing teams are taken by Plant Security Personnel to State Police Headquarters and screened for records of arrests.Detailed records are obtained on any applicant with an arrest record.

Detailed written credit reports are procured from the Baton Rouge Credit Bureau. These reports itemize credit transac- tions the applicant has had in Baton Rouge and provide a good means of evaluating "how he handles financial obligations."

Former Employer Check

Former employers and close acquaintances usually have good ideas of how an applicant "gets along" with other employees and other people. Former employers are also usually familiar with the type of work the applicant is qualified to do and the degree to which he applies himself as well as his work habits and other traits. Experience has indicated, however, that formal or written inquiries to former employers or references have resulted in very little worthwhile information.

A more effective system for obtaining desiredinformation is to telephone former employers and inquire about the character and work habits of applicants.In general, former employers are very free to discuss the applicant on aninformal and personal ba- sis. These informal appraisals have been very worthwhile in the screening and final selection of employees.

Since many applicants prefer that their present employer not know of their intered% in employment elsewhere, our policy is to not contact present employers.In cases where an applicant has had no former employment, efforts are made to contact neighbors, professors, or some other persons who may have personal knowledge concerning the applicant's character.

Summary Evaluation

The purpose of the summary evaluation is to bring to-

. gether all the people who have contacted applicants for the purpose of making an over-all evaluation of the data collected in each of the preceding steps. In attendance at this meeting are representa- tives from:

15 1. The Employment Department. 2. The Training Department. 3. The Medical Department. 4. The Departments where the openings exist.

The applicants who were interviewed by the Interview Com- mittee are discussed one by one. At this point final eliminations are made on the basis of one or moreof the following:

1. A police record. 2. Poor credit report. 3. Poor recommendation by former employers or references. 4. Recommendation of the Medical Department. 5. Low evaluation by the Interview Committee.

All of the data assembled for each applicant are examined and discussed. Additional interpretations of the data are con- tributed by staff members. Critical impressions and incidents re- corded by the Interview Committee are evaluated and related to com- mittee rankings. From these discussions comes a ranked list of names of applicants recommended for employment. This list, to- gether with a summary of all data, is sent to upper Management for review and approval before offers of employment are made.Should the number of fully qualified applicants exceed the numberof open- ings, the best of those qualified are made offers. If there are fewer qualified applicants than openings, further efforts are made to secure more applications and the process is repeated.

16 Personnel SelectionPractices in Industry

Douglas W. Bray

American Telephone andTelegraph Company

such a Personnel selectionpractices in industry cover could be done to all wide variety ofmethods that no ample justice of them in any relativelyshort presentation. This paper, there- used widely in the fore, will be focused onthree methods which are to manage the telephone Bell System forselecting the men and women those which make business in the future. These three methods are assessment use of(1) biographical data,(2) attrition, and (3) the center.

The Bell SystemScholarship Study

Since the Bell Systemrecruits up to 2,000college gradu- attention to methods of ates per year, thebusiness has long given Starting as long selecting the most promisingof those available. been made of the relationof per- ago asthe 1920's, studies have formance in college to progressin management.These studies have Recently, however, a always shown a definitepositive relationship. made - a replication most ambitious replicationof these studies was The which substantiated thefindings of the earlierinvestigations. details of this study maybe of interest.

The Bell System CollegeRecord file containeddata cards accredited colleges covering the approximately30,000 graduates of For employed by the System atthe time this study wasstarted. various reasons, the datacards covering thefollowing groups of those graduating fromcollege men wereeliminated from the study: graduation, eve- after 1950, those hiredfive or more years after These ex- ning school graduates,and those on leavesof absence. clusions reduced the study groupto about 17,000. criterion of suc- An initialproblem was to decide on a used in this study isthe cess in thebusiness. The main criterion others annual salary earnedby a man as comparedto that earned by Before making such with the same lengthof service in the Company. comparisons corrections wereintroduced to adjust fordifferences country and between between salary levels indifferent parts of the different departments.

After these adjustments weremade the salarydistribution

17 for each length of servicewas divided into thirds. It was then possible to say for each graduate in thestudy whether he fell into the top, middle, or bottom salarythird of all the college gradu- ates in the System who had thesame length of service as himself.

An attempt was made to secure the verifiedrank in gradu- ating class from the registrars of themany colleges from which the men in the study had graduated. The needed informationwas secured on approximately 10,000 men. The men were then classifiedas to the third of the class in which theygraduated with an additional classification of those whowere in the top tenth of their graduat- ing class. A comparison was then made between these scholarship breakdowns and the salary thirds describedabove. The following table shows the results.

Rank in College Graduating Classas Related to Salary Progress

Top Salary Middle Salary Bottom Salary Third Third Third

Top Tenth of Class 51% 32% 17% 100% Top Third of Class 45 34 21 100 Middle Third of Class 32 36 32 100 Bottom Third of Class 26 34 40 100

The table reveals a decided relationshipbetween rank in graduating class and progress in the System. Fifty-one per cent of those who graduated in the top tenth of theirclaso and 45 per cent of those in the top graduating thirdwere in the top salary third; this compares to 26 per cent of those in thelowest third of their graduating classes. On the other hand, only 17 per cent of those in the top graduating tenth and 21per cent of the top third were in the lowest salary groupingas compared to 40 per cent of those who had graduated in the lowest third oftheir classes.

These results parallel quite closely the results ofan early Bell System study of thesame sort reported in 1928. At that time, 48 per cent of the men in the highest scholarshipthird were in the top salary thirdas compared to 22 per cent of the men from the lowest scholarship third.At the other extreme, 25 per cent of the top scholarship men were in the lowest salary thirdas compared to 47 per cent of those ranking in the bottom thirdof their col- lege graduating class.

Colleges, even accredited ones,vary tremendously in the

18 rigor of their requirementsfor admission and forsatisfactory aca- demic work. This factor strongly suggeststhat the relationship might between rank in collegegraduating class and salary progress be even more pronouncedif some account was takenof the quality of Quali- the institutions fromwhich the men in the studygraduated. tative rankings of colleges are,however, hard to come by. For these reasons, a specialclassification of colleges was madefor the purposes of this study. The classification was based onpub- lished materials and off therecord discussions with collegedeans possible to classify and placement directors. As a result it was The colleges as ilabove averagelnnaveragelff or nbelow average.ff following table shows therelationship between rank ingraduating class and salary progressfor men from the colleges ineach bf complexity of the these three broad groups. In order to reduce the table, the separate breakdownof the top tenth inscholarship has been eliminated and onlythe percentage of menachieving the top salary third is enteredinto the table.

Rank in College GraduatingClass and College Quality As Related to SalaryProgress

(Per cent of men in eachcategory achieving top salarythird)

Above Average Average Below Average Colleges Colleges Colleges

Scholarship:

Top Third of Class 55% 42% 140% Middle Third of Class 38 30 28 Bottom Third of Class 31 25 23

It will be observed thattaking college qualityinto ac- count does make a difference. The middle third graduatefrom the above average school isin the top salary third38 per cent of the time as compared to28 per cent for average menfrom below average however, that scholarship is a colleges. It is important to note, The top third men from more importantfactor than college quality. below average schools, forexample, have done betterin the busi- ness than thelowest third men from theabove average schools.

One other feature ofthe table is worthy ofmention, in respect to both thescholarship and college qualityclassifications. group much The performance of thehighest group exceeds the average exceeds the lowest group. moresubstantially than the average group

19 The difference between the performanceof top third men from above average schools and topthird men from average schools is 13 per cent (55 per cent minus 42 percent), but this latter group exceeds the top third men from below averageschools by only 2 per cent (42 per cent minus 40 percent). Again, top third men from above average schools exceed middlethird men from above average schools by 17 per cent (55 per cent minus 36 percent) while the latter group exceeds bottom third menfrom above average schools by only 7 per cent (39 per cent minus31 per cent).

A man's college grades are anevaluation of his most im- portant activity during college - his academicwork. It is widely believed, however, that a man's success inextracurricular activi- ties may also be indicative of his future,particularly his success in busines.,. The men in this study were, therefore,classified in- to three groups according to the extentof their achievement in extracurricular activities. The following table, therefore, adds one further breakdownto the data. The men are now classified by scholarship thirds, college quality, and intothree groups depend- ing on whether their campus achievement wasclassified as "sub- stantial," "some," or "none." Again, only the percentage of men achieving the top salary third isentered into the table.

Rank in Class, College Quality,Extracurricular Achievement and Salary Progress

(Per cent of men in each category achieving top salarythird)

Top Third of Middle Third of Bottom Third of Class Colleges Class Colleges Class Colleges

Above Below Above Below Above Below Aver- Aver- Aver-Aver- Aver- Aver- Aver- Aver- Aver- age age age age age age age age age

Campus Achievement:

Substantial 67% 47% 50% 52% 38% 34% 37% 25% 26% Some 61 44 37 42 33 24 32 24 18 None 47 37 37 35 28 25 31 24 20

.211,1 table demonstrates thatextracurricular achievement is related to salary progress. In all nine scholarship and college quality groups (the columns of thetable), those with substantial achievement have fared better thanthe other men.Extracurricular

20 If achievement is somewhatcompensatory for lowerrank in class. considered, for example, only men fromZo717xerage colleges are achievement have fared a middle third menwith substantial campus achievement (S2 per cent little better thantop third men with no 47 per cent). Bottom third in the top salarythird as compared to colleges and withsubstantial men, however,from above average salary third only37 per cent campus achievementhave made the top consideration of a man's campus of the time. In other words, probable success inthe achievements is helpfulin predicting his his System, but it isnot by any means asstrong a predictor as scholastic achievement. support the conclusion The findings ofthe study strongly predictor of progress that scholasticachievement is a substantial There can be noquestion but in managementin the Bell System. succeed more in theirobjec- that collegerecruiting efforts will into the businessthe more tive of bringingcapable future managers class as a criterionof they emphasize rankin college graduating employment.

Some CautionsAbout BiographicalData

Of late there hasbeen a resurgenceof interest among biographical data toproduce certain psychologistsin the use of Some of this in- of suitability forparticular jobs. an index to the present terest and the studies onwhich it is based seem two types of danger. One of these isthat the writer to contain home biographical items maybe prejudicial,such as the size of occupation, lived in, the numberof books in thehouse, father's must be made to seethat bio- It would seemthat every effort etc. informally be free ofsuch graphical data usedeither rigorously or although In our ScholarshipStudy, for example, implications. between collegequality and there was somestatistical relationship policy is to ignorethe performance in thebusiness, our recruiting doubt that college question of collegequality. There seems little because, due toselective quality emerges as apredictive factor who go to someinstitu- college admissionpractices, the students the students who goto others. tions have moremental ability than to our recruitersthat a general We have,therefore, recommended all college graduatesin the em- mental abilitytest be given to is on the individualrather ployment process. The emphasis here which he has attendedsince it is than upon theparticular college college is based onfactors clear that manystudents! choice of other than theirmental ability andmotivation. current trends in so- The other dangerin some of the that some of theitems are not, called biographicaldata blanks is in fact, biographical but are really personalityand attitude ques- tions. Recent Congressional hearings have emphasized thepoint that whatever the predictive value of such questionsmay be, there is strong public resistance to their being part ofthe employment process.

Attrition

Although the use of objective measures like rank in col- lege graduating class and general mentalability test score defi- nitely do aid in the selection ofa more promising group of college recruits, they are like all other selection methods- fallible. In our Initial Management Development Program the recruit is told that his first year in the business will bea serious trial period. He is informed that he will be givena challenging assignment under the supervision ofa specially trained supervisor who is alreadya member of middle management and that he will beregularly and rig- orously appraised as he carries out his work. The recruit is fur- ther told that if he does notappear under this scrutiny to have the high potential which he was thought to have atthe time of em- ployment that he will be told this and asked to leave. -It is sig- nificant in this program that the supervisors to whomsuch recruits will report are especially selected and especially trainedfor this aspect of their job. Such an important evaluation is not left to just any supervisor and even those selectedare worked with regu- larly to assist them in theproper observation and appraisal of their young charges.

The Assessment Center

In 1956 we started a long range longitudinal study of young business managers which we called the Management Progress Study. As a major research tool in this study, which isnow in its eleventh year, we developed a business assessment center. In the course of operating this center in the research study, it attracted the attention of several department heads whosaw it as a possible method for improving the selection of first-levelsupervisors. Certain modifications in the methodwere made to make it more ap- propriate for the actual selection of people, anda center devoted to this purpose was opened in the MichiganBell Telephone Company in 1958. This first application was successful and today mostof the Companies in the Bell System have assessment centersto process both male and female employees whoare candidates for management.

The assessment center method involves (1) definition of the qualities relevant to performance in the job; (2)selection or

22 construction of techniques for eliciting behaviorrepresentative of these qualities; (3) systematic observation oftho resulting be- havior by trained observers intimately familiar Ymithjob demands, leading to (4) judgments of the candidate'sstrength in each qual- ity and in overall promise.In our case, observation of the job and intensive discussions with those whosupervised the first- level job led us to select some 20 qualities(or variables).of im- portance in first-level management. These included such character- istics as leadership ability, administrative skills,decisiveness, independence, etc.

The major techniques at the assessment center are abusi- ness game, a leaderless groupdiscussion, an individual fact- finding problem, an individual administrativetask known as the In- Basket, an interview, and a few paper-and-penciltests of knowledge and mental ability. Assessees are brought together in groups of twelve for a period of approximately two-and-one-halfdays. The assessment staff consists of seven or eightsecond-level super- visors, who have been first-level management themselvesand who now supervise the first-level job, and a third-leveldirector. They are given up to three weeksof training before beginning, assessment work. After writing detailed reports about theirobservation of the candidates in each of the techniques, thestaff meets to dis- cuss each candidate in turnfor a period of one to two hours. Dur- ing this discussion, each candidate is rated on the20 character- istics around which the assessment is focused and,finally, rated ar to his or her degree of promise for afirst-level management job.

Follow-up studies of the assessment process haveshown that the method does, in fact, select a higherproportion of better-than-average first-level supervisors and also aconsiderably higher percentage of those who have potential for stillfurther ad- vancement. In addition, the method has proven to be highly ac- ceptable not only to management but also to thoseassessed.

The management assessment centers describedabove have processed well over 20,000 men and women in theBell System. All of these candidates have been those who have alreadybeen employed for at least several years by the business. Recently, however, we have developed assessment methods for the selectionof salesmen. In this case, the process takes lesstime - a day-and-a-half. This sales assessment center has been applied tocandidates for employ- ment and it has been found possible to conductassessment on a pre-employment basis.

23 Relevancy for Teacher Selection

The possible relevancy of the first two methods discussed above - biographical data and attrition - to teacherselection is obvious and it would seem that no detailed discussion is calledfor. One major problem, however, in using biographicaldata would be the necessity for it to be validated against performance on theteach- ing job. I gather, although I am no expert in this area, that cri- terion measures are not easy to come by and the lack of them has, in fact, been one of the major roadblocks in the path ofimproving teaching selection. I would like to return to this problem later, after I have discussed possible applications of theassessment center.

Two major components of successful teaching would seem to be knowledge of subject matter and the possession of those personal qualities essential to imparting that subject matter to students. Assessment methods could certainly be devised to measure such per- sonal qualities. (It goes without saying, that achievement tests could measure subject matter knowledge.) Many of the qualities which we assess in the management assessment centers are relevant to the job of teaching. Among them are range of knowledge and in- terests, organizing and planning, personal acceptance and impact, interpersonal sensitivity, oral communications skills, ability to influence others, behavior flexibility, and so forth. It is im- portant to note that the assessment techniques designed to reveal these qualities do not, for the most part, require job knowledge nor do they resemble the job. They, or techniques like them, could be applied not only to the job of supervisor but also to thatof salesman, research director, army officer, or teacher, to name only a few.

The evaluation of people by means of such a technique is far more defensible and acceptable than the methods depending upon biographical items or ordinary aptitude measurement.The reason for this is that the assessment technique is similar to that of an achievement test rather than an aptitude test. It inquires whether the candidate can perform the job-like tasks before him not whether he has the aptitude for such tasks. If one fails, it is possible to make some developmental effort and try again. In the case of dead biographical facts or aptitude test scores, the individual has no recourse.

There may be impressive practical difficulties in apply- ing an assessment center to applicants for teaching positions at the time of employment. Leaving expense aside, it would be impos- sible to staff up to assess all the candidates who would have to be assessed in the short period of the year in which recruiting takes place. It is for this reason that we do not use the assessment center in regular college employment. It might be possible, how- ever, for prospective teachers to be assessed toward the end of their teacher-training, possibly by the training institution, and the results made part of the placement information provided by such institutions. It would also be possible, and this seems more fea- sible, for the teacher to be assessed during a probationary period on the teaching job. The assessment results could then be con- sidered along with observations of the teacher in the classroom and other relevant information in making a decision as to whether to re-employ the teacher in the following school year.

Now to return to the question of criteria against which to validate such selection devices as biographical data, tests, etc., I would like to suggest that the assessment center can serve also as a criterion. I understand that in spite of many decades of effort, no satisfactory criterion of teacher performance has been found. Although it certainly would be preferable to have a good measure of actual performance in the classroom, improvement of teacher selection methods should not wait forever upon the develop- ment of something which may never materialize. As an alternative, one might accept as a criterion the degree to which the teacher possessed those abilities which teacher trainers, supervisors of teachers,and teachers themselves would agree were relevant to be- ing a good teacher. You may remember that in our case we decided upon twenty such abilities in the managerial field.Assessment center procedures would certainly reveal the extent to which teachers did have these teaching abilities and would also certainly make it possible for teachers to be sorted into four or five groups as to their overall effectiveness. This classification could then be used as a criterion against which to validate selection instruments,

25 Personnel Selection Practices in NewYork City Civil Service

Bernard Berger

Department of Personnel The City of New York

Procedures clinislartLillatl

The selection methods used by the NewYork City Depart- ment of Personnel do not differ greatlyfrom those used in other civil service jurisdictions. Compared to industry, civil service selection methods tend to be more elaboratein form while more cir- cumscribed in application for actualselection. The latter stems from legal requirements that civilservice selection methods must be competitive in nature andreviewable by outside authority. These conditions force the use of alinear scale for overall evalu- ation of candidates rather than amulti-dimensional analysis.

The requirement of "reviewability"necessitates the use of relatively self-explanatoryrating scales with considerable "face validity" and the production ofreviewable records such as written test papers, rating guides, taperecordings, and work sam- ples. Further, as the courts have taken it uponthemselves to de- termine the acceptability and value of answersto questions and the level of performance before oralboards, we have been forced to shift our emphasis, to some extent,from the development of methods useful for selecting the best applicantsto the development of methods which will not be challengedby the courts. However, with- in the narrow range allowed bythe law and the courts, we use many selection methods.

In each case described, ourtests are developed by exam- iners in the central personnel agencyworking from detailed job descriptions in consultation with peoplefrom the employing agency and with subject matterspecialists.

1. Training and experience tests are usedto reduce the number of unqualified applicants, toinsure that those candidates who pass the other parts ofthe examination are able toperform the duties of the position andto predict comparable ability or competency onthe basis of historical evidence.

The requirements vary fromuhigh school graduation" to a Ph.D. in aspecified field plus X years ofspecified experience. In some cases there are fixedrequirements which must be met. In others, there are manydifferent

26 test may either sets of equivalents. The score on the numerical rating. be"qualified" or "not qualified' or a numerical rating The rating scalesused to obtain the and years may includein addition tocredit for degrees factors as qualityof of experience,evaluation of such indicated performance in previousjobs and creativity as statements, by nature ofpublications. The candidate's checked to the where these willaffect his rating, are cases, field extent warranted. This may mean, in some investigations by teamsof expert examiners. the only tests. In few cases,training and experience are positions This is morefrequently the casefor high level where it is necessaryto recruit aswidely as possible, possible, and and to completethe process as soon as of training where an evaluation canbe made on the basis employment. These unassembledexaminations and previous candidates to cometo have the advantageof not requiring to a formal test New York City orto subject themselves the selection process and to permitthe completion of within days afterapplications arereceived. determining abil- While the use of anexperience test for employment test, isthe ity is probablythe oldest form of acceptable compe- most useful form todetermine minimum primarily tence in a restricted rangeof occupations apprenticeship, and has skilled crafts whichrequire an it useful in the other advantagesnoted, we have found most cases only formaking grossdiscriminations among candidates. We tend to useonly the mostobjective fac- fixed coarse scale tors (schoolcompletion, etc.) and a ratings may be by means of whichsuperficially reliable the equiva- obtained. Once one raisesthe question of different schools lence of different coursesof study in it becomes impos- and other questionsof similar nature, sible to develop arating scale, letalone obtain reliable ratings.

to the use ofexperience to predict Problems relating in the se- future performance wererecognized very early po- lection of publicservants. During the middle ages, trained and destas, roughlyequivalent to city managers, selected on the experienced in cityadministration were usually hired, basis of theirrecords. However, they were but as leaders of ateam of adminis- not as individuals know trators which wentalong with them. We still don't

27 how to measure an individual's contribution to the suc- cess of a complex program. Unfortunately we are unable to make general use of the experience of the Italian city states.

2. Performance tests vary from testing a plumber's ability to wipe a joint and a welder's ability to make an over- head weld which will resist a given number of pounds of pressure, to the group oral type of combat or the ability to interview and calm an upset parent.We have also ex- plored the use of the in-basket tests and other simula- tion techniques.

Where the test situation accurately samples the future work situation and reliably measures the ability to per- form the sample work, as in the selection for skilled craftsmen, typists, and stenographers, we have been reasonably successful.

We are not as sanguine about our other tests of this type - particularly in supervision and management. A major reason why we are floundering is that, as indicated previously, we neither really know how to measure mean- ingfully management performance nor do Tie have the people who could apply the measures we have developed. The people who could use the measures objectively can't be- cause they don't knok the person's work performance well enough. Those who do, know the person too well to be objective. One advocate of the "in- basket test," frus- trated by the fact that the ratings of observers of managerial performance are neither reliable nor highly correlated with ratings on the "in- basket test," has de- cided that the true measure of managerial ability is performance on the test and not on the job. On this basis, all tests would be valid. But then, this is the assumption that researchers in the training field make when they measure change in behavior resulting from training by change in test performance.

3. We make greater use of paper and pencil tests than of any other selection method. In form, they.include just about every test type used anywhere. In content they range from simple one hour tests of literacy to tests running over two or more days attempting to determine how well candidates would perform, as the written test for Chief of the Fire Department.

28 Until July 1 of this year, almost all of the tests were released to the candidates. This made studies of diffi- culty, reliability and validity of little value.As the test questions were made public, "face- validity" was of paramount importance. The use of items not obviously job related although similar to those used in standardized tests for the occupation has on occasions resulted in at- tacks by employee groups, citizen groups, and members of the City Council.

14. We also conduct medical and physical tests. A qualifying medical test is part of every open competitive examina- tion. Physical tests are used in selection for positions which require extraordinary physical effort.

5. A probation period, which varies from three to eighteen months depending on the job, is part of the selection process. Some agencies make effective use of this tool while others do not use it at all.

Validity of Selection Procedures

Before discussing validity of our selection procedures, it would be useful to describe our goals. In New York City as in other jurisdictions, civil service tests were introduced to take the se- lection of public employees out of the hands of the politicians. Withuut doubt, where used, our selection methods do this. Second, the selection methods must be honest. That is, no one should be able to buy a position on a civil service list. Except for a fewer:' than a half dozen minor instances over a period of over 80 years, we have been successful in accomplishing this. Third, the tests must convince the candidates and the community that they are fair. That is, that they don't favor one identifiable group over another in ways which are obviously unrelated to valid prediction of job per- formance. Although we have reservations about this, the community as a whole seems to be satisfied. And finally, the test should distinguish, in rank order, ability to perform the specific job while at the same time selecting on the basis of potential for ad- vancement. As there is no possibility that universally acceptable criteria for job performance and potential for advancement can be established and successfully used for most of our jobs, it is ob- vious that we cannot measure the validity of the selection methods for the achievement of this objective.

Even if we change this goal merely to increase the proba- bility that "competent" employees will be selected, evaluation is appointed - either they not possible. As not all candidates are fail the test or are notreached for appointment orrefuse to take the job - it is not possibleto compare the abilities ofthose who pass the testand those who do not.Despite these difficulties, we do carry on test evaluationactivities. feedback 1. We try to get fromoperating agencies as much as we can onthe performance of theemployees selected. Prior to the preparation of anytest, we try to get in- formation about the performanceof employees selected by the previous test. As might be expected,supervisors tend to be most satisfiedwith the skills of the crafts- men selectedby performance tests.We recognize that even if agencyevaluation of performance werevalid, the level of this performance maybe more a function of the nature of the labor marketthan the selection method.

2. We have made and willmake routinely, now that our exam- inations are not released,studies of item difficulty, item - sub-test discriminatingvalue and all other sta- tistics which can be derivedfrom test results. This has made it possible to comparethe characteristics of our tests withthose of standardized tests. We have, for example, on severaloccasions used the.nation-wide test for Beginning SocialWorkers prepared by the De- partment of Health, Educationand Welfare and compared the statistics obtainedwith those resulting from the use of testsprepared by our staff.

For 3. Some agencies have agreedto study test results. example, the Department ofWelfare made a study of the Case Workers appointed over atwo year period to uncover relationships among such variables astest score, job performance, race, collegeattended, and turnover. On the basis of these results, we weregoing to do away with the written test. For a number of reasonsrelated not to test-job performancecorrelation but to the other goals described previously, we wereunable to do this. However, we arecontinuing our studies. Some weeks ago the Department of Welfareappointed approximately400 Case Workers on atemporary basis pending therating of the examination and thecertification of the list. The Department of Welfare has agreedto keep close watch on the performance of theseemployees to determinewhether or not there are anyrelationships between passing or

30 failing the written test and the other factors listed. For the first time, to my knowledge, in a civil service setting we will be able to compare the performances of all who passed a test with that of all who failed.

4. While technical validation studies are not feasible or profitable over the entire range of jobs for which we select, a rational questioning of all our assumptions and procedures is. At this moment we are examining our clas- sification to determine, series by series, whether or not all classes of positions in a series are needed in the formal Civil Service sense of conducting competitive ex- aminations between successive classes. If we conclude that they are not, we may dispense with civil service selection between some.

We are also reviewing our experience requirements. It may be that our requirements have less to do with job ex- perience and more to do with maintaining, neat monotonical- ly increasing progression between successive classes.

5. Factors unrelated to the test may influence the kind of person selected more than the test itself. For example, not all people who file for jobs take them when offered. The number who drop out of the market increases with the length of time it takes to make a solid job offer. How many are lost? Who are these people?How would they compare with those who are finally employed?

Our Case Worker experiment has already told us that where less than 60% accepted job offers four to six weeks after they take the test, 95% accept on the day of the test. The nature of the job offered and work conditions obvi- ously play important roles in self-selection. We are studying our technical and profcssional jobs to remove elements which could be performed by persons with less training. We hope to reduce our needs for skills in short supply and to attract more competent people to more challenging jobs.

The labor market is a crucial factor; in some areas, we are looking for more people thanthere are in the market. In addition to job restructuring we have instituted for- mal trainee programs. For our own staff we no longer

31 seek for people with experience or traIning inpersonnel administration. We recruit capable college graduates and train them to be personnel managers.

Relevantof Our Selection Practices to Selection of Teachers

How relevant are these practices toteacher selection? While most of our practices are directlyapplicable in the recruit- ment and selection of teachers, it would not payto discuss all of them as many are already in use.

1. The time gap between recruitment and job offermust be re- duced to as close to zero as possible. In the school sit- uation this means offering college studentsteaching jobs while they are still in school. In addition, use should be made of the "walk in" technique which weand other civil service jurisdictions have used successfully. This technique should be coupled with a selection methodwhich would permit immediate selection or rejection.

2. The recruitment area should be broadened as far aspossi- ble. Some ways in which this could be accomplished,in addition to increasing recruitment efforts, are:using unassembled tests for selection of specialists;having other jurisdictions conduct tests; and acceptingteachers qualified by other jurisdictions, perhaps subject to a probationary period.

3. Stop releasing tests. This will make possible the im- provement of selection instruments throughevaluation of test results, the reduction in the cost of theselection program, and the holding of morefrequent tests.

Li. Recruit college graduates without special trainingand education and provide the needed training afteremploy- ment. Programs such as Head Start may make itpossible to provide training and experience during summer vaca- tions which might be given graduate credit. Employ those who are good teachers in this program without further examination.

5. Review all elements of the selection processand elimi- nate or change those which are not demonstrablyhelpful. Those tests which are not reliable should bediscontinued or sharply curtailed.

While I have not studied the matter closely, it seems to me a priori that the chancefactor in classroom tests is

32 too great for comfort. It might be better to use a one- term probationary period (perhaps as an assistant teacher) during which the new teacher would be under close class- room observation and supervision and would be helped to make the grade by an experienced teacher especially as- signed for this purpose or by a supervisor.

Expend greater efforts on exploratory research. Attempts to evaluate and reach definitive answers at the present time is probably both wasteful and futile. It would be more valuable at this stage to spend some money to un- cover the right questions rather than answers to the wrong ones.

33 The Relevance to Teacher Selection of Civil Service Personnel Selection Practices in New York State

Thomas L. Bransford

New York State Civil Service Department

Introduction

Consideration of personnel selection practices in Ameri- can civil service is fraught with the twin hazards of overcomplica- tion and overgeneralization. One convenient generalization is that appointment or promotion is the responsibility of an appointing of- ficer in an operating agency. His selection however is ordinarily limited to a small number of eligibles standing highest on an eval- uative listing prepared by a central personnel agency for existing and prospective vacancies in a whole class of positions.

This separation of evaluative and selective processes de- rives from the constitutional doctrine of separation of powers of the legislative and executive branches of government. Civil Serv- ice laws at both State and Federal levels have been interpreted by the judiciary as a legislative restriction on executive powers. Provision of laws or regulations limiting the appointing authority to a single eligible have been held unconstitutional by both State and Federal courts. Consequently most American civil service sys- tems have adopted the nrule of three!! whereby the civil service agency affords the appointing authority a choice of three eligibles ranked highest among those available for consideration for a par- ticular vacancy. Some jurisdictions have further widened the zone of consideration, in recognition that important factors may not have been assessed precisely or that there may be differences among positions in the same class or simply in deference to managerial responsibility. For some classes of positions the evaluative process itself may be shared with or delegated to operating agen- cies, subject to procedural standards prescribed by the civil serv- ice agency. In some jurisdictions and agencies, however, the exec- utive branch has imposed on itself a nrule of one," with each ap- pointment restricted to the highest ranking available eligible. Personnel shortages in many occupational categories have made most of these practices and distinctions largely academic.

Historically and traditionally the competitive civil service procedures were designed to set up waiting lists of many candidates for a limited number of positions which weremore at- tractive than demanding. The competition contemplated was that of

34 merit, fitness (and many candidatesdemonstrating their relative patience) for a few covetedappointments. been engaged in an In recent yearsthe public service has of competing with entirely differentcompetitive situation, that other employers for afair share of professionalspecialty personnel in scarce supply and afair share of highpotential beginners and A generalists from largerand less specializedpools of manpower. and elaborate eval- change in tempo hasalso been required--leisurely continuous recruitment and uation procedures havebeen superseded by early offers ofappointment. starts with at- The personnelselection process properly sufficient numbers of ap- tracting the attentionand interest of unaware parently qualifiedpersonnel. If potential candidates are conditions and opportunities of or unimpressedwith or averse to the to improving andpublicizing of employment,efforts must be directed attractions and to reducingdeterrents. Salaries must be adequate, competitive, and pro- working conditions andfringe benefits must be fessional staff must besupported by sub-professionaland clerical assistance so that they maydevote themselves tothe professional aspects of their jobs. Thera must also bereasonable opportunities to grow and develop onand off the job.

Unless and until thepositions to be filled areable to there is no point to compete for the interestof qualified personnel elaborate evaluation andselection devices andprocedures. Civil Service Selection Programs andPractices in New York State

The New York StateCivil Service systemis concerned with and non-competitive, entrance and promotionexaminations, competitive service of many localunits in the service ofthe State and in the comprises roughly 100,000 of government. The State service alone jurisdic- positions in 3000 positionclasses. In addition, local personnel examining tions and publicauthorities use the State's filled in several services in keepingperhaps 75,000 positions hundred distinct classes.

Providing personnelexamining services inestablishing positions currently re- eligible lists to fillvacancies in these involving around quires approximately1.000 examinations annually with which the State 150,000 applicants. The number of applicants general civil service examinations areconcerned is in the same civil service but the order of magnitude asin the New York City and numbers of number and variety ofposition classes covExed

35 examinations required annually are many times larger. Compared with the total Federal civil service system, however, New York State is concerned with only a fraction of the numbers of candi- dates, positions, position classes and competitions.The personnel operations of the Federal system are kept within manageable limits by geographic decentralization and delegation of responsibility for promotion and occupational specialty examinations to Federal de- partments and agencies. Compared with the separate offices of the Federal Civil Service, Commission, Central and Regional, the New York State Department of Civil Service faces a similar total volume and variety of personnel selection problems in original appoint- ments and a much heavier load in connection with promotion examinations.

Much of the State's examining program is concerned with levels and types of personnel whose recruitment and selection have little relevance to the recruitment and selection of teachers.

Examining prospective personnel for positions requiring little formal training or for which the labor supply is ample in re- lation to the numbers needed presents problems and justifies pro- cedures presumably of little pertinence to the selection for posi- tions requiring considerable formal training and for which the available supply of applicants compared to needs is limited in quality and numbers.

The State Civil Service Department has minimal responsi- bility for evaluating candidates for classroom teaching positions. Only in correctional, welfare, and mental hygiene institutions are there such positions under its selection program. Teaching posi- tions in public schools throughout the State are under the juris- diction of the Board of Regents and outside the civil service per- sonnel selection system. The State Civil Service system, however, does include educational experts and supervisors in departments of education, State and local. The system also recruits and examines for many classes of professional personnel using methods and pro- cedures possibly relevant to the selection of classroom teachers.

Most State civil service recruiting and selection exami- nations begin with a public announcement setting forth a general description of the positions to be filled, any restrictive condi- tions of residence, age or sex, the prerequisite education and ex- perience required, the application procedure, and an outline of the nature and coverage of the various parts of the examining process.

Typically, there is a formal written test on a pre-' announced date at designated examining centers throughout the

36 at centers State of New Yorkand by specialrequest and arrangement positions or thoseinvolving outside the State. For higher level include an oral test. For interpersonal skillsthe examination may number of classesof positions there maybe a competitive a limited that required for ad- evaluation of trainingand experience beyond mission to theexamination. Promo- There may also beother parts of theexamination. always include addedpoints for seniorityat a tion examinations of service preannounced rate of so manyfractional points per year They may include pointsfor quality of job in the jurisdiction. credits are performance during thepast year. Veterans preference veteran who granted for a singleoccasion for eachqualified wartime service.Medical was a NewYork resident atentrance into military strength and agilitytests may be required. Usually and physical and experience and a there is a verificationof claimed education physical dis- check of policerecords and inquiryinto records of emotional ability, imprisonment orhospitalization for mental or disorders.

Some or all of thevarious parts orstages of the exam- hurdles with only those ination may beadministered as successive later or the variousparts passing an earlierstage admitted to the administered to allcandidates allowingstrengths on some may be Some parts parts to compensatefor deficiencies inother parts. passing, may be ratedin three distinctscoring zones, clearly clearly failing orconditionally passing. eventually com- The various partsof the examination are ranking of all thecandidates bined into anevaluative listing or each part of the ex- passing the entireexamination and surviving passing amination on whichthere is a requiredand preannounced mark. than one ordering In some examinationsthere may be more requirements of candidates, usuallyrelated to special or ranking Some candidates or optionsfor some of thepositions to be filled. options or specialtieswhereas may qualifyonly on one of these There may be two or morelev- others may qualifyfor two or more. candidates willing to be els of positionsto be filled with some test standards considered or meetingeducational, experiential or selectiVe certi- for only one of thelevels. A formal reranking or sometimes followedin fication of eligiblesfor a class, a practice not allowed in NewYork State Federal exams, U.S.and Canadian, is

37 civil service examinations, although of course appointingofficers who avail, themselves of the choice of three or moreeligibles with- in the consideration zone may in effect be rerankingsuch eligibles in accordance with their conception of the needs of anindividual position they are seeking to fill.

The written tests designed and used in State civil serv- ice examinations embody an attempt to effect areasonable compro- mise between generality and particularity. The major emphasis has been concentrated on planning, developing and refining amanageable number of reusable multi-purpose tests which in variouscombina- tions and with varying weights and standards may be incorporatedin many different examinations. Hundreds of position classes have important elements in common. Some knowledges and abilities are common to a number of classes atparticular levels. Others are common to several levels of particularoccupational families.Many classes involve such general fanctions as supervision,administra- tion, research and statistics, interviewing, investigation, or re- port writing or such basic abilities as readingcomprehension, verbal, quantitative and abstract reasoning,interpretation of tables and graphs, scale and instrument reading orspatial percep- tion.

The State's selection program for professional andadmin- istrative trainees illustrates a number of features ofpossible application to the selection of teachers. The basic requirement is graduation from a recognized college. Subject matter knowledge is not directly tested but consideration forspecialized positions is limited to candidates meeting relevant course requirements.A five-part differential aptitude battery is administeredto all can- didates using different weights and standards for groupsof posi- tions with different needs. The five parts of the aptitude test battery were derived from a battery developed forthe Federal Civil Service where weights and standards for differentprofessional classes were those which most sharply discriminateddifferent lev- els of job performance among .employees withwar-service duration appointments. An oral test to appraise ability to expressideas and to deal effectively with others isadministered on a pass-fail basis as a condition of appointment to positions ofadministrative trainee. Eligibles without postgraduate training or professional level experience are considered for trainee levelappointments at a salary of $6300 annually which matures oncompletion of a year's service and on-the-job training to permanentappointments at a salary level of $6675 under present pay scales. Eligibles with postgraduate training or professional experience or withsuperior academic records may be appointed directlyto the $6675 level.

38 The Validity ofSelection Practices practices may bedefined in The validityof selection different definitionsimply different manydifferent ways and validity of To attempt togeneralize on the methods of appraisal. of position of practices forthe whole gamut aconsiderable variety absurdity. public servicewould seem to verge on classes in the determining the appraisal of validityis limited to Even when the battery of testsand a set correlation between aset of scores on a major aspects ofactual job per- of independentobservations of the the practicaldifficulties formance in a singleclass of positions, areformidable. selection batteriesfor Broadly speaking,the validity of is limited bythe followingconsidera- a singleclass of positions tions: appli- The extent towhich they makedistinctions among a. differences in work cants which arenot relevant to performance. knowledges, abilities The extent towhich they neglect b. relevant todifferences and personalattributes which are in work performance. changes take placein the The extent towhich unpredicted c. individuals betweentheir motivation anddevelopment of workers. The evaluations asprospective and actual the more important longer the intervalbefore appointment this factor becomes. differences occur inthe The extent towhich unpredicted d. time to time orplace nature or conditionsof work from to place.

measurement or lackof reliabilityin the e. Errors of tests. inter-rater differencesin judging f. Lack of reliability or the quality ofwork performance. service are not im- The selection programsof the public to minimize someof the mune tothese limitations. In attempting limitations weundoubtedly magnifyothers. (factor f) is in When the reliabilityof the criterion of the test(factor e) is .9, the the neighborhood of.7 and that

39 highest validity co-efficient that could be expected would be .63 even if factors a, b, c, and d were ideal. When new test batteries are experimentally administered to present employeesfactors c and d are inoperative but correlations may be limited by restrictions in the range of present employees accomplished by the original selec- tion process and by accelerated turnover of both poor and superior employees.

Multiple correlation studies between selection test bat- teries and overall criteria of job performance in the public serv- ice have occasionally yielded co-efficients of correlation as high as .6 but .4 is more frequent.

When we talk of validity of selection practices we pre- sumably are concerned with considerations beyond a co-efficient of correlation between a set of predictor scores and a set of job per- formance criteria. The real value of a selection program lies in its effectiveness in attracting and securing the appointment of sufficient numbers of candidates who can and will perform required services at quality standards. This kind of effectiveness may be achieved with tests of moderate validity and may fail of achieve- ment with evaluative instruments whose statistical validity is as high as fallible criteria will permit.

In the present state of the labor market for teachers, basic efforts are required to improve the inducements and reduce the deterrents for entering the profession, to identify teacher potential and to enlarge and improve training opportunities.Until these long-term goals can be accomplished, improvements in public- ity and evaluative techniques cannot be expected to provide the quality and number of teachers needed to maintain, let alone expand and improve, our educational systems.

4o Selection Methods in the Federal Merl.t System

Raymond Jacobson

U.S. Civil Service Commission

Any discussion of selection methods in the Federal merit system, to be intelligible, must be made against a backdrop of understanding of the nature of the Federal service.

The outstanding characteristics of the Federal service are probably its size and its diversity.The executive branch in- cludes about 2.7 million people working in literally thousands of different occupations. The programs administered by the various departments and agencies range very widely.

Federal employees not only deliver the mail and collect the taxes, they also forecast the weather; control air traffic; repair ships, tanks and airplanes; investigate, prosecute and im- prison criminals; inspect and grade food; provide medical care; control plant pests and diseases; buy and sell all kinds of goods; etc. We even have a fairly substantial corps of teachers --- the primary interest of this conference. Most of the teachers in the Federal service are employed by the Bureau of Indian Affairs in the Department of Interior to provide education at both the elementary and secondary level to those Indians who are wards of the govern- ment on reservations throughout the country.

Another characteristic of the Federal service is the wide dispersion of Federal employment geographically throughout the 50 States and, to an increasing extent, overseas in foreign countries. This means that our system must provide felxibilities to permit adaptation of the basic system to the varying problems of different labor markets depending on the nature of the job to be filled and its location. Contrary to most popular opinion, only about 12 per cent of the Federal civil servants work in the Washington, D. C. Metropolitan Area.

The great bulk of the 2.7 million jobs are filled through the competitive civil service system. Because of the diversity of occupations and their geographic dispersion, as described above, it has been necessary to develop a system which, while it follows a common set of principles and broad policies, permits a great varia- tion in the ways in which these policies and principles are applied in different places and in the filling of different jobs.

The basic principles which the Federal merit system uses

141 in operating its competitive examining program arenot unique to the Federal service. They are essentially the principles which underlie most merit systems in thiscountry and in many countries. Therefore, there is no real need toelaborate on them to any ex- tent. Fundamentally, they consist of the following:

a. Publicity must be given so that a reasonable amountof information is made available to citizens aboutthe existence of vacancies.

b. Interested persons who have learned of the vacancymust have a reasonable opportunity to makeknown their avail- ability for consideration.

c. Realistic, reasonable, and valid standards ofcompetence and fitness must be applied impartiallyto all persons who make themselves available.

d. The standards must contain no test whichconstitutes dis- crimination based on factors other than competenceand fitness. This includes the absence of anypolitical test or political clearanceof applicants.

e. Selection must be from among those determined onthe basis of the standards to be most competent.

f. Each applicant should be able to learn whatconsideration was given to hisapplication.

g. Each applicant should have an opportunityto request and receive an administrative review of theconsideration given to his application.

The principles outlined above are applied, asindicated, in all of our competitive examinationswhether they be for laborer, resident youth worker in the Job Corps,shipfitter, clerk, teacher, economist, or nuclear scientist. A very wide variety of selection methods are used in our total examinationsystem. They include: (1) written tests designed to measureskills or knowledge, or to predict aptitude or learning ability;(2) performance tests to measure skill(in such disparate fields as welding ortyping); (3) evaluations of experience or educationbased on application forms, samples of work, referenceinquiries, qualifications ihves- tigation, etc.; and (14) oral examinations on a group orindividual, basis. The specific method or combinationof methods used in any

142 one examination is typically determined in advance by the process of occupational analysis leading to the establishment of the clas- sification or position structure for one or more related occupations, and, at the same time, to the qualification standards to be used in filling the jobs.

For the purposes of this discussion it will be most fruitful if we concentrate on the nature of the examination and selection methods used for filling entry level professional and administrative positions.

The Federal government hires about 25,000 people each year who are entering into the beginning levels of a professional career. Broadly speaking, we follow two quite different techniques in conducting civil service examinations for these professional entry level positions. The first technique is one which relies heavily on a written test as an initial screening device. It is exemplified best by the Federal Service Entrance Examination, de- scribed in some detail below. The second technique is one which relies exclusively on an evaluation of the education and experience record of candidates. Both techniques require extensive recruiting effort to attract into competition, as many top quality candidates as possible. We have long ago learned that good selection tech- niques cannot breathe quality into a lackluster set of candidates. Therefore, we consider recruiting a minimum essential to any successful selection program.

The Federal Service Entrance Examination

This single examination accounts for about one-half of all hires into administrative and professional entry positions. It is our best example of the first technique mentioned above, i.e., major reliance on a written test as an initial screening device. This examination is used for the filling of entry level jobs in over 100 professional or administrative occupations.

The kinds of jobs covered include personnel, budget, social sciences, administrative, supply, claims examining, writing and editing, etc. The examination process starts with the adminis- tration of a written test. This is a general abilities test de- signed to predict the potential of an individual to learn and develop on the job and to perform at a considerably higher level of responsibility after on-the-job training and development. Those who are successful on the written test, typically about 40 per cent of those who take it, are asked to submit an application indicating their background of education and experience. Special provision is

L3 the college workperformed made on "iisapplication for indicating and the grades earnedin college foreach course. by course title written who make a sufficientlyhigh score on the Those applicants showing a B aver- addition have anacademic record test and who in class are eli- standing in the upper25 per cent of their age or a Other eligibles gible at thehigher of our twoentry grades, GS-7. Applicants who haveadditional educa- are placed onthe GS-5 list. additional experiencealso may tion beyond thebachelor's degree or (Purely on an experimentalbasis, be qualified atthe GS-7 level. completely the writtentest re- this fall we arestarting to waive or is inthe upper quirement for anyonewho has a 3.5 grade average class, and who willbe willing toaccept appoint- 10 per cent of his studies will be made on ment at theGS-5 level. Careful follow-up before making it apermanent all appointeesund'r this provision feature of thesystem.) of a large The result of this processis the creation available (about 50,000 lastyear) which is then list of eligibles and agencies to differentgovernment departments for certification list would decide to fillvacancies. Such an undiversified as they job require- in matchingindividuals with specific be of little use background by ments; therefore, acareful coding ofthe applicant's the nature of coursestaken or subject matter area,based either on the rating process. This the experience, isperformed as part of test part scoresis recorded onpunch cards. information along with specific job re- When an agencyrequest for eligiblesis received, used to sort outthose quirements identifiedby the agency are most nearly matchthe eligibles whosebackground or test scores Thus, this rathermassive needs of the specificjob to be filled. highly refined fashionin order to list of eligiblesis used in a and the eligibles onthe list. relate closely thejob requirements successful in terms of This program hasbeen highly have attractedthrough intensiverecruit- volume of competition we it Commission and bythe agencies. Generally, however, ing by the where there is has proved to befeasible only inthose situations people in the labormarket and not an extremeshortage of qualified reasonably related tothe career re- where the testbattery used is There are manyoccupations in whichthe entry level quirements. among these areteachers. jobs are filledin quite a different way;

Other EntryLevel ProfessionalExaminations professional jobs Generally speaking,the entry level

44 which we fill by examinations,other than the Federal Service Entrance Examination, arecharacterized by shortage from the stand- point of the Federal service andmost employers today. The primary areas of shortsupply in the Federal Government aresuch fields as engineering, the physical sciences,mathematics, accounting, etc. The examination systemfor these entry level professionalpositions in short supply has been onewhich has excluded the use of the written test as part of theevaluation process.

In the case of this kindof examination, it is quite evi- dent that our real problem liesnot so much in measuring the rela- tive quality of those who applybut rather in getting enough qual- ified people for consideration. Therefore, the eseence of the examining and selection functionlies not in the detailed appraisal of individual backgrounds and themeasurement of relative skill or knowledge levels but rather inthe performance of a quality and effective recruiting job. The process followed therefore,for example in the case of engineers,is one by which the examination announcement specifies the minimumrequirements of the job in terms of education or experience.The recruiter, who is usually .n agency officialrather than a representative of theCivil Service Commission, has been designated as apanel member of a board of examiners and is thus authorized toevaluate candidates as well as recruit. This recruiter then tries to do asimaginative and crea- tive a job as possible in interestingpotential applicants in con- sidering employment with his particular agency.It has long been evident that in these areas recruitingis most effective when it is tied to a particular program. In fact, the most effectiverecruit- ing in many technical fieldsconsists of describing to a classof seniors a fascinating technicalproblem which the agency has re- cently solved and not even mentionthe need of the agency for appli- cants. This kind of recruiting, of course, canbe done most ef- fectively by technical people experiencedin the subject matter field for which they are recruiting. Frequently, teams are used in which the technical recruiter isjoined by a personnel specialist who is expert in recruitingtechnique rather than in the subject matter field.

In this type of system inwhich the only basis we have for relative evaluation ofcandidates is their academic record or possibly some experience, we stillface the problem of having to rank eligibles in theirrelative order of ability to do thework. In some cases, of course,this problem is not faced sinceit is not possible to have sufficient candidatesto fill the jobs available; therefore, regardless of relativequality, all may be offered an appointment. If it is necessary, however, toestablish a ranked list of eligibles we have devisedrating schedules which provide

4 for the uniform crediting ofpoints to different types of back- grounds. Thus, for example, the graduateof an accredited en- gineering college will be given ahigher score than the graduateof a non-accreditedcollege who still meets the minimumstandard. Likewise, the graduate with ahigh grade point average in his aca- demic work or with otherindication of academic excellencebased on faculty recommendation or honors,etc., will be given extra credit and brought nearer the topof the list. This type of rating schedule serves an effective purposein that it provides for a ranking that relates closelyto those qualities which havebeen proven to bepredictors of job success.

The key factors in a successfulrecruiting and selection program for entrylevel jobs in shortage categoryoccupations are (1) interesting and challengingjob opportunities, and(2) ability to make a firm job offerpromptly. The Federal civil service makes this second factor possibleby the technique outlined above;i.e., the agency recruiter UCiug d i_Leelweciexaminer can assign a rating and offer a job on thespot --- if he so wishes. If the shortage situation is temporarily orpartially relieved, the recruiter's authority to make job offers maybe limited to only those recruits who achieve above a particular score onthe evaluation of education or experience.However, even this kind of mildlimitation is rarely necessary in these areas.

Interviews

The selection method which Ihave talked least about in this paper is the one whichis commonly considered to bethe oldest and most widely used methodof all --- the interview.While we still use the formal oralexamination as part of the examination process in a veryfew situations in the Federal service sys- tem, we have been rapidlytending away from its use in thisfashion.

This is not to deny theimportance of personal qualities or attributesin the prediction of job successin many kinds of oc- cupations. We believe that we candetermine the degree to which an individual possesses these necessaryqualities better, in most cases, by acarefully designed inquiry directedto former super- visors, colleagues, etc. thanby an interview. Therefore, we have tended to put more of our resourcesinto the better design of in- quiry letters or into personalinvestigations than into improving interview techniques.

As a safety valve inthe system, we do permit agency ap- pointing officers to submitobjections to an eligible based onthe

46 results of an interview performed as part of theappointing offi- cer's right to select one of the top three names certified tohim. If the objections are based on a carefullystructured interview with adequate documentation that tends to show thecandidate to be significantly lacking in necessary personal qualities appropriately measurable by the interview, we will support the objection.This practice has saved considerable time, effort, and paper work - -- with no indication of abuse so far.

Conclusions

The above describes two quite different selectionmethods in wide usage in the Federal service for fillingentry level pro- fessional and administrative jobs requiring college level back- ground or equivalent. Both of these methods have been tested heavily in the intensive competition for quality personnelduring the last decade. On the whole, while we are constantly changing and improving these techniques, they haveboth met this test very well. The Federal service --- while not getting enough topquality talent to meet our needs is getting its fair share of what is available.

The specific application of these methods to teacher re- cruitment and-selection for a large metropolitan schoolsystem is a matter which obviously needs considerable careful study.Without knowing more about the way in which such a system operates,it would be presumptuous of me to make specific proposals asto the best combination of selection methods to be used forteacher re- cruitment. I feel strongly, however, that anythorough-going study designed to produce the most effective selection systemhas to ap- proach the problem with a hard-hitting and searching analysiswhich gives full consideration to some of the possibilitiessuggested by this paper. Impact of Critical Shortages on Teacher Recruitment and Selection Policies

William B. Brown

Los Angeles City Schools

Purpose of This Paper

This is a brief statement relatinghow the Los Angeles City Schools recently encounteredmajor difficulties in teacher re- cruitment which, if not solved, wouldhave resulted in seriously adverse effects on the pupils andthe educational program, and haw well-established policies and procedures wereadjusted in the face of the crisis to meet staffing shortages. I hope the experiences which I will report here and ourtentative conclusions will be pertinent to the objectives of this conference.

The questions we are seeking to answer are:What are the best steps to take to meet an anticipated,crippling shortage? What measures will bring thetreit results for the schools in the short period of time available?The answers vary considerably from district to district, but we hopethere are some common elements on which mutual discussion will bebeneficial.

Finally, this paper seeks to serve as anargument against too much rigidity in selectionpolicy. Policy must be broad enough and flexible enough to meet a greatvariety of teacher supply- demand conditions--general oversupply orgeneral undersupply; over- supply in some secondary fields andundersupply at the elementary level; oversupply in one part of thedistrict and undersupply in another; keen competition withsmaller, more attractive districts; or unexpected andsudden increases in demand as a resultof enact- ment of specially funded programswith local, state, federal, or foundation resources. All of these require practical,vigorous, adaptable policies which will producethe greatest possible recruit- ment results in the limited timewhich is generally available.

The critical elementary teachershortage which the Los Angeles City Schools facedthis year is an excellent illustration of the type of staffing problemencountered and the procedures in- stituted to meet the situation. The findings and conclusions are based upon our experience withthis particular shortage, and with other shortages experienced sinceWorld War II. These policies and procedures are fully developed inpersonnel guides. They are ready for immediate implementation, and aresufficiently flexible so that adjustments can be made in terms of aparticular emergency and a particular kind of teacher which maybe in short supply.

I8 Los Angeles City.lchcJls -- An Overview

A little background information regarding the Los Angeles City Schools, the second largest school system in the nation, may be in order here.

The district is responsible for the education of 13.5 per cent of Califlrnia's elementary school pupils and 17.3 per cent of those in the junior and senior high schools. When the current school year started in September, Los Angeles had a total of 367,000 pupils in 428 elementary schools, 136,000 students in 71 junior high schools, and 128,000 in 53 senior high schools. Our total enroll- ment, including adult schools and junior colleges, is approximately 794,000, and is expected to go over the 800,000 mark next February.

Our teaching staff now totals approximately 31,000 in- cluding adult education and junior colleges. Our non-certificated staff now totals in the neighborhood of 20,000.

Unlike most other major metropolitan school districts which have large studeni, populations and serve more limited geo- graphic areas, the Los Angeles district serves a sprawling area of almost 800 square miles. Its service area is roughly twice as large as the City of Los Angeles and includes, aside from the City of Los Angeles, eleven incorporated municipalities and large sec- tions of unincorporated County territory. On a north-south axis, one can travel approximately 60 miles from the northernmost school in the San Fernando Valley to the southernmost school overlooking the Los Angeles Harbor in San Pedro, and never leave the district.

The Los Angeles district serves areas of wealth, but it also serves major areas of poverty. Almost 200 of the 599 schools are located in "disadvantaged areas."

The implications and relationships of these characteris- tics to recruitment are many. I mention but one. This year, a surplus of teachers wished to work in suburban San Fernando Valley, a largely middle class, Caucasian area, whereas there were teacher shortages in some other areas such as the central city.

Teacher Recruitment Situation in Los Angeles

Since World War II, Los Angeles has experienced a short- age of teachers of one type or another almost every year. A seri- ous elementary shortage existed for a number of years during and immediately after World War II. Then, there was a gradual improve- ment in the supply at that level between 1955 and 1960, while, dur- ing this same period, shortages began to develop at the secondary level. The latter have intensified and continued for most of the past seven or eight years.

49 September, 1965, was the first year since World WarII in which Los Angeles did not encounter major shortages at theelemen- tary or secondary level except, ofcourse, in mathematics and one ..Nr two technical fields. This favorable situation was due to two major factors, an 11 per cent increase in district starting sala- ries for teachers, which brought the minimum annualpay to $6,120, close to the highest in the state; anda major change in State licensing requirements undera new credential structure. The large number of candidates was directly traced to the efforts ofyoung people to qualify under the old credential structure before thenew requirements went into effect.Had it not been for these two fac- tors, the shortages experienced during precedingyears would have existed in September, 1965. So for most of the period from World War II to the present, Los Angeles has had the challenge andre- sponsibility of meeting teacher shortages of varying degrees of seriousness and type. The problem is not unusual inour city. Rather, it is a normal situation which must be continually facedby the personnel staff.

The Los Angeles district has showna consistent annual growth, following the end of World War II, of from four to sixper cent. Since the early 1950's, we have been faced with the task of recruiting an average of 3,000 teachers annually to meet growth and replacement needs. But, in preparing our recruitment program for September of the 1966-67 school year,we were certain of one over- riding factor--this was to be no "ordinary"' recruitingyear, not that there ever is an "ordinary" recruitingyear.

For one thing, our operating divisionsgave us the task of recruiting a total cf 4,500 new teachers for this schoolyear. The unusually large demand was caused by the projected implementa- tion of several programs of a compensatory educa+,ion nature, to be funded out of state and federal resources.

Personnel staffs throughout the country are all too familiar with the problems incident to participation in state and federally funded special programs--ESEA and the Economic Opportun- ity Act, for example. These programs have added a new dimension to planning for school districts involved, generally because of delays in approval of legislation but particularly because of last minute "under- the - wire" administrative approvals from the fundingsource.

Aside from the uncertainties attributable to the imple- mentation of special programs, Los Angeles recruiterswere being asked to find the needed teachers in the face of a severe shortage of elementary teachers, the worst in manyyears, and it is to this problem that the remainder of this paper will be directed.

5o Elementary Teacher Shortage and Steps Taken toMeet It

The serious shortage of elementary teachers anticipatedat the start of the spring recruitment drive was basedprimarily upon a 33 per cent drop in the number of elementary studentteachers trained at the 12 colleges and universities in the LosAngeles area, which have been the source of two-thirds of the newelementary teachers for the District. Thus, this serious drop in enrollment, combined with a doubling of demand, pointed very clearly to acriti- cal problem facing the Los Angeles CitySchools.

Other factors which have had their effect onthe number of potential candidates available can be tracedto the implementa- tion of new credentialing legislation. These include:

1. Lack of adequate, up-to-date information inout-of-state institutions concerning California credential require- mentL,

2. Requirement of a fifth year of college and a subject matter major for full credential status,

3. Difficulties out-of-state recruits have in obtaining graduate standing in local institutions and inobtaining needed courses to meet the requirement of a fifth year.

Our normal needs for growth and replacement havebeen ap- proximately 900 elementary teachers for the fallsemester of each year. This spring we were faced with the recruitmentof an addi- tional 900 teachers to meet the following needs forSeptember:

1. Implementation of legislation which reduced class norms to 31 in grades one through three...293 additional teachers,

2. Additional kindergarten teachers to staff a districtin- crease in the kindergartenprogram...54 additional teach- ers,

3. Additional teachers for Elementary and Secondary Educa- tion Act programs...92 additional teachers(600 addi- tional had been employed the previous February because of ESEA).

4. Implementation of new state legislation which reducedthe pupil - teacher ratio to 25 to 1 in disadvantaged areas... 462 additional teachers. Intensified s rinrecruitmentprogram: A number of steps were taken by theistrict staff tomeet t e anticipated teacher shortage: elementary

1. Recruitment trips toother parts of the UnitedStates were more than doubled over those of the precedingyear; 200 campuses were visited this past springas against 80 the year before.

2. Both local campus and out-of-state recruitmenttrips were scheduled at earlierdates than in precedingyears to place the district ina more advantageous position.

3. The time for processing out-of-state recruits andmaking contract offers was cut in half over thepreceding year. This time is now five to ten days from dateof interview, depending upon whentranscripts are receivedat the Personnel Division.

14. Credential service to naw recruits was substantiallyin- creased, includinginformation and explanationof new regulations, individualcounseling, and evaluationof transcripts. This servicewas necessary toovercome con- fusion and misunderstanding concerning thenew credential structure.

The results of the,spring recruitment drivefor new elementary teachers were disappointingbut not unexpected:

1. Only 415 were recruitedat localcampuses, a drop from 600 the previous year. This 31 per cent reductiontal- lied closely with the drop in student teacherenrollments mentioned previously.

2. The number of out-of-state recruits showeda similar drop over the preceding year in spite of the expandedrecruit- ment program. Only 150 elementaryrecruits were obtained from out-of-state sources this past spring comparedwith 2140 the previous year. This 37.5 per centreduction par- alleled the reductionexperienced by otherSouthern California school districts.

An additional 250new teachers were obtained fromthree other sources: former teachers returningto the District;approved secondary teachers itsurplus fields, suchas social studies, who were found to be qualified for elementary teaching service;and a limited number of teachers from other districtsin the state. The spring recruitment drive thus had nettedthe district only 815 of the 1,800 new elementary teachers expectedto be needed

52 for the start of the fall semester. The personnel staff concluded that a critical shortage would existthis September unless emer- gency action were taken. Two steps were therefore taken:

1. A request was made to the StateBoard of Education for authorization for Los Angeles to use theprovisional cre- dential. (This credential in California is granted on an individual district basis when applicationto the State Board shows existence of a shortage andthe fact that proper steps have beentaken to meet the shortage.The credential requires the possession of a baccalaureate degree only; no teacher training isrequired.)

2. Plans were made for an intensive summercommunity recruit- ment drive to secure a minimum of600 additional qualified teachers who might be employed on regular orprovisional credentials. This drive was undertaken duringAugust and early September. (Partial relief from the overall goal of 1,800 new teachers needed for the September openingof school resulted from a delay inimplementation of one of the new state laws. This reduced the number needed by 400.)

The intensified summer recruitmentdrive was carried out with excellent support from newspapers,television, and radio. A community-wide appeal was made for qualified persons,housewives, and persons in other fields of workto respond to the urgent need for additional elementary teachersfor the district.A spe-ial ap- peal was made to former teachers andto teachers on long-term leaves of absence to return to service thisfall because of the shortage.

The response far exceeded expectationsand was better than the response to similar drivesheld in preceding years: 900 applications were received, and600 of the candidates were approved for contract assignment;300 were employed on regularcredentials, and 300 on provisional credentials. As a result of this drive,the elementary schools of the districtstarted the fall semester with an adequatesupply of new teachers.

For the second semester of thecurrent school year, which starts in February, many more additional newelementary teachers will be required, 350 for growth andreplacement, and 460 to imple- ment the delayed specially fundedstate program for disadvantaged area schools. Once again, the Personnel Divisionwill be confronted with the need to conduct a majorrecruitment program in the face of serious shortage at this level. Plans will be similar to those car- ried out during the summer monthsand will be conducted with every

53 expectation of finding all ora major part of the new teachers needed.

The qualities we seek in all new teachersare:

1. Strong interest in boys and girls, 2. Knowledge of educational principles and practices, 3. Knowledge of subject matter, 4. Command of English funclammtals, 5. Good scholarship, 6. Ability to get along with others, 7. Appropriate appearance, 8. Pleasing voice and clear speecli, 9. Ability to present ideas, 10. Alertness and maturity of judgment.

Initial employment screening proceduresare exactly the same for all teachers--regularly credentialed or provisionallycre- dentialed. It is emphasized that all of the 300 teachers hiredon provisional credentials were required to meet thesame district standards as regularly credentialed teachersapproved on a condi- tional one-year contract basis, including:

1. A baccalaureate degree, 2. A strong college academic record, 3. Strong personal qualifications for teaching, 4. Passing of two objective tests,

a. English usage b. Principles and practices of elementary education.

The only difference between these two types of teachers is in the credential requirement. The regular teacher entering service on a four-year or partial fulfillment credential basis must havecom- pleted student teaching or have had equivalent teachingexperience. The provisional teacher does not have to meet thisrequirement. To offset this lack, the Los Angeles City Schools,whenever possible, require all new provisional teachers to undergoan intensive summer training, in the district, of not less than two weeks,and to serve the first semester under the close guidance andsupervision of a master teacher. It is preferred that all new teachers successfully complete a full student teaching assignment beforestarting service. Hawever, in an emergency such as the presentone, when this is not possible for all new teachers, district pre-serviceand in-service training programs can provide much of thenecessary technical preparation.

A seven-point district training program for new elementary provisional teachers includes the following:

54 1. Those hired prior to the opening of elementary schoolsum- mer classes are required to observe and participate in these classes for a minimum of two weeks.

2. Those hired after the close ofsummer programs are given special orientation in the area offices.

3. All provisional teachers spenda minimum of tw' U. days in area induction meetings with supervisory per,44nel prior to the opening of school.

4. Attendance in a semester-long in-service trainingcourse is required. This provides orientation to subject fields, methodology and curriculum of the Los Angelesschools.

5. A permanent regular teacher :.s assigned full timeas teacher advisor to each ten provisional teachers. The teacher advisor works in the classroom with thenew teachers and provides conference time forassistance with problems.

6. Each area supervisory staff devotes specialattention to the needs of provioloilal teachers.

7. Principals give additional special assistance and guidance to the provisional teachers.

Current Problems for Recruitment Review

Each year new ideas and innovationsare evolved to help meet critical staffing problems. The accumulation and use of this experience enable a district to face future shortageswith consid- erable confidence.

It has become axiomatic, for instance, that policiesmust be readily adjustable to meet emergencies,no matter how short the warning. Also, selection procedures must be considered withinthe context of the recruitment function, for there isno value in hav- ing sound selection procedures ifno candidates will apply for em- ployment.

In "tooling" up to meet the year-by-year needs fornew teachers, it is important that recruitment planningand research provide guidelines for future action. There is often insufficient study and planning behindemergency recruitment programs. There are a number of basic questions to which we need to give increasing attention, if planning is to be more effective. Some of these are as follows:

55 1. Has the job of the teacher changedsignificantly in re- cent years?

2. To what extent is discipline becoming a moreserious prob- lem than before and whatimplications does this have for the recruitment selection process?

3. Have criteria and specifications forselection been realistically revised in recognition of anyimportant changes in the work of teachers?

4. Have teacher training institutionsbeen effectively in- formed of changes in the teacher's job?

5. Has teacher training been modifiedto meet the changes in the work of the teacher?

6. Have these institutions recognizedthe great importance of preparing teachers for service ineconomically, educa- tionally deprived, disadvantaged, orculturally "dif- ferent,' areas, and are the teacher traineesassigned to take at least part of their studentteaching in poverty area schools?

7. Have the colleges informedschool districts realistically in their appraisal reports concerningstudent teaching success in handling pupilsin poverty areas?

8. Has recruitment been adjusted to meetchanges in the na- ture of the teaching job?

9. Is a prime recruitment objective theselection of teachers who are qualified to serve in alltypes of situations in urban districts, or can the criteriabe specialized to meet varying community needs?

10. Are all possible school andcommunity resources being utilized for assistance in this recruitmentprocess?

11. Are we learning about andprofiting from the influences which lead teachers to select large urbancenters and to make a life career of teaching inthese centers; con- versely, are attempts being made toascertain factors which keep prospective teachers awayfrom urban centers?

12. Have urban districts learned how toutilize recruitment to improve the image of the large city as afine place in which to teach and live?

Need for more adequate supply-demanddata:With current means

56 of communication and data processing, monthly reports on both sur- plus and shortages should be provided t- school districts on na- tional and state levels. In Los Angeles, and in California general- ly, the current elementary teacher shortage could have been foreseen if all the pertinent data had been assembled and compiled in late 1965.

Events during the past year have made available supply- demand information completely inadequate.The war in Viet Nam with the attendant effect of the draft on teachers and prospective teachers, the civil rights problems in many of our cities, the specially funded programs to aid deprived areas--these and other influences are not readily translated into technical data. But translated they must be, and the data must be disseminated widely so that school districts and teacher training institutions may plan in advance to meet shortages most effectively.

Conclusions Concerning Recruitment Policies

Teacher shortages may not be as permanent as death and taxes, but, at least for the foreseeable future, recruiting of ade- quate numbers of well-trained teachers will be a perennial issue for personnel staffs of urban school districts.

The time has long passed when those' of us in the person- nel offices of big city school systems, particularly those cities experiencing periods of growth, can afford to "sit back" and wait for fully-trained teaching candidates to flock to our personnel offices.

Rather, we must formulate an aggressive, flexible, and comprehensive policy, which should be based upon the following facts:

1. Close working relationships between public schools, teacher training institutions, and the State Department of Education are essential to meet shortage problems. The greater the shortage, the more cooperative effort on an immediate day-to-day basis is required. A team ap- proach is essential in coordinating credential policies, teacher training policies, district selection practices, and school staff requirements.

2. A large and excellent reservoir of potentially fine teachers exists in metropolitan areas such as Los Angeles, which can be drawn upon in times of critical need. This reservoir is made up of college graduates, housewives, and persons in other fields of work, who have excellent

57

,z,i,r44ii4Whiiiiretii*Atil****X041farikWEP scholastic records in college,the personal qualifica- tions, and the background ofexperience for working ef- fectively with children. This reservoir should betapped without hesitation when it hasbeen determined that the supply of regularly credentialedteachers is exhausted.

stations can give very 3. Television, newspapers, and radio valuable help to a school districtin a large metropoli- tan area, in recruitingneeded new teachers from the ranks of housewives, recentcollege graduates, and per- sons in professionalfields other than teaching.

highly favorable. The 14. Teacher salaries must be kept minimum salary must be thehighest or close to the high- est in the state. This is essential if thelargest city area and thelargest school district areto compete ef- fectively with smaller and moreattractive suburban school systems for the limitedsupply of new teachers.

procedures for the employ- 5 The well established screening ment of regular probationarteachers cannot be allowed to interfere with therecruitment process in meetingcrit- ical shortages. At the same time, aneffective, stream- lined screening procedure mustbe utilized for initial employment on a contract basisto assure that district standards will be met. To state the matterbluntly, get the qualified recruit onthe job first, then examinehim closely for career status inthe system. An important part of the screeningprocedure is the requirementthat the formal examination mustbe passed during thefirst year of service.

6. The avenues of entry andrecruitment contact must be manifold:

a. District employment officeslocated at con- venient places within theschool system, not restricted to one centrallocation,

Interviews with recruits onall local campuses of major colleges anduniversities,

c. Out-of-state interviews atrecruitment centers and on college and university campusesin various parts of the nation.

Such recruitment requires aminimum staff for year-round operation, and amplified stafffor peak recruitment dur- ing the spring and summermonths.

58 in the 7. Employment standardsshould not be lowered even face of serious shortage. For our district, only one ex- ception to this basicpolicy is made; thatis, the meet- ing of the studentteaching requirement. High selection standards are essentialto preserve theeducational pro- gram of thedistrict. Recruitment must beintensified, salaries may need tobe raised, processingof teachers pre-service and in- may needto be streamlined, and service help to newprovisionals may need to beexpanded; but under no circumstancesshould personal andacademic standards be lowered. sound Technical credentialrequirements, which may be very 8. not be al- when the supply-demandbalance is good, should lowed to stand in the wayof recruitment in timeof short- however, age. Temporary waiving ofsuch requirements, does not warrant adistrict's lowering itsstandards in any way. staffs should 9. Resources of thesupervisory and teaching be utilized to providetraining and guidance37.)". provi- sional teachers neededin an emergency. Lack of teacher insur- training, includingstudent teaching, is not an mcantable obstacle. If shortages 10. Recruitment, above all,needs to be honest. exist only in disadvantaged areas,the community should be told this asfrankly as possible. Recruitment should meet this be undertaken directlyand specifically to shortage, seeking outthe kinds of personswho not only have strong teachingqualifications, but alsogenuine interest in teaching inthese areas of ourcities.

Looking Ahead essential factors 1. Much more needs tobe known concerning qualified teachers for in successfulrecruiting of well function large urban districts. The entire recruitment warrants greater studyand research than ithas received the to date. Also, more researchis needed concerning recruits quick- most effective waysto screen and process employment from ly so that they willrespond to offers of the urban district.

Much better state andnational information onthe teacher 2. Speedier and more supply-demand situationis required. frequent checking, bothwith colleges and publicschool systems, are needed. The current elementaryshortage is

59 to the 10 or15 a classicexample. A telephone call larger districts in thecountry in late spring orearly summer of this yearwould have revealedthat most had, or were anticipating, aserious shortage. The hurricane flag was up, the barometer wassteadily falling, the supply-demand relationship wasfar out of balance;but it studies had was"statistics" as usual. The supply-demand been closed for the year. Clearly, a major overhauland up-dating of our stateand national informationin this area isessential. freedom to 3. Large urban districtsmust have the necessary solve their ownrecruitment problems. Too many outside to the agencies areinterfering. Those who are closest recruitment and selectionof teachers and wholive with the problems the yearround and have themin sharpest fo- cus, are the onesto take action andget the results. State and federalfinancial assistanceis of increasing importance, but interferencethrough, technicalcredential regulations, such as existsin some of ourstates, is be- recruit- coming a seriousobstacle to achieving necessary study of the forces ment results. There is a need for a at work at the statelevel, which oftenbecome major problems for localschool districts incarrying out their time is staffing responsibilities. Far too much staff regula- now beingspent in studyingproposed new state them after tions for support oropposition, understanding If they are adopted, andmaking necessaryadjustments. local districts aregiven greater freedomand additional resources, thestaffing problems of ourlarge urban school systems willbe effectively andpromptly solved. One of the most The task ahead willrequire our bestefforts. valuable ways to meetit is to have the kindof pooling of experi- this confer- ence amongthe great cities of theUnited States which ence provides.

60 Desirable Policies and Procedures for TeacherSelection

George B. Redfern

American Association of School Administrators

That teacher selection procedures in publicschools vary widely is an understatement of considerable magnitude.This vari- ability is to be expected since the control ofpublic schools in this country is so highly decentralized andlocal in nature.

In general, however, selectionprocesses fall into two broad categories.One places a premium upon written and oralex- amination procedures; the other puts emphasisupon other less pre- cise criteria and reliesmore upon the oral interview. At the risk of oversimplification, the formermay be characterized by the term objectivity; the latter, subjectivity.

That one process is manifestly superior andmore effec- tive than the other has been argued formany years. During the early 1950's the proponents of "both schools ofthought" used the agenda of annual meetings of the American Associationof Examiners and Administrators of Educational Personnel)to debate the issue of examination-oriented selection proceduresversus non-examination techniques. Neither side quite convinced the other of thesuperi- ority of its point of view. The great debate endedmere in stale- mate than in victory.

The use of well-designed, time-tested and carefully validated examinations in teacher selection insome school systems is justified on the grounds of historical precedent,legal require- ment, urgent necessity orsome other compelling reason. At the same time, other systems use different procedures,justifying their use on equally logical grounds. The point is that school systemsare at different stages in time and circumstance intheir developmental growth. They reflect unique local requirements and conditions. They conform to specific demands made of them.

The assumption is sometimes made that examination-oriented selection procedures are preferable, that theyare more "scientific" and more effective in selecting good teachers thannon-examination techniques.

1Since1959, called American Association ofSchool Personnel Administrators.

61 If this assumption is valid, then it seems to follow that other systems should adopt more examination-type procedures.Why do they not do so?Is it the result of unenlightenment?Is it due to lack of professional resources and skilled know -how? Is there an insufficiency of commitment to sound personneladministration? Or is it due to other plausible reasons?

It is logical also to raise the question as to whether or not it is possible to increase the objectivity of so-called subjec- tive selection procedures without, however, completely adopting specific examination instruments.

This paper is addressed to the consideration of the poli- cies and procedures of teacher selection in the Cincinnati Public Schools where I was, until just recently, Assistant Superintendent of Administration and Personnel.Written and oral examinations are not employed in Cincinnati but care is taken to compile ttsubjective data!! in a manner that will yield eligibility scores which make it possible to rank applicants so that those with higher scores can be employed before those with lower scores. Cut-off points are estab- lished to eliminate the marginal and weak candidates.

The purpose of this paper, then, is to show how so-called subjective selection procedures may be made objective, eligibility scores established, candidates rankedand selection may be made in terms of relative rankings.

Certification - Written and oral examinations are more necessarywhen qualiMations for certification are being ascer- tained. On the other hand, however, many state departments of education grant certificates on the basis of approved patterns of preparation without requiring the candidate to take qualifying examinations.

In those instances, therefore, where state departments of education certificate teachers, it is possible to assume that some degree of selectivity has already taken place first, during the teacher education process and secondly, at the time the certificate is granted. This means, consequently, that the employing school system has some assurance that properly certificated applicants with sound preparation from accredited teacher educationinstitu- tions have already been screened and evaluated to some degree.

The relevance of the above observation is that school systems not responsible for examining for certification have cer- tain assurances about the relative qualifications of candidates without administering comprehensive testing procedures.Certifi- cation and credentials provide varying degrees of the necessary evidence for determining eligibility for appointment.

62 Preparation - While there is still considerable variation in the quality of preparation in different teacher education insti- tutions, higher and higher standards of preparation are required in an increasing number of colleges anduniversities. NCATE2 approved institutions have particularly high preparational requirements. Candidates from these institutions have been subjected to many meas- ures of selectivity. This gives further assurance and removes some of the necessity to assume that total selection is the responsibil- ity of the employing school system.

The Cincinnati Selection Plan

After applicants have been identified through recruitment efforts, a careful selection process is used to determine those best qualified for employment. The employment policy of the school system is to employ from the available applicants those best quali- fied to fill the positions which are open.

Procedure - Step 1 consists of processing the application. This involves inspecting the application form to see if it is com- pletely and properly filled out.Care is taken to make sure that the record of past employment is complete, without gaps. A com- plete transcript of college credits is required. A copy of the teaching certificate is also required if the applicant holds one. If a beginning teacher, the applicant is required to show evidence of having fulfilled the requirements necessary to obtain the cer- tificate.

Step 2 involves the processing of professional references. Evaluation forms are sent to all persons whom the applicant has given as professional references. Care is taken to ascertain if the names given represent, persons who are in a position to give responsible, recent and relevant evaluations of the candidatete qualifications. If this is not the case, contact is made with the applicant for additional references. All references are then evaluated as described later.

Step 3 is the analysis of the application. The applica- tion, references, evidence of certification or eligibility for cer- tification are analyzed to determine if the minimum standards for employment have been met or can be met.

Step 4 is the oral interview. If the candidate meets min- imum standards on the basis of academic preparation and professional references and if vacancies exist for which he is qualified, an oral interview is scheduled.

National Commission for the Accreditation of Teacher Education.

63 When the interview is held at theheadquarters of the Board of Education, every effort is madeto have at least two inter- viewers. One represents the personnel office;the other either someone from the departmentof instruction or administration. Fre- quently, a principal is invited to serve onthe interviewing team.

In campus interviews, the teaminterview is not always possible, although the practice of sendingteams on campus inter- viewing trips is on the increase. In campus interviews, the oral interview takes place before theapplication is processed and cre- dentials are received.However, care is exercised to study the available records on file in theappointments' office of the training institution.

Four factors are stressed inthe oral interview:

(a) Scope and quality of trainingand preparation (b) Aptitude for teaching and working withchildren (c) Dedication and desire to teach (d) Character, personal fitness and mentalhealth

Determination of Eligibility - Uponcompletion of all application procedures, aneligibility grouping is determined for each candidate as follows:

Priority Group A - This is thepreferred group and all candidates in this group areconsidered for employment before candidates in all otherpriority groups are con- sidered.

Priority Group B - This groupis considered as satisfac- tory, meaning that minimumemployment standards have been met. Those in this group are consideredafter all those in Group A have been employed orhave rejected offers of employment.

Priority Group C - These arethe marginal candidates and are employableonly in emergency situations. These ap- plicants usually are lacking in one ormore of the basic requirements for full contractemployment.

Priority Group D - Persons inthis group are not employ- able.

The following criteria areused to determine priority If an groupings. The first criterionis professional references. experience but hascompleted student applicant hashad no teaching those of hiscooperating teacher teaching, thereferences used are The referencesof experienced and the universitysupervisor. have administrators andsupervisors who teachers must beonly those observed teachingperformance. references, themedian ref- In case of anodd number of If there is an evennumber of (in terms ofscore) is used. erence two priority groups,the appli- references equallydivided between are group. Professional references cant is assignedto the higher follows: A-Outstanding;B-Strong; classified intofour levels as C-Acceptable; andD=Weak or below. If there are criterion is theoral interview. The second When there aretwo the medianrating is used. three interviewers, by the repre- the ratings vary,the rating given interviewers, and used. (The rationale the personneloffice shall be sentative from interviewers in thepersonnel determination isthat the for this to do interviewsand on the aver- office have beencarefully trained greater validity overthe years in agetheir ratingshave shown relation to successin teaching.) interview are: A=86 - 100; The four levelsof the oral D=75 or below. B=80 - 85;C=76 - 79; and academic preparationof the The thirdcriterion is the of credits isanalyzed. If the point candidate. The transcript is), the is given(in most cases it hour ratio onthe transcript assign priority groups. In the event following scaleis used to given, it iscalculated bydividing the point hourratio is not the total numberof qualitypoints. the total numberof hours in

Overall eligibilitygroupings

groupings forreferences, academic 1. The priority converted into record and theoral interview are points.

Priority Group Points

A 3 B 2 C 1 D 0

references, academicrecord and oral 2. The points for interview aremultiplied asfollows:

65

V./m.4W s x Points

References x 2 Transcript x3 Oral interview x 5 academic record and 3.---The total pointsfor references, is made the oral interview areadded and an assignment to the properpriority group.

Priority Group Total Points

A 26-30 B 16-25 6-15 D 5 or below position to an appli- Health Examination -The offer of a 4173TIEFE7N7F7FFEantment on the basisof references, aca- cant who passage of a demic record and oralinterview is contingent upon by a Board ofEducation physical examination. This must be given psychiatrist. If the physician, thephysician-in-charge being a the offer of aposition individual fails thephysical examination, is withdrawn. and due to scheduling In some cases,late in the summer the examina- problems it may notbe possible to getthe results of reports to hisassignment. When tion until afterthe individual appointment will belimited this occurs, it isunderstood that the to one year withcontract not beingrenewable. of such a nature Obviously, if thephysical deficiency is released and a re- risk to children,the individual is as to pose a receipt of the placement is obtained asquickly as possible upon results of thephysical examination. by general Most of the physicalexaminations are given reviewed by practitioners but theresults of theexaminations are in charge. Thus, if there aresymptoms the physician-psychiatrist office may be alerted of mental healthdeficiencies, the personnel so thatappropriate action maybe taken. during the Periodic physicalexaminations are continued These may be at tenure of the teacherat intervals of three years. of Board physicians Board of Education expense,using the services physician provided theexamination or theindividual may use his own is reported onforms provided bythe Board. selection are the References - The threebasic criteria in

66 references, academic recordand the oral interview. It is recog- Some are nized thaw protessionalreferences vary in reliability. relatively %mrthless;others are quite valuable.

Greater reliability isachieved by requiring therefer- and scored as has ences on specifiedforms which can be analyzed weight among been earlier described.References are given.the least reliability, i.e., all the factors inrecognition of their varying 20 per cent of the total.

Academic Record - Academicachievement, especially in student teaching, has provedto be a moderately reliableindicator of competence in teaching. There are exceptions tothis generaliza- with excellent grades in tion, however. Occasionally a candidate with college does not do wellin teaching. On the other hand, one does very well in theclass- average orslightly above average grades room. record, the On the whole, however,the better the academic In more promisingthe prospects for successin the classroom. weight, the academic recordis given 30 per cent.

Oral Interview - Theoral interview is giventhe most that it is weight in selection, 50 percent. Care is taken to see thorough and consistentlydone. Multiple interviews arepreferable factors used, it tends to single interviews. Of the three selection to have the mostconsistent and predictivevalue in forecasting teaching success. The three fac- No single criterionis adequate, however. Mistakes occur, on occasion, tors in combination dothe best job. selection process regardless of the care andfidelity with which the and volume of workat peak is carried out. Pressure, rush of time The incidence of that periods in teacher selectionintroduce error. part error, however, canbe kept small bydeliberate effort on the of personnel administratorsand examiners.

Conclusion

Much has been writtenconcerning selectiontechniques and validity are difficult,regard- measures. Complete reliability and less of the typeof procedures used. Insightful personnel adminis- qualities trators and examiners canidentify a number of relevant those that a good teacher musthave. The problem is how to assess qualities with certaintyand consistency. This is the point at this which difference ofopinion exists andwhich is the concern of conference. be relatively Pre-employment selection,at best, can only

67 successful. For this reason, it seems feasible to consider the pro- bationary period as a part of the selection process.During the first three years of service, especially the first or second, much reliable "evidence" may be obtained which can be used to determine ultimate success in the classroom.

Selection policies and procedures must be continuously studied and tested for validity and reliability to the extent that this is possible. Most school personnel administrators and examin- ers are quite conscious of thisimperative. As promising techniques are revealed through exchanges of information inconferences of this type, each school system participating is enabled to re-evaluate and refine its own procedures.

The great need is for more research and a greater exchange of information. There needs to be more dialogue among personnel ad- ministrators and examiners regarding the implications of this research and the data it reveals. The Problem of Recruitment

Albert Schiff

Detroit Public Schools

At the time of this writing,the author was awaiting the results of last minutefrantic Labor Day week-endrecruitment ef- forts of our personnelstaff to enlist the aid ofqualified teachers teaching and in the greater Detroitmetropolitan area to return to in some measure alleviatethe teacher shortage--which inmid-August We was over600 vacancies in our elementary andsecondary schools. had just two weeks earliermade special appeals throughthe media of television, radio, and the pressto enhance our efforts.

Some solace was found whenhe learned that othercities were in the samedifficulty. The New York Times ofSeptember L, 1966, reported that teacher scarcityin die public schools of our nation was the mostcritical in a decade. Its editorial of the same date stat-d that many factorshad contributed to thescarcity. This But the was well knownto school people throughoutthe country. editorial emphasized thateducators ought to re-examinethe suppl lines in the light of theirbroadened mission. In brief, it not facilities were numerically andqualita- that if present training ; tively inadequate, then theymust be reorganized, reformed,and augmented by new types ofinstitutions, probably with federal assistance.

The gauntlet has been thrownto educators indicating that we should usethe teacher shortage not as an excusewhich besets the need for a nation- many schoolsystems--but rather should signal wide analysis of teachertraining and teacher utilization sothat all levels of education canbe assured of a steadyflow of high quality instructors.

This is the crux of theentire area governing teacher se- lection and should pinpointthe direction we pursuefor the future. screening de- For the most part, largecity school systems set up and less de- vices and techniques to 'tweedout!' the less qualified from teacher sirable teacher candidateswho have already graduated training institutions. Boards of examiners andother personnel officials set up programs oftesting, interviewing,examination of credentials, etc.--at greatexpenditure of funds--toinsure that their school systems do notemploy teachers who cangive less than quality education to theirstudents. This is deplorable especially at a time when staffsof this nature could bedevoting thefr time and energy in working out newerand better ways and meansof, selecting teachers for the .presentand fUture.

69

-- At a recent seminar devoted to high priority areas in school personnel administrationl--one of the speakers suggested that in the post-war years we haven't been too concerned with selection because of the acute teacher shortage and predicted that this would soon change. In fact, he quoted a personnel director of one of the California areas who said he wasn't quite sure that things would be any easier once there is an adequate supplyof teachers. He felt he would find himself getting into more trouble when there are several teachers to choose from than when he was just lucky to get a posi- tion filled. I cannot share this optimistic view as it relates to large city school systems. In Detroit, for example, even if suffi- cient funds were available for the employment of teachers we need- - it would be necessary for us to find approximately 2,000 teachers annually for the next several years to (1) maintain and expand cur- rent services, (2) provide for the normal leaves of absence, retire- ments, and resignations, (3) make provision for additional compensa- tory teaching staff in the lower socio-economic areas of the city, (4) provide for the expansion of the Federal Programs which augment and enhance our school program. These reasons, coupled with the potential loss of many male students who are being classified to I-A under current Selective Service requirements, convinces me and many others that the normal channels of teacher education--e.g., the four- year accredited teacher-traininginstitution will not be able to meet the needs of our large city school systems in the latter decade of the sixties and the coming seventies.

Standards and Procedures for Teacher Selection

Standards for school personnel administration were clearly delineated in a bulletin issued by the American Association of School Personnel Administrators in1960.2 This bulletin stated that whether selection is the outcome of a carefully designed inquiry and evaluation or is based upon less thoroughly developed processes, cer- tain facts should be known about a candidate prior to his initial appointment.

1. Investigation and Screening - The amount of iaforma- tion available on each candidate during the screening process determines the completeness of thepreliminary picture of his abilities and has to represent what is

HILL Priority Areas in School Personnel Administration: Bridging the Gap Between Research and Practice. Edited byTarold E. Moore. Bureau oT"EUUTitional Research and Services:Arizona State Uni- versity, 1965.

Standards for School Personnel. Administration. American Associa- tion Personnel Administrators. The Association, 1960. foundation upon which known about him. It becomes the for ini- asuperintendent justifieshis recommendation tial or forprobationary appointment. applied to The selectionprocedure is uniformly a. The everycandidate prior tohis, appointment. for selection procedure isessentially the same all candidates. information b. Prior to appointmentto a position, standing in: is on file showingthe candidate's

(1) Subject area competence. con- (2) Cultural knowledgein such other areas as stitute the schoolcurriculum, e.g.,English, suCli science, mathematics,social studies, and related areas asforeign language,arts, and music. including history of (3) Professional knowledge, education, childpsychology and development, educational psychology,and such related tests and areas ascurriculum, guidance, measurements, andmethods.

reasoning ability. (4) Intelligence, alertness, or

(5) Physical fitness.

(6) Mental health. descriptive reports onhis c. Written evaluations or practice teaching orfull-time teachingperform- ance havebeen reviewed.

file, includingtranscripts, d. College records are on personnel folder, andrecord of collegeactivities. used to assess Systematic andconsistent means are e. should be re- "personality." Independent opinions corded of at leastthree individuals asrevealed through such sources asemployment or performance and infor- records, collegerecords of activities, mation independentlysolicited by theemployer. lists f. Ranked order scoresheets or eligibility where used aredeveloped from objectiveexamina- tions or objectiveevaluations. Scores are not ordinarily influenced byletters of reference.

71 g. Score sheets or eligibility lists, where used, are made up of scores sufficiently reliable and valid to justify their use in promulgation of the ranked listing of candidates.

h. If scores are used to represent performance or findings and if such scores are so combined and/or weighted as to show ranked order of candidates on score sheets or eligibility lists, such sheets or lists are strictly adhered to in the employment of candidates.

2. Interview - The term interview here refers to the pre- employment interview 7araistinguished from an informal or introductory interview.

a. The employment interview is conducted by at least three fully certified and experienced staff mem- bers who are well acquainted with the procedures and problems of the profession,

b. Members of the interview committee represent not only the subject area concerned, but significant related areas or professional concern as well.

c. The interview is designed to enable the prospec- tive employer to learn those things about the candidate, such as personality, manner, and point of view, which may be only partly shown by his application and supplementary data.

d. Instances where a single interviewer is authorized to employ an individual, the data collected for each applicant interviewed is inclusive and con- sistent. The interviewer has access to the pro- fessional opinion of at least two additional persons who know the candidate well.

3. Appointment - Initial screening and final selection culminate in the superintendent's recommending the candidate for appointment. Only the Board of Educa- tion has the legal power to make a contract with the candidate.

a. The Board of Education entrusts selection of teach- ing and nonteaching employees to the superintendent through his professional staff and does not enter into the professional processes of initial, proba- tionary, and final selection.

72 b. The public understands and the Board of Education assures that teachers are employed for their pro- fessional ability above all other considerations.

c. Appointments are made only after all phases of the selection procedure are completed; theyare then acted upon with reasonable expedition.

d. Appointments are recommended by the superintendent and are consummated by the Board of Education.

e. Appointments are made in writing and constitute binding contracts between the Board of Education and the educational staff member.

f. The contractual agreement clearly indicates the nature of the appointment, whether probationary, regular, temporary, or substitute, and designates the term of service.

g. The appointee receives a copy of the pertinent law, regulation, resolution, or appointment action governing his contractual situation.

I would like to discuss briefly the method of selection of teaching personnel for the Detroit Public Schools. However, I do wish to call to your attention that this selection procedure isdone after the college or university has had the student enrolledin their respective institutions anywhere froma period of three to five years. I include the selection procedure of Detroit onlyas an information item--which perhaps has some features currently in operation in other cities as well.

The selection of teachers for the Detroit Public Schools is a continuous process involving the services of staff members from the schools, the Division for Improvement of Instruction, and the Personnel Division. The primary responsibility for teacher selec- tion rests with the Personnel Division.

Beginning with the completion of application forms, the process follows step by step to a final judgment on the part of the superintendent or his Personnel Staff memberas he examines evalua- tion summaries of candidates recommended for placementor non- placement upon the eligibility lists.

Requirements for most teachers (other thanemergency sub- stitutes, vocational day trade, and building trades apprentice teachers) are (1) a bachelor's degree froma college or university accredited by the North Central Association of Colleges and Secondary

73 Provisional Schools, or anequivalent agency,(2) a Michigan State (Elementary or Secondary) orLife Certificate,(3) a or Permanent record satisfactory college oruniversity record,(4) a satisfactory (5) status as a native or of supervised ordirected teaching, an individualwho has naturalized citizenof the United States, or become a citizen,(6) a satis- filed his declarationof intention to report on a chestXray. factory physicalexamination including a substitute teacher The requirementsfor employment as a seniors and for a contractteacher, except that are the same as sat- who have completedtheir student teaching post-degree students substitutes while they are isfactorily may beemployed as emergency for their provisional enrolled in courses necessaryto complete work certificates. file the following Applicants forteaching are asked to (1) Application for aDetroit TeachingPosition, credentials: responsibility of the college records. It is the (2) Transcripts of should request applicant to filehis own transcripts. The applicant send a transcript. This is each college oruniversity attended to from one collegeto because, althoughcredits transferred necessary grades earned in another are summarized ontranscripts, the actual Student copies oftranscripts may be each course arenot furnished. transcripts must befiled used in makingapplication, but official folder from collegeplacement before employment,(3) Recommendation bureau, if available. requirements are Applicants who donot meet the above cannot be consideredfor em- notified of thespecific reason they credentials are received onwhich to ployment. As soon as enough invited to participatein the base a judgment,applicants can be interviews are scheduled. selection process, andappointments for students of local Bachelor degreerecipients and graduate National Councilfor Accredita- colleges and collegesaccredited by the in a specialselection pro- tion of TeacherEducation participate tests provided theysatisfy cedure and may beexcused from the group conditions shown below: Graduate Students Bachelor's DegreeRecipients At least 15 hoursof The candidate wasin the "Upper 1. 1. graduate work with amark half" of hisgraduating class. of "B" or better.

work, A mark of "B" orbetter was re- 2. In undergraduate 2. the candidate wasin the ceived in directedteaching. "upper half" of agraduat- ing class from anaccredited college oruniversity.

74 3. In undergraduate work, a mark of ulin or better was received in directed teaching.

Applicants meeting these requirements and who are recom- mended by their colldge officials may be employed on the basisof a committee interview at the Personnel Division, at a Detroitschool, or at a college campus. "Upper half" applicants are usually inter- viewed by a committee of two, inclusive of supervisors,administra- tors, and/or members of the Personnel Division.

"Upper half!! applicants not recommended for employmentby an interviewing committee maybe asked to participate in the regular selection process.

Applicants participating in the regular selection process are required totake the National Teacher Examination or a local battery of tests.

As a final step in the selection process, ameeting is scheduled with a Personnel Committee, usually consisting of a teacher and supervisor from the subject field; a principal, an as- sistant principal, or a consultant; a school social worker or a mem- ber of the staff of the Psychological Clinic; and a chairmanfrom the Personnel Division. The selection committee reviews the college transcripts and recommendation folders, ratings, and test results. After the personal interview with the candidate, eachmember of the committee casts a secret, independent ballot rendering his judgment of the candidate.

The recommendations of the personnel committees are sent to the Assistant Superintendent and/or Divisional Director ofPer- sonnel for review and approval. His approved appointments are then presented to the Superintendent and then the Board of Education for official action. All candidates are notified of the results in writing.

Successful candidates are informed that they are placed on the eligibility list contingent upon the approval of a MedicalExam- iner of the Board of Education and the filing of anofficial tran- script indicating a bachelorts degree from a university Jrcollege accredited by the North Central Association of Colleges andSecondary Schools or an equivalent agency, and a Michigan StateProvisional or Permanent (Elementary or Secondary) or Life Certificate.

Candidates who meet the specific requirements for "upper half" candidates and are recommended by aninterviewing committee

75 for placement accordingto the date theapplications are approved by an interview- and transcripts arefiled. Those who are approved contingent upon satisfactorytest results areplaced ing committee Candi- eligibility list accordingto the date of thetest. on the satisfactory meet a regu- dates whose testresults are not entirely date for place- lar teacher selectioncommittee. If approved, the the regular selection ment on the eligibilitylist is the date of candidates are placed Those not classified as"upper half" meeting. by the teacher on theeligibility list on thedate they are approved selection committee.

Applicants who arenot approved toparticipate in the se- for employment by lection process andapplicants not recommended qualifications have notbeen selection committeeswho believe their Selection Review properly evaluated mayappeal to the Teacher Committee. Directions for the Future Problems AffectingTeacher Selection and Perhaps no This now brings usto the core of theproblem. mortality than theteach- profession has such arate of turnover and work out tech- It thus becomesapparent that we must ing profession. has a com- that would identifythe candidate who niques of selection for a life- and subsequentlyremains in the field mitment to teaching personnel ad- Selection procedures--asthey pertain to time career. Although our selection ministrators--begin withthe college senior. most large city procedures in Detroit areperhaps as adequate as need for teacherselection school systems, Iquestion the continuing administrators of schoolsystems when theselective by personnel earlier than the process forfuture teachersshould begin much college studentapproaching his senior year. National Commission on In 1965 a specialcommittee of the Standards studied the areaof the Teacher Educationand Professional misassignment of Americanteachers. It discussed the assignment and teachers, and entire problem ofstaffing the schoolswith effective problem interlockedwith came tothe conclusionthat this was a This report brieflydis- several broaderissues in Americanlife. competition betweenother cussed (1) thecontinuing and increased individuals professions and theschools for theservices of talented school districtsthemselves, (2) thepossi- and competition among students at the the right teacherwith the right bility of placing social or right time whendifferent schooldistricts had conflicting curriculum offered inthe intellectual goals,(3) the lag between in the public colleges that preparesteachers and that offered teachers and teacherorgani- schools, (1) theincreasing demand for administrators to satisfy zations placing agreater burden upon question of whether teacher's demands for properassignments, (5) the

76 colleges and universities offer adequate programs to prepare teachers to meet conflicting demands of the great cities and the rural areas, (6) developments in organization and technology cre- ate new and different positions for both professionals and non- professionals in the schools, (7) the influence of consolidation and reorganization of rural school districts on the preparation and proper assignment of teachers, (8) the misassignment of teach- ers as a result of insufficient numbers or theinertia of those who are principally concerned withassignment.3

The report concludes that these and other major ques- tions do impinge and complicate the problem of staffing schools with effective personnel.

In Detroit we often are hurled the challenge - -"Whycant we compete with thesuburbs in terms of quality education, salary, etc. ?" The sooner we face this problem realistically and indicate to all concerned that Detroit and other large cities are not sub- urban areas and that our teachers must reflect those ideals and characteristics which are necessary for effective teaching in a large, industrial, and cosmopolitan city--then and only then can we come to grips with certain basicprocedures to be followed in the selection of our teachers for the future.

In Detroit, over the past few years, the Personnel Division has sponsored clubs of future teachers ranging from the elementary through the junior high and the senior high schools. In 1957, at the beginning of our efforts, only 12 high school clubs served 300 members. Now, over 4,500 pupils are participat- ing in 161 local school units.An everage of 25 to 30 members is the usual size for these groups, although clubs range from as few as 8 members to 130 members.

Primary emphasis in the daily activities of club members is service to school and community.This is common to all levels. Since 1962 there has been a continuous increase in cadet teaching and tutoring which carries out a second purpose of the clubs,to familiarize students with the training, activities, and responsi- bilities of a teacher.

All levels explore various fields in education and help students plan for college. At the elementary level, this planning is often the simple but inspiring visit to a metropolitanuniversity,

3 The Assignment and Misassignment of American Teachers. National Commission on Teacher Education and Professional Standards. National Education Association:Washington, D. C., 1965.

77 level such visitscrystal- a goalfor the future. At the secondary concernentrance requirements, lize specificenrollment plans and programs, costs,and facilities. Clubs and those The success of DetroitFuture Teacher apparent in readingthe credentialsof begin- acrossthe country is colleges. An increasingnumber ning teachersinterviewed at the membership as anextra-curricular activity list FttureTeacher Club in the highschool.

Recruitment for themost part nowbegins with college shortages and other seniors and studentteachers. In view of this does not supplyenough problems that facethe large city demands of our society.Recruit- qualified teachersto meet the at high schoolsophomore ment should startwith great emphasis and speakers toshow This can be donethrough literature level. that teachers mustface pupils the new roleand responsibilities have thousandsof young people for the future. In our schools we enrolled in work-studypro- working as TeacherAides and/or are experiences thatreadily grams andother serviceswhich provide for a futureteacher. This could lend themselvesto a good start meetings sponsoredby the personneldivision be augmented by service ex- and city meetingsand to enable through conferences Such experi- in differenttypes of schools. change experiences expectations,, would be ences, plusthe raising ofaspirations and These experiencesshould especially valuablewith Negro youth. but should bein the entire not be confinedonly to the city Metropolitan area. it is in termsof Even in oursociety--as mobile as of a lifetime--thegreat- changes teachersmake during the course city rests withthe high est source offuture teachers for any of that city today. I hate to school students inthe classrooms true that - -"Theteachers of tomor- use an oldcliche but it is so today.fl row arein our highschool classrooms then should be The selection ofprospective teachers university levelbeginning at continued throughcollege and the School systemsshould provide acoordinator the freshman year. The aim teacher trainingcolleges in their area. to work with shortage areas, should be to helpthe prospectiveteacher, suggest opportunities throughclassroom visitations, provide child-study counselors, securing teacher conferences,getting jobs as camp in materials forclass employment in year-roundyouth groups, aid extend the servicesprovided term papers, etc. In brief, we must research divisionsto the end by teachereducation and educational achieved betweenthe school that greatercooperation can be system and theteacher traininginstitutions.

78 One of the outstandingdevelopments which ought to be nurtured and developed ingreater depth is the internship program The as an integralpart of the preparation forfuture teachers. internship can be extraordinarilysuccessful not only as a train- ing device, but also as a meansof achieving a functionalpartner- ship in teacher education andpublic school systems.

Michigan State University, in abulletin describing its internship program, states thatif the internship approach to teacher education is going torepresent a significant stepforward de- in teacher preparation, itshould be incorporated in programs signed to meet certain criteria:

1. Internship programs, to have areal impact, must be brought into the mainstream ofteacher education. Unfortunately, most internship programs nowoperat- ing are on the periphery andinclude only an insig- nificant portipn of the studentspreparing to become teachers.

2. To continue as a growingedge in teacher education, such programs must be designed sothe internship can be self-supporting.Many such programs presently operating are dependent uponfoundation or other outside support.

3. An internship program,to be worthy of the name, must provide careful andmeaningful supervision of the intern by competentclinical supervisors. With- out such adequate supervision, aninternship ex- perience can be sterile andlead to little profes- sional growth.

intern prior to internship 4. The preparation of the should approximate thepreparation of any other be- ginning teacher. In the cooperativerelationship with public school systems,teacher preparation in- stitutions must remember thefirst obligation of the schools is to the youngstersin the classroom, not to teachers-in-training. Therefore, interns must be well enough prepared at thetime they begin teaching to provide high qualityinstruction.

We should look favorably uponthe use of Federal fundstc help us in our search forquality teachers for the future. There be en- is a vast untapped sourceof college graduates who could couraged to consider teaching as a careerif we would provide the experience and training--whilethey are receiving reimbursement

79 during the period they are completing requirements for full certification.

Special needs of a metropolitan system demand racially integrated school staffs and training of prospective teachers to meet the needs of "deprived" and/or "disadvantaged" youth.

We call upon teacher training institutions not to recom- mend candidates for teaching about whom there are any questionsof general intelligence, competence, personality, or other factors which preclude success in teaching. School personnel administra- tors have a right to assume these candidates are ready to "tackle" the rigors of a classroom given the aid of adequate supervisory and administrative staff.

I believe large city school systems will need to modify rigid teacher recruitment and selection procedures to alleviate the teacher shortage by the adoption of temporary stop gap meas- ures--while still attacking the problem of effective long range programs.

80 Criteria: Problems in ValidatingTeacher Selection Policies and Procedures

David G. Ryans

University of Hawaii

Nature of Criteria with regard to A good deal ofconfusion appears to exist criteria, is em- just what we mean whenthe term criterion, or bench-mark used to ployed. A criterion issimply a standard or provide a frame ofreference for judging orevaluating something. which comparisons may be It may be thoughtof as a model against about accept- made. Usually criteria evolvefrom common agreement public utilities, able standards--regulatoryboards for insurance, banking, contracting, andsuch operate with aset of agreed-upon standards (criteria) as amodel. In many circumstancescriteria that are held to beimport- are arbitraryand relative to values particular time ant by some particular groupof persons at some of "values" and"value systems!! is and place. Indeed, the matter basic to the considerationof criteria. opinions and In relation toteacher selection, just as preferences (values) ofindividuals vary withregard to the compe= tencies and behaviorsexpected of teachers,the criteria against will often which teacher selectionprocedures should be compared community; vary (at least,in certain features)from community to nearly always require and validity studiesof teacher selection replication adapted to thevarying conditions.

In taking this positionthat criteria aredetermined by communities, I am value contexts thatdiffer among schools and implying that the first stepin the considerationof criteria against which to judge ateacher selection programmust be to de- termine the expectationsthat are held locallywith regard to there may be teaching and teacherbehavior. The extent to which the assurance with which concensus aboutmajor issues, the greater educational researchers may ap- a schooladministration or its components, and the proach the designationof criteria and their validity studies greater the possibilityof conducting meaningful of teacher selectionprocedures. criteria and I shall return tothe relationship between to comment further on value systems later,but I first would like neglect of and confusions the nature ofcriteria--and the frequent about considerationsrelating to criteria.

81

1.* Here I do not restrict my remarks to studies of teacher selection and research on teacher behavior; they are no more vul- nerable than a great deal of research in the behavioral scInces where the problem of the dependent variable, or the criterion, has been neglected. From years of reading research reports and re- search proposals I conclude that many otherwise elegantly designed researches--well designed from the standpoint of sampling, control of the experimental variable and other independent variables, data analysis (and often involving real ingenuity of approach)--have, almost as an after-thought it seems settled upon some available in- strument, or perhaps hastily thrown together some test, inventory, or other observational technique without great regard to itsvalid- ity and reliability, proceeding on the assumption that such instru- ment satisfactorily reflected, and provided useful estimates of, the criterion behavior. This happens to be a pet peeve of mine- - the fact that so many investigators seem to neglect or give too little attention to control of the dependent variable (criterion behavior) and that instead of considering this important problem from the very beginning of their research they appear to give it only cursory attention some time later in the investigation. I suspect in many such cases the researcher is introducing a source of Type II error; or when statistically significant relationships between experimental variables and the assumed dependent variable are obtained, the relationship really may bebetween the experi- mental variable and an unintentionally biased and unsatisfactory estimate of the criterion behavior. Although I feel my accusation is fairly generally applicable to research, it is one about which we teacher selection researchers certainly need todo some con- sidered soul-searching.

Teacher Selection in Perspective

It seems to me there is some similarity between what we are interested in when we plan teacherselection procedures and subsequently study their usefulness and the sorts of things a curriculum developer is concerned with.

Typically, I believe, the planner of instructional tech- niques and course materials considers the ideal procedure to be followed as consisting of: (1) designation of course objectives, goals, and expectations; (2) breaking down the objectives into de- scriptions of (a) expected teacher behaviors and (b) expected pupil behaviors, i.e., the pupil behaviors it is intended the course or curriculum will help to nurture and develop; (3) planning and de- velopment of specific curricular materials and instructional tech- niques that are hypothesized to aid in developing the intended pupil behaviors; (4) selection of appropriate means of measuring the at- tainment of the behaviorally described objectives (Recall that any

82 one of a number of methods may be used--e.g., measurement of sam- ples of the pupil criterion behavior (samples of the expected pupil behaviors), measurement of aspects of teacher performance, measure- ment of teacher opinion about the efficacy of the program, citing of critical incidents, measurement of pupil behavior known to be related to the criterion behavior, measurement of pupil test re- sponse to verbally described situations related to the criterion behavior, etc.); (5) assembly of data (which may be in any of sev- eral kinds of units, scores, ratings, etc.) yielded by the measure- ment devices that were assumed to reflect attainment of the speci- fied objectives; and (6) evaluation of the course materials and/or instructional procedures by drawing inferences from the collected data about attainment of the course objectives.

It may be laboring the obvious to spell out the closely related steps that are involved in the development of teacher se- lection procedures and their evaluation..Nevertheless I am going to describe what I believe to be a procedure that provides an ap- propriate rationale from which teacher selection should proceed. (Note that I consider this procedure to represent an "ideal" one- - one which often cannot be followed step-by-step in practice. Prac- tical considerations often demand that we skip early phases and proceed to set up teacher selection techniques on the basis of best available judgment--and I should not imply the selection procedures thus developed necessarily will be poor; they may be based upon substantial wisdom growing from experience, and upon testing them out they may be found to yield results that can indeed be shown to relate to valid criteria of teacher behavior, even though these criteria were not determined prior to the planning of the teaching program. I think we are getting the cart before the horse to de- velop selection procedures and then at some later time turn atten- tion to the criteria to which we think these procedures ought to relate--but sometimes this is the best we can do.)

But let me get on to a statement of what I think we might agree would be a desirable way to proceed if we wereoperating with- in an ideal situation. It is a procedure that is fairly similar to the curriculum development paradigm I spelled-out a moment ago. I will refer to some ten steps or phases.

(1) Selection and designation of general aspects of the alue system framework of the school/communityas they re- late to teacher behavior. I am referring here to the agreed-upon qualities that are desired, or expected, of teachers in a particular place and in particular kinds of teaching situations. (Note again that this process of arriving at criteria necessarily is subjective and a mat- ter of the values individuals or groups of individuals may possess in common.When we designate criteria we

83

v.a..11.11101alf proceed from a context of anaccepted value system.We view teacher behavior in lightof a set of attitudes, opinions, and viewpoints thatreflect the sorts of teacher behavior we approve and preferand also the kinds of be- havior we disapprove and findunacceptable. Value judg- ments and the value conceptsand systems on which they are based grow out of highlypersonal biases, preferences,be- liefs, opinions, and attitudes wehold as individuals. To the extent any group of personsshare in common cer- tain expectancies, preferences, orbiases about teachers and teaching, criteria ofteacher behavior may bedefined for that particular group. Thus value systems concerning teaching, and criteria of teacherbehavior, are likely to be relative rather thanabsolute. Although some "'valued teacher behaviors!! may be heldin common, by a large cross section of citizens andeducators at a particulartime, still other uvalued behaviors"that must be taken into ac- count in specifying criteria may varyfrom one community or school toanother.)

(2) Identification of kinds ofsituations in which the agreed-upon "valued teacherbehaviors!! may occur--and in which they may be observedand assessed.

(i.e., description in terms (3) Operational description of actual teacherbehaviors) of the agreed-upon valued behaviors that are to comprisethe criteria of teacher behavior.

estimating the operationally (14) Selection of methods of (i.e., behaviorally) describedvalued behaviors.This is the problem of instrumentationrelative to the criterion behavior and obtainingassessments of the criterionbe- haviors. (Assessment relates to quantified, orquasi- quantified, description.When we make an assessmentof some characteristicof some thing or somebehavior, we are concernedwith the degree to which thatcharacter- istic is manifest.)In assessing some aspect orcharac- teristic of the criterionbehavior of teachers we are trying to estimate the extentto which that defined characteristic is manifest by someteacher.

(5) Identification of observableproperties of teacher classroom behavior that may berelated to the specified operationally described criteria(i.e., the descriptive cataloguing of teachercharacteristics and behaviorsthat occur in theclassroom).

814 instruments and procedures (6) Development of selection that will re- that are hypothesizedto yield estimates flect the operationallydescribed teacherbehaviors to (criterion behaviors)--which,in turn, are assumed reflect the valueframework of the schooland the community served. by the teacher selection (7) Assembly of data yielded instruments and proceduresnoted in Step 6 above. used to (8) Assembly of data yieldedby the procedures estimate the criterionbehaviors--Step 4 above. estimates of the (9) Analysis ofrelationships between estimates behaviorally definedcriterion behavior and the selection of teacher characteristicsused in the teacher procedure.

(10) Evaluation of theteacher selectionprocedures by those the drawing of inferencesabout the validity of procedures for predictingthe criterion behaviors designated in Steps 1through 3 above.

ects of the CriterionProblem Common Confusionsin Dealin with As

One of the reasons wehave difficultywith the criterion different problem is that wesometimes fail todistinguish between "principals' rat- aspects of what isinvolved.We have all heard I think it ings,' referred to as acriterion of teacherbehavior. principals' ratings donot consti- is not nit-pickingto note that behavior. They tute a criterion or adescription of a criterion of obtaining data are onekind of estimate,derived from one method related to some speci- that may, under someconditions perhaps, be fied aspect of thecriterion behavior ofteachers.

Allow me toillustrate what I meanabout confusion of terms with one or twoexamples.

Let us suppose thevalue system in aparticular school degree of capability community expects itsteachers to possess some We may think ofthis, I with respect to"classroom management." that community. It believe, as a criterionof teacher behavior in description of cri- is, of course, a verygeneralized and abstract to observe terion behavior at thisstage. Before we can proceed teachers with respect totheir capabilities forclassroom manage- kinds of behaviors that ment we need tospecify still further the either (a) sam- comprise this domain and weneed to try to determine be observed ples of the criterionbehavior that may in some manner

85 and assessed and/or (b) known or assumedcorrelates of this cri- terion behavior that may be observed andassessed. As one example, of many that could be used, wemight choose the teacher's response to a situation involving activities onthe part of some pupil that interfered with the activities of his classmatesin pursuit of the objectives of instruction.We are still talking about criterion behavior but we now have broken it down into adescription (al- though still somewhat general) of a sample ofthe criterion be- havior. Now we might choose any one of severalmethods of estimat- ing the criterion behavior under consideration. And the method we would use would determine, at least to alarge extent, the kinds of assessments or estimates of the specifiedcriterion behavior we would obtain. We might choose to employdirect observation of teachers in the classroom by trained observers,and one of the kinds of estimates we might obtain bysuch a method would be ob- servers' ratings recorded on a scale representingqualitatively de- fined degrees of appropriate teacher behaviorin the disciplinary situation referred to. Or, we might choose to use principals' re- call of teacher behavior in situationsinvolving classroom manage- ment and this method might yield estimatesin terms of some sort of ratings, rankings, etc.

As a second possible example, supposeit was agreed that one aspect of the criterionbehavior of a teacher should be his capability of communication of knowledge. In our attempt at be- havioral description of the criterion, one aspectof communication of knowledge might be determilied to be theteacher's behavior in a situation involving the presentation and explanationof specified subject matter content. (This could be made still more specific- - we might specify behaviorthat emphasized clarity and directness of presentation, or perhaps subject matter depth, orabsence of ir- relevancies, etc.) In light of such criterion behavior wemight resort to an observation method that involved the useof teacher examinations which would yield estimates of theteacher's knowledge and understanding of the specifiedsubject matter. Or, we might again resort to direct observation by trainedobservers and obtain ratings, frequency counts, or other kinds ofestimates. Or, in- stead of employing a sample of the criterionbehavior of the teacher per se, we might choose to view the criterionin terms of known or assumed correlates of'nteachercommunication of knowledge." In this case we might choose to measurepupil knowledge of particu- lar facts, principles, etc. that are assumedto be a product (at least in part) of the teacher's behaviorin the communication of the specified knowledge. In this case as a method of estimating the criterion behavior we might elect to testpupil knowledge be- fore the teacher presentation and immediatelyafter presentation, obtaining estimates of the differences intest-estimated pupil kaowledge before and after exposure to theteacher's presentation;

86 or we might testpupil knowledge before theteacher presentation and again after some specifiedperiod of time--to obtainestimates of the extent to which theteacher communicatedknowledge was re- tained by the pupil; or wemight use a method of determiningthe success of the pupilin later situations forwhich the specified knowledge is presumed to be a necessaryprerequisite--such proce- dure yielding estimatesbased on test scores andgrades in subse- quent units of a course, inadvanced courses, etc. of I have used these examplesto help distinguish aspects the criterion problem thatsometimes are confused when wediscuss such matters. The methods of estimationand the estimates yielded by different methods ofestimating criterion behaviorshould, I think, be clearly distinguishedin our thinking fromthe descrip- procedures may be tions of criteria againstwhich teacher selection evaluated. The criteria themselves arethe behaviors of teachers that are held to be of value. And of particular importanceto validity studies of selectionprocedures, we need to recognize identifiable behavior samplesand known correlates ofthe "valued" behaviors that are accepted asthe criteria of teachercompetency.

Some Considerations in theDesignation and Estimation ofCriteria

I would like now to note,at least in outline, someof the kinds of problems wemust face in dealingwith criteria. I will restrict my commentsto two types of problems. One set of concerns has todo with (a) the validityof the description of samples and/or correlatesof criterion behavior(i.e., the validity of criterion descriptionsin light of the valuesystem involved) and (b) the generalizabilityof descriptions ofcriterion behavior. The other set of problemshas to do with thevalidity and relia- bility of procedures that maybe used for estimatingspecified criterion behaviors of teachers. I shall mention, butnot discuss standpoint of the in any detail, threedifferent concerns from the validity of definitions anddescriptions of criterionbehavior in teaching.

One such area of concernhas to do with judgmentsabout the dimensionality of thecriterion behavior underconsideration: (Needless (1) Is the criterion behavioruni- or multidimensional? to say we usually agreethat teacher behaviorinvolves a number of dimensions that interact incomplex combinations.),(2) How do be- haviors that comprise importantdimensions of teacher behavior ag- gregate--what are the behavioraggregates or patterns thatreally are relevantfrom the standpoint ofteacher classroom behavior? What is the relative importanceof various dimensionsof criterion behavior in teaching--and howshould these be weighted incriterion description?

87 A second set of concerns having to do withvalidity of the criterion descriptionregard the logical consistency andinter- relatedness of criterion dimensions--(1)How are the component dimensions of the criterionbehavior patterned? (2) How do they overlap?

Still another area ofconcern from the standpoint of validity of criterion descriptionhas to do with the sampling ade- quacy or representativeness of the criteriondimensions that are selected to reflect criterion behavior. This is essentially the problem of trying to arrive ata criterion description that is as free as possible of bias. A number of sources of criterion bias were described almost twenty yearsago by Brogden and Taylor in their classic articleon "The Theory and Classification of Crite- rion Bias" (Educational and PsychologicalMeasurement, 1950, 10, 159-186.)I reviewed the bias problem drawingheavily upon the in- sightful Brogden-Taylor treatment, andother considerations of the criterion, in The Journal of GeneticPsychologz in 1957 ("Noteson the Criterion Problem in Research,with Special Reference to the Study of Teacher Characteristics,"J. Genetic Psychology, 1957, 91, 33-61.) Since more detailed discussions exist,I will only remind here that in designating criteriaagainst which to judge teacher selection techniques we must know howto recognize and must becon- stantly aware of conditions thatmay bias (and make useless) our criterion descriptions.I refer particularly to contaminationbias, opportunity bias, experience bias,rating bias, deficiency (incom- pleteness) bias and distortionbias.

With regard to the generalizabilityof the criterion def- inition and description, (and hereI am speaking of the replicabil- ity of the criterion description under different circumstances)I again note the relation of criteriato value systems espousedby a group and the probable variation in adequatecriterion descriptions in different communities. We need be concerned whether criterion descriptions may vary fromone kind of teaching situation to another for the same teacher and fromone sample of teachers to another.

Once the problem of criterion descriptionhas been faced, we must deal with considerations relative tochoice of the method or methods, of estimating the criterion behaviorand the kinds of measurements or estimates thatmay be employed.Here again we are faced with the problems of validityand the reliability--this time, the validity and reliability ofthe ,instruments and the data they yield with respect to the criteriondescriptions we have selected with view to their validityor relevance.

A variety of approachesmay conceivably be applied to judging the validity of criterionestimates. Often the researcher

88 a

concerns himselfonly with "face" validity,where the method for estimating the criterionbehavior is superficially judgedto be re- flecting the criterion behaviorit purports to measure.The be- havior elicited is assumedto be isomorphic with thecriterion behavior.

Sometimes an approach whichI will refer to as "'postula- tional validity" is employed.Here the method and estimatesfor assessing the criterionbehavior are judged, in lightof postulated relationship of the behaviorelicited to the criterionbehavior, to be measuring the criterion.Various sub-approaches tothe determina- tion of the postulationalvalidity include: validity by definition; validity judged from theexistence of reliable differencesbetween Individuals when the methodis applied; content validity, orassump- samples tion of validity based uponestimates derived from selected of the criterion behavior;and validity in terms ofconceptual con- sistency--validity of the estimationmethod judged in light ofthe method em- apparent relationship betweenestimates provided by the ployed and some inferentiallyidentified "construct" orbehavior.

Further important considerationswith regard to criterion reliability of data estimates or measurementshave to do with the yielded by a particularmethod; and also with thefeasibility, or practicability, of an estimatingtechnique. Certainly these cannot be neglected.

Approaches to CriterionDefinition and Description

Returning to the matterof descriptions of criterionbe- havior, I should like tosimply note some of theapproaches that may be employed.

Criterion description isbasically a function ofthorough dealing with--in and detailed acquaintancewith the behavior we are this case, the behaviorof teachers as they carryout the responsi- bilities demanded of them inparticular school situations. Such acquaintance usually is bestacquired by controlledobservation. Too often, I sus- This is a particularlyimportant consideration. pect, we try to accomplishcriterion description byarm-chair and associative recall methods. The generality and usefulnessof a criterion description islikely to be proportionateto the extent that essential details ofthe behavior under study havebeen iden- tified and classified.And the most appropriate wayof becoming knowledgeable about behaviorsthat may contribute tothe criterion is by observation undercontrolled conditions.

Generally speaking, theusefulness of satisfactorinessof a criteriondescription will be greaterwhen:

89 under consideration, or its (a) the criterion behavior products, can be operationallydescribed, directly ob- served, and objectively recorded;

interpretations of the (b) the possibility of varied criterion behavior and itsproducts by different indi- viduals is minimum;

(c) the observations directedat the identification of criterion behaviors and thedata based on observations are analyticalrather than global; criterion behavior are (d) meaningful aggregates of the distinguishable from irrelevantbehaviors and attention is given to the determinationof such behavior patterns;

to, (e) the investigator iscognizant of, and attentive the more prevalent sourcesof criterion bias(e.g., contamination by concomitantbehaviors, by "opportunity," E37727571717, by rating sets, etc.;deficiency, or incompleteness of the criterion;and criteriondistortion);

identification of criterion (f) observations directed at behaviors have been extensivelyreplicated (e.g., an ade- quate number of individual caseshave been observed and observations conducted in avariety of times, places,and circumstances). descrip- And we must rememberthat underlying all criterion values and expecta- tions is the matter ofidentifying the prevailing tions that form thecontext for, and dictate,the criteria we formu- such as: late. We must first seek answersto value oriented questions with regard to pupil behavior? Are teachersexpected to be permissive Are they expectedto maintain rigorousstandards of pupil learning Are and control? Are teachers expectedto be rigid disciplinarians? teachers expected to behighly knowledgeable aboutsubject matter of subject content?Are they expected tobe available repositories matter knowledge, or arethey expected to arrangefor the pupil to in "discover" information?Are they expectedto take an active part directing learning, or to arrangelearning situations forindividual administrative progress?Are teachers expectedto participate in policies and decisions, or aresuch matters to beleft to the administrative staff?

Obviously these are only afew questions illustrativeof a the value climate of a kind that might be askedin trying to assess community or school. The questions referredto admittedly relate to would have to be de- global sorts ofvalues--behavioral descriptions rived in greater specificityto be useful. Questions of this sort

90 do not refer to all-or-none value judgments. They are not neces- sarily mutually exclusive. And the answers do not spell-out the behavioral criteria to be employed in judging the validity of a teacher selection policy or procedure.Nevertheless, such ques- tions, together with many others, do provide the necessary first step of determining the value climate before the process of desig- nating specific criteria can be engaged-in.

The actual description and definition of criterion be- havior may follow a variety of approaches or strategies. All too frequently (or so it appears at least) no strategy at all is fol- lowed, i.e., criterion definition is completely neglected, or at best given only brief attention resulting in non-critical assump- tion of the criterion behavior involved. Among the studies I have reviewed I find practices covering a wide range of acceptability: completely non-critical assumption of the criterion behavior (either failure to consider criterion definition or unsophisticated acceptance of a criterion definition with no attention to its (a) completeness or (b) freedom from contaminating and distorting con- ditions); criterion description based upon analyses of judgments of presumably qualified authorities; criterion description based upon, the analysis of responses to some response-evoking technique which is hypothesized to reflect some criterion behavior (here the cri- terion description is derived from the method of estimating the criterion--a procedure that should give us pause); and criterion behavior identified by analysis of records based upon observation of (a) behavior in situations presumed to involve the criterion be- havior or (b) products of behavior in situations presumed to in- volve the criterion behavior.

Approaches to Obtaining Criterion Data

Assuming we can describe our criteria satisfactorily we can now turn to ways of obtaining criterion estimates, i.e., the basic records and indices of criterion behavior against which data derived from selection procedures may be compared.

As we have noted before a variety of methods of estimat- ing criterion behavior are available--methods which vary in ration- aleand also in usefulness. May I just mention some of these in outline form:

Some methods of obtaining criterion estimates

A. Obtaining samples of the criterion behavior 1. Direct measurement of --samples of the criterion be- havior in process (i.e., on-going behavior)- - primary criterion data.

91 uncontrolled typical a. "Natural"behavior--i.e., behavior behavior b. Standard samplesof the criterion of behavior Direct observationand assessment (1) observers (including interview)by trained

(a) someobservation approaches

--Systematic, withimmediate assessment (time sampling) --Retrospective(nonsystematic) --Analytical --Global --Relative --Absolute

(b) procedures

--Rating devices --Check lists

preserved (2) Observation andassessment of records of criterionbehavior in process (e.g., videotapes)

untrained observers (3) Assessment by the criterionbe- Measurement of samplesof products of 2. 717971713ry criterion data havior--presumedproducts of samples of Direct observationand assessment a. behavior behavior products e.g.,on-going pupil

(i.e. products in natu- --Uncontrolled products ral situations) --Standard samplesof products eliciting the b. Use of devicesfor immediately products of criterionbehavior performance, e.g., (1) estimations of maximum pupil test results

estimation of typicalperformance, e.g., (2) reaction ques- pupil responsesto personal tionnaires (selfreports of opinions, temperamental responses,etc.)

(a) change in process,or (3)Measurement of (b) change inproduct

92 (a) change in estimates of samplesof di- rectly observed on-goingteacher behavior

(b) change in estimates ofsamples of a pre- sumed product of thecriterion behavior, i.e., pupil behavior

B. Identification of correlatesof the criterion behavior (i.e., behavior Alin process, orproducts which may be used as signs of thecriterion behavior)--secondary criterion data.

In my opinion the mostvalid of the various methodsof es- samples of the timating criterion behavioris that of focusing upon on-going criterion behavior andresorting to direct estimation based on observation ofthese samples of criterionbehavior in process.

Ideally, in the studyof the validity of teacherselec- tion procedures one wouldprefer to work with ',identicalelements" directly of the criterion behaviorin which he is interested--to observe and directly measurethe samples of the criterionbehavior on whichattention is focused. We would like to employ measure- ments based on ',work samples,' orthe ',natural', or ',typical',be- havior in process, or, as asecond best choice, uponsimilar ob- se'Arations of a product of thecriterion behavior. In many cases I think we can accomplish ourstudy in this manner. In others, it is true, we must be satisfiedwith the indirect estimates or corre- lates of the criterion behavioragainst to which j-idge ourteacher selection procedures. Such correlates-typeestimates may involve (a) behavior or products fromsimulated situations(e.g., perform- ance situations,simulating those situationsin which the criterion behavior occurs) or,(b) even presentation of graphicand/or verbal descriptions of situationsinvolving the criterionbehavior.

As I come to theconclusion of my remarks I feel astrong As sense ofinadequacy; of having bittenoff more than I can chew. great deal of is the case with mostof you present, I have given a thought to the problem ofthe criterion, particularly asit relates study of teacher to teacher behaviorand to the problem of validity selection devices. I find it easy to identifyand recognize many faced in trying of the problems anddifficulties with which we are behavior of to develop satisfactorydescriptions of the criterion teachers and techniqueswhich will yield valid estimatesof the criterion behavior involvedin teaching. I recognize the sources of bias in the descriptionof criterion behavior and theconditions making, for invalidity ofthe estimates yielded bydifferent methods

93 of assessing criterion behavior. But I am admittedly frustrated by the difficulties involved in obtainingcriterion data which are, on one hand, inclusiveand complete and, on the other, exclusiveand free of contamination. I know it is not easy to lick theseprob- lems, particularly when we mustfrequently conduct validity studies in situations where we have been usingcertain teacher selection de- vices that were selected on a priori basiswithout the benefit of guidance of adequate criterion descriptions. And now, after the fact, we are faced with the problem ofproviding procedures that will yield estimates of criterion descriptionsagainst which to test our selection data. I do not think the situation is animpos- sible one, but I cannot help but recognize, asI think most of us must, that we are faced with practicalconsiderations which force us to compromise andemploy make-shift methods that precludethe carrying out of validity studies of thequality we would like.

94 Some Notes on ValidatingTeacher Selection Procedures

Donald M. Medley

Educational Testing Service

As far as I know, allteacher selection procedures pres- ently used are based on the modelof aptitude testing; in other words, in constructing selectioninstruments the effort has been to devise a battery which wouldpredict success on the job. Val- idation studies have, accordingly,sought to establish predictive validity against some kind of acriterion measure of teacher ef- fectiveness obtained after the teacherhas been admitted to employ- ment.

A selection battery thatcould do this job fairly well would be useful indeed; it wouldenable the selection agency to compile a list of candidates withthe candidate who would make the best teacher at the top and the onewho would make the poorest at the bottom. Then it would be possible to appoint as manyteachers as were neededin a given year, beginning at the topof the list, knowing that the best possible setof candidates had been chosen. This is a beautiful ideal; but it justwill not work in practice. To my mind, the sooner all attemptsto validate selection proce- dures in this way are abandoned the better.

In the first place, when youconsider the nature of what you are trying topredict--teacher competence--it seems highly im- probable that it can ever be measuredwith a paper-and-pencil test, or any other devicewhich could conceivably be used on thescale necessary for teacherselection in large cities. There is con- siderable experimental data which confirmsthis pessimistic point of view. Most of the predictive validitiesobtained in studies done in the past have been below.30; very few have exceeded.40. And the improvement in predictiveefficiency obtained with such small correlations is practicallynegligible.

In the second place, even if the,correlations obtained were large enoughto improve selection, their valuewould be sus- pect because of the limitedvalidity of the criteria on which they would have to be based. At the present state of the art of measur- ing teacher competence, it is fair to saythat the part of any teacher effectiveness criterion we can predictwith a selection test is probably irrelevant to teachercompetence anyhow.

Finally, it is extremely difficult actuallyto carry out a validity studybased on this model, since to do so requiresthat--

95 for experimental purposes--a goodsized random sample of all who apply for positions in aschool system be admitted toteaching with- out any kind of selection. This is probably illegal, butin any case is not practical,in most large school systems.

A more fundamental reasonfor abandoning the aptitude test model is the fact that its use isbased on an untenable assumption; the assumption that the majorfactors which determine whether a can- didate will succeed or fail as ateacher operate before he enters the school system. Such things as what kind of aschool and neigh- borhood the teacher is assignedto, the characteristics of his pupils and the facilities andmaterials available to him whenhe tries to teach them, and the amountand kind of support he receives from his peers and superiors inthe school system are seen asdis- tinctly less important than suchthings as what college courseshe has had and what he learned inthem, whether he has worked in sum- mer camps duringhis undergraduate days, andhow happy his childhood was, indetermining his future as a teacher. This assumption is clearly implied by a conceptionof the selection problem as oneof identifying among applicants forpositions those predestined tobe- come competentteachers.

It seems more realisticnot to assume that the future of any of thecandidates has been (or shouldhave been) decided at the time when the selectiontakes place, but only thatthe candidates will vary in the degree towhich they have mastered that partof their preparation which can beobtained before they enter the sys- tem. The selection problem wouldthen be seen as one of assessing past learning ratherthan one of predicting futureperformance.

The problem of validating aselection battery would then become a matter of content validityrather than of predictive val- idity; and this kind of validityis much more likely to beachieved by a paper-and-pencil test or oneof the other techniq"eswhich a practical selection battery islikely to contain. From this point of view it may be said thatteacher selection should bebased on an achievement model rather than aprediction model.

After a teacher has beenadmitted to probationary status in the school system, we arestill faced with a problem ofeliminat- ing those candidates who havesatisfactorily completed their pre- service training but cannot teachsuccessfully. This could be re- garded as a problem inprediction; but I prefer to saythat it is another problem in achievementtesting. The first years of a teacher's career should be viewed as apart of his training; if by the end of his probationary periodhe has not learned to teach,he is not ready to be admittedto permanent tenure and shouldprobably be let go.

96 for a ability to teach weremade a prerequisite If proven of a teachingtest ad- appointment as ateacher, some kind permanent period would become apart of the ministered duringthe probationary This might be aformal affairinvolving final selectionprocedure. lesson underobservation, or aninformal the teachingof a standard it should be by supervisors. In either case, oneinvolving ratings anything that valid selectioninstrument than a muchfairer and more level. could be usedat the pre-service model forvalidating selection If we adoptthe achievement how can we findout whether our instruments as Ihave suggested, of achieving theirultimate purpose overall selectionpolicies are teaching in theschools? improving thequality of (so far as I approach to thisproblem which There is one the con- This approachwould involve know) has neverbeen tried out. of the qualityof teachers inthe tinuous androutine monitoring industrial pro- to qualitycontrol on an school system. An analogy sample of theobjects be drawn here. In industry, a duction line may line at certain is removedfrom the production being manufactured thorough Such an inspectionis much too intervals forinspection. the line; to every itemwhich comes off and expensiveto be applied to randomly selected,it is possible but if objects soexamined are of the averagequality of the obtain a quitesatisfactory estimate sample of it. entire output byinspecting only a could select a fashion, the cityschool system In similar in the sample of allthe teacherswho earn tenure stratified random each thorough andcareful study of system each year,and make a The results ofthe study as teacher after hehas receivedtenure. be keptconfidential, and guaran- related to any oneindividual would affect his careerin the system;but tees made thatthey would not inferences aboutthe teacher popu- group datawould be usedto make provide preciseinformation about lation as a whole. Such data would brought into theschools year by the overallquality of teachers would also trends could bedetected; they year, sothat important information about areasof strengthand weakness provide diagnostic changes couldbe made. on thebasis of which would be usedwould make The fact thatonly group scores procedures whosereliabilities to employinstruments and it possible cl:;v:gnosis; andthe fact that low to be usefulfor individual are too would be smallrelative to thetotal the number ofteachers tested would make itpossible to userela- number employedin the system not procedures. Although we do tively expensiveand time-consuming of indi- of measuringthe competence presently havethe capability confident that we reliably andeconomically, I am vidual teachers competence of allthe teachers could reliablyestimate the average

97 in a system, using techniques already available to us, at a per teacher cost which would be quite reasonable.

I have introduced this idea of quality control of teach- ers in a system in the context of the problem of monitoring teach- er selection policies and procedures, because that is our immediate concern here; but I would point out the obvious fact that such in- formation would have many other uses, some of which might be re- garded as more important than this one. I see no reason why all of these purposes should not be achieved at the same time. Nor do I see any other feasible way of achieving the one we are concerned with - -that of assessing the effects of selection policies and practices as they are used.

98 Criterion Problems inValidating TeacherSelection Policies and Procedures

Harold E. Mitzel

The PennsylvaniaState University

Instead of following myassigned topic for thisbrief to stimulate dis- paper withits emphasis onproblems, and in order and development proce- cussion, I have chosento propose a research procedures. dure which wouldlead to a new setof teacher selection the notion that many of By adopting thisstance, I hope to convey problems are of our own so-called teacherselection criterion our conceptualization of making and generallystem from an inadequate without diffi- My proposed solutionis, I'm sure, not the task. that the proposal culty in implementation. FUrther, I'm well aware expensive, and politically is deceptivelysimple, fantastically difficult to bring about. data-gathering de- If we look atteacher selection as a mul- vice instead of as aprocedure, and if weview the result as binary selected-rejected tiple variablesinstead of as a single been looking in the wrongbe- scale, then it isclear that we have information about teacher havior domain forthe critically relevant disciplines have taught us selection. Psychologically oriented future performance arevari- that the bestpredictors of a person's similar situation. Thus, ables based on'hisperformance in a prior student's high schoolacademiP record as amajor trust- we use a best pre- worthy indicator ofhis college success. In industry, the quality of a person's dictor of performance on aspecific job is the In the military, ex- performance on previousclosely relatedjobs. have been used asmajor sources pensive simulatorsand training aids performance under battleconditions. of data forforecasting adequate these sources arenot perfect To be sure, thedata derived from on-the-job behavior, but as aclass of vari- forecasters of direct used predic- ables they seem tostack up better thanthe currently of teacher selectionis tors of teaching success. The real problem who have met the sameeli- to choose from amongall the candidates degree, passed a setof gibility standards(i.e., baccalaureate will pro- state-approved certificationrequirements) the ones who in the classroom. Of vide the best patternof teacher behaviors wanted have to beidentified and course, whatteacher behaviors are easily accomplishedby the made explicit, butthis task should be educational leadershipin any given schoolsystem. candidates with ob- After screeningout eligible teaching consider that symbol vious physical andpsychiatric defects, I

99 manipulating ability as assessed by paper-pencil devices is the major variable which saturates the selection procedures of most large school systems. Systems that use undergraduate grade point averages as indicators of teacher quality areundoubtedly tapping the same academic variable.Large school systems that employ a written test, particularly the short answer variety, are banking on the same general factor, butattempting to measure it as re- liably as possible. Indeed, many of the dubiously valid interview rating techniques require the rater to form some sort of global assessment of mental ability. I am not advocating the selection of unintelligent teachers for our schools, but it does seem to me that we have exhausted bookish intelligencein its wide variety of forms as a viable predictor ofteaching performance. I am suggesting that we turn to measures of classroom performance derived from sys- tematic classroom observation as a data source for improving the selection of teachers.I am reminded of an analogy recently in the news. For years the annual slaughter on the nation's highwayshas been attributed to a class of variables associated with the per- sonal adequacy of the driver, but suddenly, in spite of the power- ful propaganda influence exerted by auto makers, it becomes evident that there is an important class of accident and injuryrelated variables directly traceable to the mechanical condition of ve- hicles and their lack of safety engineering. For teacher selection, we need to discover a whole newset of variables.

To exploit the potential of a new and different set of variables for teacher selection first requires a decision to quit trying to find valid predictors of teaching performance in school grades, interview rating scales, oral and written examinations and U. S. Office of Education guidelines.Second, we must attempt to develop classroom performance variables in the context of the class- room situation, thus giving ourselves anoptimum chance of producing valid predictors. From the standpoint of the scientific approach, the procedure is straightforward and can be listed in eight steps:

Step 1. Develop empirical descriptions of classroom per- ETaance by systematically recording samples of observable behavior for existing teaching staff members. These ob- jective records of teacher behavior may be process- oriented or content-oriented, and preferably both. It is extremely important that the arbitrary global evaluations of fl good teacher!! and fl poor teacher!! be avoided in anyat- tempt to record teacher behavior objectively. The re- search on systematic observation reported by Withall, Flanders and Amidon, Smith and Meux, Mitzel and Medley, and Bellack would make good starting points.

2122 2. Construct a comprehensive sample observation

100 instrument from the dataobtained by Step 1. Here the judgment of educators whoknow the teaching processand the subject matter willhave to be employed inorder to select those behaviorswhich fit together to make uppat- terns deemed desirableby the school. For instance, if pupil participation inclassroom activity planningis wanted in a school system,then it should be fairly easy to identify a cluster ofspecific teacher behaviorswhich clearly foster pupilparticipation. Conversely, a cluster of specific behaviorswhich inhibit pupilparticipation can also beidentified. Experience shows rather con- clusively that these behaviors canbe put together with unit scoring to form areasonably reliable scale.

observation device to all Step 3. Administer the sample teacher candidates in anaturalistic classroomsetting (i.e., student teaching,internship).Of course, if some of the candidates areobviously unfit fof reasons unre- lated to their classroomperformance (i.e., infection with a communicabledisease, defect in moralcharacter), then these may be eliminatedin advance. This step is important since it enablesthe selection staffto obtain Research by Medley a wide rangeof behavior patterns. and Mitzel shows thatit is necessary toobtain multiple samples of the observedbehavior of a given teacher can- didate in order for areliable pattern to emerge.

This is perhaps the Step Li.. Employ all candidates. hardest decision to make, evenin times of a teacher shortage, but it is a necessarystep in order to validate a newinstrument. the on-job be- ISItep 5. Systematically observe and score avior of all teachersin the employed sample. The scor- ing should be done on apriori determined trialscales which were generated fromthe extensive observationsof the behavior of in-houseteachers. This step should be accomplished as soon asfeasible after employmentin order to minimize theself-selection effects of early dropouts.

5 Step 6. Using the observationdata generated in Step weights for the asa criterion and employing optimum of several behavioral components,rank order the sample demarcation between a new teachersand establish a trial satisfactory group and anunsatisfactory group. (The procedure itself does notdemand that anyone bedismissed.)

101 Step 7. Analyze the pre-employment observation scale data against the on-job observation criterion obtained from the same individuals. This step makes it possible to assess the predictive power of the pre-employment observation information.

2122 8. Repeat the process beginning at Step 2 on a new sample of teacher candidates utilizing improved observa- tion techniques and behavior scales.

Now the oversimplified eight-step process described above has a number of pitfalls. If the pre-employment teaching situation in which the predictive data'are gathered has in it a lot of content- specific or situation-specific elements, then the predictors do not have a maximum opportunity to be closely related to the criterion. For example, if a candidate is observed during student teaching in a comfortable, middle-class, suburban school and then is measured on- the-job in a slum school, one would expect this difference in situa- tion to have some impact on his instructional behavior pattern. Similarly, if the predictive measures for a candidate were obtained from a series of mathematics lessons and the criterion data were generated for the same person on several art lessons, some of the observed differences might well be attributed to the change in subject matters.

These problems of the specificity of schoolroom situation and content can probably be lessened by careful planning in large school systems.

That the use of classroom observation as a source of se- lection data would be a revolutionary development is confirmed by the recent survey reported by Gilbert and others. This 1966 report shows that, in almost sixty per cent of the large public school sys- tems, candidates are not at all observed. In an additional twenty per cent, only one observation is made per candidateand the data derived from this procedure are undoubtedly a mass of subjective non-predicting rating scales which tell more about the rater than about the candidate. It seems to this observer that we must strike out in new directions on the task of predicting teacherperformance and discontinue our perseveration with unproductive sources of data.

102 Needed Researchin Teacher Selection

John C. Flanagan

American Institutesfor Research in the BehavioralSciences

in teacherselection This discussionof needed research under four mainheadings. These are will discussresearch needs of teacher behaviorswhich de- (1) the determinationof the pattern evaluation of theeffectiveness fines effectiveteaching, (2) the prediction of teachereffectiveness, of teachingperformance, (3) the teachers to leaveteaching. and (Li) factorswhich cause good that research ondetermining thebehaviors It is proposed The effective teachingbe given a neworientation. which constitute orientation is thatinstead of principal characteristicof this new specifications for therequirements of aneffective a singleset of be a very largenumber of sets ofrequirements teacher, there should of each which make foreffective performance defining the behaviors Although such an ana- aspects of theteaching task. of the various circumstances it becomes lytical approach hasadvantages under any where the activitiesof the of great importancein the situation with the adventof new teach- teacher are likelyto change markedly technology and theprovision of ing methods, newobjectives, new teaching aides. clerical assistanceby computers and involved many activi- Effective teachingin the past has discus- lecturing, maintainingdiscipline, leading ties, such as correcting students' errors, sions, checking onstudents' learning, students' interests. stimulating students'thinking, and arousing likely to be importantfor Although many ofthese activities are of the future,it seems probable effective teachingin the schools change in emphasis.More import- that there willbe considerable activities as individualtutoring; assist- ance willbe given such developing a sense plan theireducational program; ing students to educational develop- in the studentfor his own of responsibility interests and specialabilities; ment; helping himto discover his satisfactions that canbe obtained from helping himdiscover the him generallyin his personal creative activities;and assisting and socialdevelopment. will be able to pre- Since it is unlikelythat most of us teacher's role andactivities in the dict the precisenature of the considerable importancethat we schools of thefuture, it is of for effectiveteaching carry outresearch to definethe requirements important that ingathering thisinforma- of many kinds. It is also individual studentinvolved in the tion thecharacteristics of the

103 teaching activity be clearly specified. It seems likely that the requirements for motivating one type of student are likely to be different from those for motivating another, for example.

In carrying out research in this field, it is proposed that tasi, analysis procedures and the critical incident technique be used. These can be supplemented by other observational and analysis procedures. A device which has already shown its value in this type of research is the video-camera which can capture the teaching situation and play it back for observation and analysis as many times as is required.The procedures of task analysis are likely to be of special value in developing requirements for new types of teacher activities. Of course in using this procedure, the characteristics of the student, his previous knowledge and many other factors can be specified and varied at will. The critical incident technique also makes possible the detailed description of the specific student involved and the definition of the particular aim of the teacher's activities. It should be of great value to collect many thousand incidents each of which is classified accord- ing to the type of student and the specific teacher activity involved.

The second area of needed research is in the evaluation of the effectiveness of a teacher. Of course in this case also, the focus should be on the effectiveness of the teacher in each of a wide variety of activities with each of a number of types of students. Teacher effectiveness may be evaluated in terms of either the process or the product. As assessment procedures are developed which measure all of the desired changes in student behavior, and computers provide efficient storage and analytical procedures for these data, it seems likely that much more detailed records can be collected and analyzed than has been characteristic of past evalu- ation procedures. It appears that if individualized educational development procedures are used which set specific obtainable tasks for each student, the details of whether a particular teacher's students obtain the goals specified will provide a much more pre- cise record of teacher effectiveness than has been previously available. High priority should be given to research and develop- ment work on procedures for a comprehensive evaluation of the teacher's effectiveness in many activities with many types of students based on systematic measurement of the effectiveness of the instruction.

It also seems desirable to continue research on process variables in teaching. It is suggested that the most promising lines of research on process variables involve systematically de- veloped observational record forms on which specific observations by a competent judge are made of effective and ineffective perform- ance on the part of a teacher. These, of course, should be related

10 it to the particular activity and the particular type of student involved.

A factor which could have considerable influence on the quality of teachers is the use of merit increases and the assign- ment of tenure status only as a result of demonstrated effective- ness. The lock-step promotions characteristic of many schoolsys- tems and the failure to eliminate the ineffective teachers produce a situation attractive only to the mediocre and marginal teachers. Of course, improvement in these areas must be based on valid pro- cedures for the evaluation of teachers.Research on the evaluation of teacher effectiveness should be given much attention for another reason, which is its essential importance to other types of research on teacher selection.

The third area of research relates to the development of predictors of effective teaching.It appears that long-range follow-up studies are among the most promisiag approach to this problem. It is anticipated that data from students tested as 12th graders in Project TALENT who have graduated from college and en- tered teaching will become available this fall. Similarly, in the next three years data from other classes will be collected andana- lyzed. These data should be of distinct value in defining the type of individual who enters a teaching career. Of even greater value will be the ten year follow-ups planned for Project TALENT which will begin in 1970. These studies will provide data on those who have not only selected careers in teaching but also have been found to be effective teachers over a substantial period of service. As in the case of the other types of research studies mentioned, it seems important that predictors not be focused on the quality of teachers in general but rather the effectiveness of the teacher in performing specific roles or activities. It is also believed that the trend in research with respect to predictors will be in the direction of a very large number of specific predictors rather than a few more general measures.

The last area of needed research relates to the condi- tions of teaching with special emphasis on those which have nega- tive selective value. It is well known that the turn-over in teach- ing is very high, and it seems important to determine why teaching loses some of its most effective individuals. One of the problems calling for research in this area is the effects of family duties on remaining in teaching. It would seem worthwhile to study the question of whether part-time employment or somewhat less than full- time employment might retain some of the better women teachers. Other research topics in this area relate to the study of specific teacher activities, particularly with the view to determining whether or not some of the clerical and administrative duties could be performed by teacher aides, clerks, or computers.

105 Another valuable studymight be directed atthe question early in their of "why many promising youngteachers leave teaching practice of as- careers." It seems likely thatthe administrative them down with signing new teachers todifficult classes, loading and similar pro- the more unpleasantextra-curricular assignments, cedures which give the newteacher a bad initialimpression of I teaching are responsiblefor losing effectiveteachers.

In conclusion it isbelieved that the newmethods of teaching will make itpossible to evaluateteaching effectiveness This much more preciselythan has been the casein recent years. will make research onthe characteristicsfor effective performance As pro- in specific teachingtasks both feasible andimportant. developed they should be posals for new educationalprograms are accompanied by plans forresearch on the teacher'srole and activi- greatest needs for re- ties in the new program. Perhaps one of the systems concepts to the search in teaching isthe application of include com- teaching and learningsystem. This application would prehensive measures ofthe input and outputof the system as well For such studies to beeffec- as studiesof the process variables. of the system in tive much better measuresof the input and output concepts, and abilities terms of assessingthe students' behavior, in will be required than are nowavailable. Perhaps the first step and develop- a programof teacher selectionshould be the definition Unless we have valid ment of measures ofthe products of teaching. types measures ofthe effectiveness of ateacher's efforts, other exploratory, and perhaps of research must beregarded as tentative, in many cases, trivial.

106 Needed Research inthe Area of TeacherSelection

Harry B. Gilbert

The Pennsylvania StateUniversity

I Assumptions

There are three basicassumptions that underliethe spe- explicit. cific proposals Ishall outline. Permit me to make them minimal 1. Interest in the areaof teacher selection is based upon the actualamount of research under way. teacher However a great dealof interest does exist among personnel selectors anduniversities. The problem is to make patent what'slatent.

Professional teacher selectionpractices re rarely em- 2. using ployed. In large school systemsthat presume to what is done. selection techniques,screening is actually rejections In smaller, affluentschool districts, hunch and global perusals,sometimes in actualobservations serve as selectiontechniques. shortage of A moment1sreflection on the current teachers, particularly inthose school districts per- that ceived to be fltougho,will confirm the probability this assumption iswarranted.

teacher selection is, agreat big area 3. Since the field of up foi grabs,it is desirable thatresearch be encouraged in varieties ofapproaches, without toomuch specificity, It follows, of course,that wide disseminationof re- search be encouraged andthat investigators besupported IIITh the notion that hypotheses maybe rejected as well as verifiedby experimental data. Regrettably this simple dictum, readilyunderstood in universitysettings, seems to be aheretic notion in an agewhen innovations are publicized assuccesses beforeevaluation.

II Proposals at- 1. Attention must be paid tothe process of stimulating tention to the methodologyof teacher selection andto research, validity of presently which must inevitablybe an assessment of the Surely employed procedures andthose yet to be tried ordesigned. teacher behavior - and on we can agreethat fundamental research on pupil-teacher interaction as anoutcome of theteacher-stimulus understanding of why so im- must be encouraged. What is needed is portant a field forresearch continues to beneglected, despite

107 promising starts by such workers asWithall, Mitzel and Ryans. The simple answer - too difficult - seems notenough in an age when com- plex problems are tackled with the resourcesmade available by con- temporary cash flow.

I suggest therefore that socialpsychologists and sociolo- gists be invited to investigate broadlythe field of research in teacher selection f,r their aid in understandingavoidance attitudes of potential research workers. There is no disposition to hide the obvious - that I have made a value judgmentand that the purpose would be to be able to persuade researchtalent into action in this sphere.

I propose too that we evaluatethe effectiveness of the design of this conference, with itsinfluential participants, as a means of stirringinterest and action in teacher selectionpractice and research.

2. We should take advantage of currentsupply and demand in the area of teacher selection.Specifically I refer to the shortage of teachers as the country begins towake up to the great need for teachers and pumps green blood into localsystems from the great big Federal artery. Teacher shortage gives the personnel research man a rare opportunity. He can lower his selection cut-off scoresto pro- vide a greater range for assessment ofpredictive validity. It will take some degree of courage to do this,although some reflection on the lack of predictive validity datashould be encouragement enough.

At the same time, there is always anover-supply of appli- cants for supervisory or administrativepositions in education. This should serve to be a source ofcomfort for those who would ar- gue that one cannotbe selective, or try to be, withteachers when we need all whoapply. Furthermore, it is palatable to applicants to be fed a variety of selectiontechniques when there are more ap- plicants than positions to be filled. This presents the oppor- tunity to employ varied and multipledevices, to a socially accept- ing, even willing, group.

3. I new come to some specificproposals. First, I make reference to the 16 "Suggestions forFurther Research" on pages55- 57 of "Teacher Selection Policies and Procedures inLarge Public School Systems in the UnitedStates." There should be enough ideas here to generate several bushels ofPh.D.'s or Ed.D.1s.

I should like to take the time to expressmyself on the perennial dissuader - the lack of soundcriteria. The fact is that so .nd criteriaexist only in relatively simple occupationsand not in complex professions. Worker output, in terms of quantity, or

108 employed as cri- quality, or time, or acombination thereof, can be when teria for production lineemployees. But where do you go everybody knows how hard itis to define "successin teaching!! par- with) ticularly since this amorphousgeneralization (bad to begin keeps shifting in differentschool settings, atdifferent levels and with differentsubjects?

At least we havelearned from previous studythat the is a concept to overall, general estimateof "success in teaching!! performance, be discarded. We can work withdimensions of teacher These dimensions as the localhiring school systemdefines them. may be withrespect to teacher behavior,such as "evocative of interaction,!! "ac- pupil participation,""encouraging pupil-pupil to the cepting of deviantbehavior," etc. They may be with respect ultimate in expectancy,namely pupil development,in skills, knowledge, or attitudes.

Let's not beat a deadhorse. We can all agreethat re- de- search is necessaryto refine and definethe dimensions and to to assess termine methods objectively - orreasonably objectively - of the extremes in them. I propose that weaccept crude estimates dimensions and not wait forthe millennium. I propose that we en- educators, school adminis- gage taskforces of teachers, teacher trators and personnelexperts to agree onworking definitions of dimensions of teacherbehavior on a scale from"most desirable!! to for extremes on these "least desirable." Then obtain nominations Various procedures can be em- scales - the Hbest!! andthe "worse." nominative tech- ployed for these nominations.They could be peer nominations and niques, observer ratings,supervisor ratings, pupil measured pupil performance.Hopefully a variety oftechniques would be employed. with best I am arguing in short,for our getting started others among us con- available techniques toobtain criteria while tinue more basic research. 1966) I In another setting(APA Annual Meeting, September personnel minds have also urged that weutilize the best informed While I would hardly clas- to review presentselection procedures. sify such activity asresearch, I do maintainthat some improvement in present procedures canensue, withoutwaiting for .esults of years of study.

In short, I amurging a redirection of energyto the procedure, in a reasonable problem of teacherselection research and ongoing need for basic and practical waywithout sacrificing the research.

Finally I would liketo propose the needto establish a

109 clearinghouse .Lor research and practice in teacher selection. I don't know who woulu establish this. I believe the field is im- portant enough that some university interested in developing this branch of educational administration might be encouraged to assume such leadership. Certainly a repository of findings with dissemina- tion facilities would be a major resource in development of interest in the field. SECTION III. CONFERENCE PROCEEDINGS

Greetings

Paul Denn

Board of Examiners

Good evening, ladies and gentlemen. Everything that we say from now onwill be taped. We do have to telescope this eve- ning a good deal of what we have to say andtherefore some of the humorous remarks which I attempted to gathertogether at the very last moment will have to be omitted. In connection with this tap- ing, please speak clearly and distinctly. If you ask a question, would you state your name so that it maybe recorded.

We are all seriously engaged in what manyof us - I think L11 of us perhaps - think is justabout the most important thing in education, the selection of good teachersfor children of our schools.Whatever we can contribute at this conference towardsthe improvement of such selection will put us all thatmuch ahead and the nation that much more in debt to all of us.

I'd like to express the thanks of our boardto the people who worked so hard on this conference: our chairman, Dr. Bogen, who I'm now informed is in the hospital andwill undergo an opera- tion tomorrow; Dr. Gilbert, who worked veryclosely with him on the research committee; Dr. Lang and Dr. Kalick, bothof whom were for a brief time ourentire research staff. This evening I'm going to call first on Dr. Gilbert to outline for youthe background and objectives of the conference.

Background and Objectives of Conference

Harry B. Gilbert

Pennsylvania State University

Thank you, Paul. I am here to outline the background of this conference and I shall be brief. I think it has to start with the role of the Board of Examiners inteacher selection. It's just a plain factof the size of New York that New YorkCity's Board of Examiners is the largest teacher selection agencyL the country. It is involved more with selectingteachers than any other agency. Over 50,000 applicants a year, aschool system that has 55,000 pro- fessional personnel at last count and stillrising, a budget for

111 selection that's well over a million and a half, all of thesefacts give evidence of the large scale involvement in teacherselection. For years the Board of Examiners has beeninterested in improving its selection techniques, but the Board of Education hasnot seen fit to support research in teacher selection. The Board of Exam- iners employs one research associate and has not even beenable to obtain funds for a secretary for him. This seems incredible and as I sit in my own little lair on Mt. Nittany now I findit hard to believe, but year after year after year, the Board of Education has simply refused to recognize that you can't spend all this money on selection without evaluating the process. At the same time we've grown tired of hearing criticisms byinformed people that we ought to be doing some research on our selection techniques. The less in- formed people simply say "Abolish the Board" without even thinking, without even realizing the implications. But these important facts of our background, the desire to evaluate the process of teacher se- lection and the failure to get the support that's necessary has caused the Board of Examiners to look elsewhere for support in this very vital process. And in the looking elsewhere it has recognized that there is a high degree of interest in the process. The in- terest is latent and this is something we'll talk about later in the meeting, but all of this background is necessary to understand why the Board of Examiners has come up with a5-step procedure to generate more interest in the process of teacher selection. I want to outline the five steps and then delineate the objectives of this conference.

1. The first is a survey of procedures. The Board of Exam- iners undertook with aid of the US Office of Education to find out what large school systems throughout the country say they are doing in teacher selection.

2. Second, is this present phase - an invitational conference in which a group of experts representing industry, univer- sities, school systems, state education departments, civil service and teacher organizations, people who know the process of selection and can apply it to school systems, have been in- vited to come here to discuss the implications of the survey and to make recommendations for action.

3. Third, we intend to disseminate widely the results ana recommendations of this conference.

4. Four is the action step where we hope evaluation of pro- cedures, development of new procedures and research in the basic problems confronting those interested in teacher selec- tion will take place in a variety of settings.

112 5. And fifth, the establishment of aclearinghouse somewhere where all the results of research andthe variety of forms and procedures utilized in teacher selectionwill be collected and then further disseminated.

I now conclude with a simple andstraightforward delinea- tion of the objectives of this conference. There are three objec- tives as I see them. In the first place we have justcompleted a survey of large schoolsystems, the largest and some that are reasonably large.What does this mean in terms of selection? There are some obvious implications ofthe survey that personnel people with experience and know-how canmake with respect to on- going procedures. More important though, what kind ofresearch do we need to do?Secondly, having looked at the implicationsof these surveys, let's put themin very specific terms with respectto teacher selection practices. This is not to say that we forty people here can recommend action which weknow will be followed. We do think, however, we should setforth what should be done in terms of needed research in teacherselection, and any related mat- ters that may come up. You just can16 lay out ideas to acreative bunch like this expecting that it willfollow a pattern.And this in turn should result in a summarizingdocument which can then serve as a guide for researchand practice throughout the country. We think - and I'm quite frank and boldabout what I'm about to say - we thinkthat the prestige and the brain power nestledright here which might result in a documentwhich says this is a set of useful guides for people who are interestedin teacher selection might then serve as an important stimulantfor the country. We hope it moves in directions that can'tbe predicted right now. But above all, we hope it moves becauseright now the whole field of teacher research and teacher selection isunfortunately stagnant and very vitally important. Thank you.

Paul Denn: I'd like now to introduce MissGertrude Unser, who was chairman of our Board last yearand who will lead tonight's panel discussion.

113 Session No. 1: Teacher Selection Mothodsin the United States

Chairman: Gertrude E, Unser, Boardof Examiners

Teacher Selection Methods inLarge Public SchoolSystems in the U.S.: Gerhard Lang, MontclairState College

Teacher Selection Methodsin 62 School SystemsAffiliated with the MetropolitanSchool Study Council: Perry M. Kalick, Hunter College ofthe City University of NewYork

Gertrude Unser: You know, I'm an oldschool teacher and Every I think one neverreally overcomes the habitsof a lifetime. school teacher assumesthat the class has donethe homework but, just in case, she thentells them what theyshould have learned. called I'm assuming that youhave read the two publications, one Public School "Teacher Selection Policiesand Procedures in Large Policies and Pro- Systems" and the othercalled "Teacher Selection cedures in School SystemsAffiliated with theMetropolitan School Study Council. "' The first was the majorpiece of research under- certain size, those taken and dealt withpublic school systems of a piece of having 12,000 or morepupils. The other was a parallel research related to andgrowing out of the first,but dealing with smaller school systemsthat were relatively morefavored economi- the study cally. I think that each of usbrings to the reading or of this kind of document acertain predisposition, butif you have read them carefully, Ibelieve that certainconclusions are quite has been considerable ex- inescapable.Despite the fact that there perimentation in the curriculum,methodology, and organizationof schools in recent years,we're struck in readingthese documents that in the (and we know from our ownexperience too) by the fact procedures used to selectteachers we're stilloperating in a sort that many of of horse-and-buggy age. The conclusion is inescapable expediency, that is, the techniques used arereally dictated more by time or considera- the availability ofapplicants at a particular tions of cost in timeand money, rather thanreasoned and knowl- selection procedures. edgeable considerationsof what are the best I think, too, that weall know that there is adearth of research other criteria. in the validation ofselection procedures against members of my Board I am particularlyinterested, and I think other the pressures in a periodof are, inunderstanding the effect of teacher shortage uponselection procedures. There are many more things that strike one as onereads these documents. I don't want have read them and I to go into them nowbecause I assume that you but assume alsothat the following speakerswill talk about them, just to refresh your memoryconcerning the proceduresand conclu- who was sions of these studies, we aregoing to call upon Dr. Lang, Examiners when this study was a researchassociate with the Board of being made, to discuss verybriefly the teacher selectionprocedures Profes- in large publicschool systems. Dr. Lang is now an Associate We'll then call upon sor ofEducation at Montclair StateCollege. Dr. Kalick to discussthe pariiel study, theselection procedures used in smaller schoolsystems that are morefavored economically. There are certainstriking resemblances andcertain differences. think they will be Their presentationswill be brief and then I happy to answer any questionswhich you may have. Now we'll ask Dr. procedures used in large Lang first to discussthe major study, the public school systems.

Gerhard Lang:I see that Miss Unseroperates under the namely, that people do same illusionthat we do in the colleges, most of you have had a their homework. We hope, however, that chance to browse through thereport on our study oflarge public school systems in theUnited Statesl or have atleast digested the summary of thisstudy. My purpose here is notto recapitulate the to point out mass of statistics,but rather to refresh your memory, regarding the a fewsalient features, and to answeryour questions various aspects of thisstudy, which is the pointof departure for our conference.

We conducted a surveyof 320 large publicschool systems, defined as those systemswhich have at least12,000 pupi101 with respect to 15 areas ofteacher selection policiesand procedures, namely:

1. Analysis of existing staff resources 2. Preparation and use of jobdescriptions 3. *Resources used in therecruitment of applicants information re- 4. Means of givingprospective candidates garding the school systemto which they areapplying 5. Non-local selection ofteachers 6. Use of the applicationform positions 7. Professional preparationrequired for teaching 8. Use and follow-upreferences etc.) 9. Use of examinations(written, oral, physical, 10. Interview techniques inteacher selection 11. Classroom observation ofcandidates

1Harry B. Gilbert, IsidoreBogen, Gerhard Lang, andPerry M. Kalick. Large Public School Sys- Teacher SelectionPolicies and Procedures in Research emsiri the United States. Unpublished report, Cooperative Department of Health, Project No. §:77347 Officeof Education, U.S. Education, and Welfare. 1966. 63 p. 12. Use of eligibility lists 13. Timing of notification of appointment 14. Declination of offer of appointment 15. Appeals from the decision of theselecting authority.

What were some of the majorfindings? We found that only about 25 per cent of the respondents preparejob descriptions for teaching vacancies and only5 per cent use a specific job descrip- tion form. Less than 13 per cent of theschool systems give exam- inations as part of their selection process. Physical examinations are required by.about 60 per cent of the schoolsystems, or con- versely, 40 per cent do not require aphysical examination, indeed a mostsurprising finding. All large school systems interview can- didates. About 30 per cent of the systems almost never oronly oc- casionally train their interviewers.While the time allotted to the interview ranges from 10 minutesto over one hour, the majority of school systems(55 per cent) devote between 20 to 30 minutesto their interview. In this short time interval,interviewers are ex- pected to assess up to 13characteristics of the candidates, e.g., "ability in the subject matterthat the candidate proposes to teach," "logical thinking," "attitudes towardshis work," "potentiality for professional growth," "personal appearance,"and "philosophy of edu- cation." One may well question the validity aswell as the relia- bility of the interview conductedby the typical large public school system. References are almost universallyrequested from former employers and college professors. Surprisingly, however, 25 per cent of theschool systems ask for testimonialsfrom friendp of the candidate, a ratherquestionable procedure. What about sam- pling the candidate's teachingability?About 6o per cent of the school systems do not observecandidates, only 20 per cent of the systems tend to make one observation,and 6 per cent apparently make two observations. Despite our chronic concern aboutattract- ing people to enter a teaching careerand our considerable invest- ment in the selection process,large school systems tend to notify candidates rather late of theirappointment. May seems to be the modal month of notification. Moreover, the larger the school sys- tem, the greater the tendency tonotify candidates later. One can only conjecture as to how manycandidates are lost because of the lateness of their notification.

These and the many other findingsof our study are sub- ject, of course, to a number oflimitations.We have relied on a detailed questionnaire as ourdata-gathering instrument and have assumed that the respondents gave ushonest and correct information. We must also bear in mindthat the conclusions drawn fromthe data are pertinentonly to the population of 320 largepublic school systems which had returned useablequestionnaires. The findings cannot be generalized to populations ofother school systems.

116 Nonetheless, the analysisof our data left uswith a representative in- strong impression,reinforced by our visits to dustrial concerns, such asAmerican Telephone andTelegraph, Standard Oil of New Jersey,Metropolitan Life Insurance,Macy's, Authority of New York. The New Jersey BellTelephone, and the Port impression is that, comparedto generally acceptedpractices used in industry to selectpersonnel who are at aprofessional level in large similar to that of teachers,teacher selection practices inadequate and un- public school systems appearto be by and large better efforts are expended sophisticated. It seems that more and than teachers for our to select first orsecond level supervisors large public schoolsystems.

Realizing that there exist many gapsin our knowledge of practices," we haw what constitutes"effective teacher selection concluded our report with aseries of 16 suggestionsfor further We are optimis- research. Our survey representsonly a beginning. sessions will bring tic that this conferenceand its brain-storming teachers, so vitally us closer to ourgoal - to select qualified Thank you. necessary inthis critical period ofAmerican society.

Perry Kalick: The questionnaire thatDr. Lang referred school systems in the to was also used in astudy of a group of Metropolitan School Study metropolitan area. These systems are the New York City and Council Systems(MSSC). They're clustered around to do re- they've engaged TeachersCollege, Columbia University, 22 search for them. Two of the schoolsystems were in Connecticut, of in New Jersey, and38 in New York.What are the characteristics these school systems? On an averagethey spend about two-thirds Their more perpupil than the large publicschool systems (LPSS). abou. 51 median size is approximately150 teachers and they have teachers per thousandpupils as opposed to38 teachers per thousand pupils for the LPSS.

What are the main findingsof this study, whichcompared those in the teacher selection practicesin the MSSC systems with the inter- LPSS?As is the case withthe LFSS, the MSSC systems use view as the prime selectiondevice. The MSSC systems make a greater effort than theLPSS in extending theradius of search for candidates; they make twice asgreat an effort as theLPSS in the and somewhat more search for candidatesbeyond a 500 mile radius MSSC systems exert than twice the effortbeyond a 1000 mile radius. proportionately three andone-half times the effortto make at least Ninety-two per cent of one classroomobservation of a candidate. candidate as con- the MSSC systems telephonethe recommender of a attempt to trasted with 66 per centof the LFSS. The MSSC systems

117

;4344iit,' interviews by theprimary use of a improve thereliability of their There has beenenough researchto indi- committee ofinterviewers. consensus ofthe committeeis more cate that adecision based on a the decision. As comparedwith effective than oneindividual making twice as many MSSCsystems will em- the LPSS,proportionately almost selection ofcandidates. When ploy the committeeapproach in the committee, they willinvolve the prin- the MSSC systemsdo employ a frequently and thesuperintendent ofschools cipal four times as is a functionof readily as the LFSS. Of course, this five times as systems as op- Forty-five per centof the MSSC smallness of size. candidates of their posed to 23 percent of the LPSSwill notify selection by theend of April. The smallerschool It's not allone-sided, however. interviewers who aretrained in the systems are lesslikely to have cent of themtrain inter- conduct of theinterviews. Some 1i5 per 64 per cent of thelarge schoolsystems. viewers ascontrasted with to interviewblanks are to the LFSS,check lists, aid As opposed the rule, in the MSSCsystems.. very muchthe exceptionrather than one-sixth the usethat LPSS make of The MSSC systemsmake roughly selection process.What are the im- data processingsystems in the advantages inherentin smallness plications here?Very likely, the comparatively highexpenditure level per of size coupledwith the account for thesuperiority of MSSC pupil are thefactors which respect to the large publicschool systems with school systems over with a practices that Ihave cited. A small system the selection to level per pupilmight be in abetter position high expenditure the selection proportion ofadministrative time to provide a greater found that there arecertain Even in the LFSSstudy it was process. Certain advantages advantages of thesmaller of thelarge systems. Here, real smallnessin size coupled accrue tosmallness in size. level per pupilincreased the chances with a veryhigh expenditure number of recom- teacher selection. We came up with a for effective There is a panel suggestions forfurther research. mendations and However, at this morning to considerfurther research. on Friday would appear tobe like to pointout a few that point I just would greater has been discussed. First, is there a pertinent to what systems desirable practicesin those school clustering of more Also, at whichpoint in which rank high on acriterion of quality? to introduce adata school systemsize does itbecome feasible process? And, how doselec- processing systeminto the selection systems comparewith other systems tion practicesof MSSC school large schoolsystems? which are clusteredaround other very

Discussion

have some questionsthat you'd Gertrude Unser: Now if you I think they'dbe happy totry to like to addressto either speaker, be based notonly on theirbrief answerthem. The questions may

118 presentation, but en your study of thedocuments to which they re- ferred. They needn't necessarily be questions;perhaps you have comments that you'd like to make.

Charles Mackey: I'd be very interested in having Dr. Kalick elaborate further on the useof data processing on teacher selection device.

Perry. Kalick: By the use of data processing I meanto be able to develop cards, programcards which list characteristics, qualities of the staff that you currentlyemploy as well as the need in various vacancies as they existthroughout the system.The match- ing of these is made easier by havingdata processing particularly in an exceedingly large system likeChicago or New York City. The data processing also allows forarticulation; with job descriptions, needs can be tied in with characteristicsthat have been established for each candidate or teacher.

Charles Mackey: Has this been used successfully sofar? The reason I ask that is not tobe facetious, but that the frequency of divorce among sponsors who havebeen matched according to data processing reveals a higher rate ofdivorce than in any other type of matching. I was wondering if this would applyto teacher selec- tion as well.

Perry Kalick: I'd like to think that the emotionalfac- tors here are not involved, althoughthe emotional factors in work- ing in a specific type of classroom are verymuch involved. And this is most important - that weidentify the characteristics of the vacancy with the characteristicsof the teacher if we know the characteristics of the teacher.

Charles Mackey: Is this process being used now?Data processing in teacher selection?

Perry Kalick: Yes, in only 5 per cent of the MSSCschool systems and some 29 per cent of the largepublic school systems.

Charles Cogen: I guess this is a question thatapplies to this entire conference not only what wehave heard tonight. And perhaps I shouldn't ask it, but maybethe answer: is obvious in the light of what Harry Gilbert said in thebeginning. Has there been any attempt atall anywhere to make any study of the outcomesof the examination system - that is, theproduct, any correlation be- tween the competency of the teachers as shown onthe job and any of the processes used in the examinationsystem? Do you have any such information?

Perry Kalick: Charles, I think if we had the answerto that we wouldn't be here. The fact is that there are very few studies of the predictive validity of anyselection devices in the teaching field. It's our hope that we will stimulate this and

119 we are going todevelop this point at much greater length in the next day or two so I don't want to belaborit. Let me just point out that we've made a few little passes atthis and we have some tentative findings - it's not our purpose to gointo those now - that lead us to suspect that we have somepromising paths to pursue; but some of the basic research on it, evenreliably assessing what happens in the classroom, are still to be done. They have been opened up and started.

Harry Rivlin: I'm wondering whether there isn't abasic fallacy underlying the studies and the proceduresof this and every other teacher selection agency. There is a sort of implication that one knows what a good teacher is, and if oneknows what a good teacher is, then one can test to see whether a personis or isn't a good teacher. Various attempts to define what a good teacheris have failed - some more expensively, some lessexpensively. Instead of selecting good teachers, the school's jobis to develop them. The selection of teachers is basedbetter not on what happens before a person beginsteaching, but after. As the various criteria were read off, the one thing that I don'tthink I heard was an evaluation of past performance. There is another kind of agency that selects teachers, our universities.When our universities select their faculty, the one thing that they do mostintensively, if it's an in- stitution that cares at all aboutselection, is to make a thorough study of the past performance ofthe candidate. They don't check his eligibility because there are agreat number of courses being taught in college which weren't availablewhen the members of the faculty got their doctorates. It isn't, "Has he had the courses in this area?" but, "Can he learn them?" Most of the university's best effort goes into the evaluationof past performance. In the case of beginnersin college teaching, they can't evaluate past per- formance, but.the universities willevaluate, and evaluate rather rigorously the performance before a personis granted tenure. In short, you have the conflict betweenthe need to be legal, the need to be impartial, and the need to beintelligent about how you find the best person for the job.When the need for new teachers is great, you're not really selecting,you're rejecting those who are obviously unsatisfactory for one thing oranother. Does it make much sense to spend so much effortin determining whether this fel- low gets an 82 or an 81 - it'sreally a pass or fail. In short, I wonder whether the emphasis cannotbe put first on judging a per- son's potential by what he demonstrates,making the major part of the examination not the pre-servicepart but the post-service part or the in-service part,and in the case of people with experience, developing ways of evaluating what theyhave done rather than try- ing to disregard all that, or atleast placing very littleemphasis on it.

Perry Kalick: I wanted to make a shortcomment - I think

120 this is why we're here.We're going to study the implicationsof this. As Dean Rivlin says, if wedon't know what characteristics of a good teacher are, how can weselect?I think if we want to shift that entire fallacy to thewhole business of teacher prepa- ration we might say let's not do anyteacher preparation either, because we don't know what a goodteacher is. I think this is what we're going to develop in the nextfew days.

Jay Greene:On Dr. Rivlin's contribution,the plain fact is that selection is taking place -selection of teachers and selec- tion of civil service workers andselection of people for jobs in industry. It's being done because there are,in somesituations, more people who wantjobs than there are jobs. Now the question then arises if you are going tohave to select - if you have 3 people applying for one job,whether it's for teacher or principal or chairman, thenwhat is the best way of selection?Dr. Rivlin's assumption of throwing all theweight on the preparation and the post-appointment feature would bewonderful if you didn't have more people applying than you needed. Furthermore, it doesn't consider another practicality - that if youappoint somebody to a school in order to observe him in action,if you appoint the wrong person,the damage that may be done by thisindividual in a learning situation for a year or two is unfair topupils. Looking back at Mr. Cogen's comment - he asks what studies havebeen made of our evaluation. One of the difficulties ofsuch a study is that if you aregoing to compare what youbelieve to be a valid selectionprocedure with an invalid selection procedure, it meansthat you have to put the in- validly selected people into jobs sothat you can then compare them to the people that you had validlyselected. In basic research in Publi9 administration thisis morally wrong. How can you in con- science say, "These people webelieve will make good principals.We will appoint them. These people we don't believewill make good principals but we'll appoint themfor a study's sake in order to compare the good tothe bad." This has been one of the greatdiffi- culties of this sort of research,although there are some others. I'm delighted that we're focusing on somebasic questions that we seldom get to discuss.

Gertrude Unser: I'd like to comment on oneaspect of what Dr. Rivlin said. As I understand it, he wasasking among other things, why there was not a greateremphasis upon an evaluation of past performance.The vast majority of the peoplewho are selected for teaching positions have no pastperformance, unless we consider their college preparation as theirpast performance. Now I see two problems associated with selectingteachers on the basis of college records. First, we have some 1200 teacherpreparation institutions. Those of us who have dealt withmarks and recommendations have learned how unreliable these ratingsand recommendations are, what

121 great variations there are. Standards differ widely. Second, al- though we confess that we have no proved meansof validating selec- tion procedures against performance on thejob, I think we have even less in the way of establishing therelationship between success in college preparation and success on thejob. These two are quite different things.

David Darland: I hope that we don't lose track ofwhat Dr. Rivlin raised, because I think heraises this as a process that be- gins in the teacher education program,if I heard it and reaches be- yond what we now consider to be thetermination of the teacher edu- cation program. Is that what you're saying?The selection process?

Harry Rivlin: I'm sorry. I must not have said what I meant to say. I wasn't thinking of the teacherpreparation as the basis for selection, but I'm saying youhave in a sense two dif- ferent groups. You have an experienced group and aninexperienced group. The experienced group consists ofteachers who are coming up for administrative positions, or peoplewho've been teachers or ad- ministrators in other cities. Haw can you evaluate past performance where there is no past performance?Where there is no past perform- ance, as in the caseof a new teacher or a, new principal, Iwouldn't rate him on the basis of his recordin college. What I'm saying is that when you appoint him, rememberthat you're appointing someone without a past record and you mustevaluate the "past" that he is creating in his classroom.I suggest that you select your new people as well as you can, but rememberthat you are selecting only risks, and that the important partof the evaluation comes at the end of the initial appointment.

Jules Kolodriy: I would direct my attention toseveral of the comments made here. First to Dr. Greene's comment thatsociety doesn't want to set up a researchtechnique by putting incompetent people into the field.We have school systems, (as we'vebeen told), that have different kinds of selectionprocedures. It seems to me that we ought to determine whether ornot the products (mainly the students who have been turned out bythis institution called public education) are better, the same, or worse,depending upon the selec- tion procedures. To my knowledge, the colleges havemade no such indication that those school systemsthat use a particular kind of selection procedure tend to give thembetter students than those that use a different kind ofselection procedure. I haven't heard this come from industry, either.We must remember that while we don't have the intangibles andthe emotional factors and possibly some of thecitizenship factors involved in education,the fact is that in terms of the things thatmake youngsters employable,that make them eligible for academictraining, I would asse...t in nowise depends on whether yau'Use oneprocedure or another. In other words, there are good schools that useprocedure A and there may be good

122 school systems that use procedureB. It seems to me that we aretry- ing to determine whether theprocedures are sound in vacuo. If the procedures are sound, they'llturn out good products. If the proce- dures are unsound, they won't turnout good products. We have to follow the conveyor belt of what happensto the students who have been selected by procedure A, byprocedure B, etc.I would assume that they get lost in the general reviewof our culture and society and the employment world, the industrialworld, the commercial world and the collegiate world. Consequently, it may well be(if I remember my undergraduate daysand Mills Canons ofInduction) that these par- ticular factors that we'refocusing on are really irrelevant to what we're concerned about.

What we are more concerned with iswhat goes on in the classroom. Therefore, this relates to what Dr.Rivlin has been talk- ing about. In other words, maybe we'resetting up a straw man and we feel that wehave to have certain kinds ofprocedures because traditionally we've used them. I do concede that the kindof selec- tion procedures that are not exclusivelybased upon subjective fac- tors will tend to eliminate discriminatoryfactors, in terms of race, religion, general attitudes and appearance,possibly. A system that relies exclusively on interviews maynot have as many long-haired teachers as the system that relies upon aneffective written examina- tion; but, I'm not so certainthat this is the thing we ought to direct our attention to Frankly, I'm not impressed by anyof the research and I would not want to appropriatelarge funds for such re- search if I were a member of a Boardof Education. I would look at one thing primarily as youfollow the products of your schoolsystem which uses a particular kindof institution for selection:Are your products better, worse, or the same ascompared to the products of other school systems that use differentkinds of selection procedures for their staffs?

Gertrude Unser: I'm addicted to responding toeach speaker. But I think you say verytruly that the ultimate, the endproduct of of all this, is the student andwhat happens to him. But certainly the quality of the teacher is only oneof the many factors which forms the student. We aretrying to find out what makes a good teacher and how a good teacher makes asuccessful student but to say that this is the only factor isoversimplifying it enormously. There We are so manyother factors which enter into astudent's success. have reached the end of the timeoriginally allotted although I must say we did startrather late. Is there anyone else who has aquestion or comment?

Bernard Berger: I wanted to avoid the questioningof the research done except that all I wantto say is that I think what has been done is valuable and Ithink we should know preliminary to any further research the characteristicsof school selection systems as

123 WIPE+ ...... -110

this presented in the twostudies; but I was alittle unhappy, and attempt at relates to what Mr.Kolodny said, thatthere wasn't some find out something aboutcommunity satisfaction the same time to had a so- systems used. For example, where you with the selection in called primitive systemdid the results,the teachers obtained the community satis- this way, did theyperform satisfactorily, was nothing but fied, because up tothis point of theresearch you know various kinds ofselection proce- a seriesof facts, that there 44re dures used in varioussize systems. school systems what Harry Gilbert:All we did was to ask If we would do they say they do interms of selectionof teachers. lot better but it over again today wecould do that little survey a Now I'm sure we canget all we did was say,"What do you folksdo ?" including this all kinds of leadsfor how that couldbe followed up all of us that vital one, but at this verymoment I have to remind ourselves to theobjectives of all we want to dotonight is orient mainly recite to you,review for you, what we this conference and of started to do. Let's save all thisbecause I think every one the program laid out these questions will come upin the course of for the next day and ahalf. This meeting Paul Denn: Thank you, ladies andgentlemen. is adjourned.

121 Session No. 2: Personnel Selection Practices in Industry -procedures currently utilized - validity of selection procedures - relevancy cf industrial selection practices to selection of teachers

Chairman: Jay E. Greene, Board of Examiners

Paul Baker, Standard Oil Company of New Jersey Douglas W. Bray, American Telephone and Telegraph Company

Paul Denn: Good morning. Yesterday we did a little brain- storming and we did review some of the basic questions in connection with the selection of personnel; some of us asked, "Should we select??? Others, "Can we select???Some others said, "Who should select??? And, finally, "How should we select??? We brought together representatives of industry, who select personnel for their purposes; civil service, and so on today. And this morning we'd like to hear about procedures that have been feasible or successful in both business and in civil service to see to what extent they can be helpful to us in the selec- tion of school personnel. To begin with, then, this morning, I'd like to introduce my colleague, Dr. Greene, who will, in turn, introduce his panel.

Jay Greene: At one time, it was thought that the problems of selection in industry were remote from problems of selection of pro- fessional personnel in schools, and while this might have been true years ago, more and more the problems, as I reviewed them, seemed to be close together: close in their complexity. We often over- simplified their problems, I think, but their problems of staff se- lection are complex. I'd like to introduce our first speaker, Dr. Baker, who has a strange title for Standard Oil of New Jersey, Ad- visor on the Social Sciences for Standard Oil of New Jersey. I had to read that a few times because furthest from my mind was the in- volvement of Standard Oil of New Jersey in the social sciences. In this capacity, he does consult with managers of affiliated companies around the world on problems of selection, placement, evaluation, management development, sensitivity training and all the other good things that a good psychologist is interested in. Dr. Baker.

Paul Baker: Thank you very much, Jay. I will try to follow the pattern set last evening in the beginning session by keeping my remarks rather brief, hopefully to the point and hopefully to open myself up for lots of pointed questions in an inquiry period to fol- low. As Jay indicated, I think that we will find in the course of these discussions that the problems of selection in industry are not materially different from the problems of selection any place else. I'm fairly confident that Standard Oil's selection procedures are not, in any real significant way, different from any other industry's

125 much the same thing. I selection procedures. We all do pretty Oil does a much, much choose to believe,however, that Standard better job than anyother company in itsselection.

The kinds of things wedo are the kindsof things that you needs, try to antici- do in your teacherselection. We analyze our that we're' going toneed. We do pate the numbersand kinds of people terms of the long-range it in terms of theimmediate future and in what our needs aregoing future. We try to keep inmind at all times selection problems. to be twenty yearsfrom now, as we consider our about the changing Quite literally, thisis true. We are thinking of people that we're that's going on in ourindustry and the kinds the right mix of going to have to selecttoday in order to have will be twenty years talents and skills to manthe industry, as it going to be considerablydifferent in many im- from now, phich is needs, we go portant and fundamental ways. Having anticipated our recruitment; we advertisein through all of thetraditional steps of colleges and universities, the journals, we scurryabout the campuses, people in coming towork trying to interestthe young professional aggressively recruit We even in somecritical areas rather for us. true for secretarial help. This is particularly in the high schools is tight for example, wherethe clerical situation in New York City, candidate in ap- at the presenttime. Having interested some young all cases, and there are plying for work in our company,we in most battery, tests, quesion- exceptions to this,administer a selection fairly elaborate test naires, and again inmost all cases these are with proven validityand in the batteries. They are well-researched and overseas. testing programs bothin the United States

I think this isthe one place where wehave concentrated research, the kind ofresearch and have been ableto do real sound thesis research, andin that would stand up asacademic research, or develop selection tools or se- all cases, we findthat we are able to And our test- lection tests which do a very,very goodjob for us. to the tests that we ing programs, the teststhat we have developed Admittedly, many use, a catalogueof them is a fairlythick book. languages of the world, of them, it's the sametest in all different but still they aredifferent tests. .60 We usually aim for avalidity coefficientof at least around .70. I think that and seldom fail;mostly our validities are distribute all of our validitycoefficients, the modal if we were to into value would probablybe around.65, but we have many that run the high .70's whichis good. rather extensively onthe We, of course,depend, as you do, the interview process interview. We have done abit of research on interview data cast intoquantita- and we find thatwhen we can get validity. But tive form that it too cancontribute to our overall

126 this also depends upon the amount and quality of training that is given to the interviewers and in many cases, in many of our instal- lations, we have trained essentially the whole of the management staff in selection interviewing techniques and in cases like this and in these cases where we can get good data from interviewers, we find that although the collective validity of the interview informa- tion by itself is rather modest, perhaps in the .30's, this can add a significant amount to a statistical prediction, if you will.

More importantly, and I think at a point there are many differences in the selection procedures we follow, deriving from the differences in the nature of the problems that we face, 'differences between industry and education, but one of the things that I think is a crucial difference is the fact that we consider the selection process not merely in the sense of selecting an individual once; ad- mittedly, this is the crucial selection, that is the new employee who goes to work for our company, but there is a continuing process of selection from there on, and in many, many steps that one takes, one is faced with another selection procedure. That is, a craftsman, for example, is initially selected to enter training to become a craftsman in some future years and it may be a very brief period of time, four or five years, up to ten or twelve years, that this in- dividual will again be faced with the selection procedure for fore- manship training. And again, at this point in his career, he is submitted to exactly the same steps of being advised or recruited, if you will, of the opportunities, being interviewed, being tested, being examined rather thoroughly, and his work record then also be- comes part of the selection procedure, and this can continue on up into still higher ranks of management.

The same thing is true to a lesser degree of clerical help; it is very true in terms of our selection of young professional people; as they are being considered for management positions, they get probably the most exhaustive testing program of any in our com- pany in our management testing program.

We are particularly concerned with the problem of select- ing managers and it's the one selection process upon which we have concentrated in the last ten years, just now doing ten-year follow- up study on predictions that we made ten years ago about managers. We have done other follow -up studies, but not based upon this particu- lar selection package and in our follow-up studies, we demonstrated that we do have long-range predictive validity; again, in the order of magnitude of something over .60, and we have extended this particu- lar management selection program; we are in the process of extending it around the world, again in all languages. We've completed the study in Spanish in Latin America; we have in progress studies in Den- mark, Finland, Norway, Sweden, Holland, and the Scandinavian area.

127 the study in Canadaand in the UnitedStates, and We have exhausted procedure, this has ahigh de- have again foundthat as a selection gree ofvalidity. continuing selec- I've mentioned thesuccessive steps of between business andindustry. tion as one ofthe major differences probably other differencesbetween business and I think there are differences in perhaps our em- industry that wouldgive rise to some opportunity that we havefor placement; phases; that is,the broader able to achieve agreater degree in other words,I think that we are validity in astatistical sense because we canaccom- of predictive selection and when you broader means oftalent in our modate a much who with the problem onlyof selecting aclassroom teacher are faced rather than as a is going to be aclassroom teacher, as a career, trying to select a person whois coming in withthe idea that we are fail, in a sense,of finding good person fordevelopment and when we particular line, wehave lots of a personwho is going to move up a which we can put thisindividual. I don't think you other lines in institutions, so perhaps have quite thatopportunity in educational statistical studies of the glittering successthat we had in our prediction are afunction of the sedifferences. to Well, with thatkind of comment,perhaps I would like questions out to youbecause start the questionperiod by throwing try to inform, anddirecting these I come here tolearn as much as to concerned with teacherselec- questions to thepeople who are most questions that we examinein our tion; and these arethe kinds of But there are thesimple processes ofconsulting with managements. who select teachers,do you logical questions,like those of you hire people whoturn out to be ever makemistakes, do you ever Another basic question,the failures, and, if so,how do you know? teachers, whatdifference does people who areresponsible for hiring in business, when we it make if youmake a mistake? In other words, and we can make itstick, pin down theaccountability for mistakes while to do a goodjob of se- and we make itworth the individual's does the same thingobtain in lecting people andhe knows it, but that if theanswer to the educationindustry?But I would submit make mistakes, and youcan see the first questionis yes, we know we most surely existsfor you to these mistakes,then the opportunity have done in termsof determin- do the same kindsof research that we good teacher, whydid we ing what are thecharacteristics of this to avoid it inthe future. make this mistakeand what we may do We had agreedthat Jay Greene: Thank you very much,Paul. presentations before thequestion both speakers wouldmake their in hearing Paul's I was interested, asI am sure you were period. validity and the problems comments on thedifficulties of predictive mistakes, and thatbrought to mind thisbrief anec- concerning making American dote of a very youngassistant professorwho was teaching

128 ...... NINWOor0.1..10,401.04.10..400,041110.

wrinkled and History in auniversity and he hadin hi? class a student who was therefor a degree, whohad a gnarled old Chinese professor asked the of mind. This young history philosophical bent Revolution of1776, class to namethree good resultsof the American The elderly Chinesegot up and and he calledthis elderly Chinese. said, "It's too soonto tell." validity. This is one ofthe problems ofthe predictive teacher and it would seem,on the faceof it, easy to You select a depending upon theproduct, say, 'filets agood teacher, abad teacher, how well they do.If he's a i.e., measuringthe students to see students will dowell." Then the anecdote comes good teacher, the appraise begin to ask yourselfin what ways will you to mind and you attitude? Their factualknowledge?Their changes of the students. what will this?Is it theimmediate effect, or How do you measure All of ten or fifteen years, orin a lifetime? happen to them in including key and troubles allof us who are here, this is complex, Bray, who is in industry. That leads.me to Dr. people in selection Telephone and Tele- the Director ofPersonnel Researchfor American institution, and whcprepared for graph Company, agrand national select universities. Doug Bray, his position atquite a number of you havethe floor. When I prepared mytalk for this Conference, Douglas Bray: devices used to se- present an overviewof the various I planned to with and their relativedegrees of success, lect business managers, in some of themfor the the thought that youmight see usefulness to your discussionlast problem of teacherselection. As I listened in that shortsession evening; however, I wasimpressed that even It seemed to raised most of thecrucial selectionissues. you had contribution byaddressing therefore, that I couldmake more of a me, central problem andfocusing myself to what Iconsider to be your technique which mayoffer a solutionto on onemanagement evaluation that problem.

the pleasure ofattending an excellent sym- I recently had American Psycho- Dr. Gilbert andDr. Lang at the posium organized by of meetings, a symposiumalso on the subject logical Association Hagen of One of thepanelists, Dr. Elizabeth teacher selection. difficulties in the Teachers College, read somequotations about the for research. These quota- selections of teachersand the great need made in this roomlast eve- tions sounded verymuch like the remarks old! My prediction is ning and yet they werethirty to forty years research problem, you can that unless you face upto your central twenty years withoutgetting have meetings likethis one for another much further ahead. of course, whatpsycholo- Your centralresearch problem is, Unless you cansomehow agree on gists call the"criterion problem."

129 teadher an aver- methods for determiningwhether a person is a good rest of age teacher, or apoor teacher, youmight as well forego the problem, I am probablygoing your discussions. In getting into this to impinge on the papersof some of the panelistsscheduled further going to speak about along in this Conference,such as those who are I hope you problems of validity andthe design of neededresearch. will forgive me if I goahead anyhow. to de- The management evaluationtechnique which I am going scribe can serve as aselection method. This is in fact the way in which we use it. It seems to me, however,that there might be prac- selection of teachers tical difficulties inusing the method in the which would be very hard tosurmount. touch on these later. might yield a cri- More important is theprobability that the method terion, and if you could agreeto accept this criterionthe way would be open for research onthe validity of thekinds of predictors biographical data, used in the selectiondecision, such as interviews, college grades, ratingsin practice teaching,etc.

We in business werefaced with the same typeof criterion and gone over problems as you alie. Various appraisal plans have come the years, and yet few are eversatisfied that we can rateexcellence that or- as a managerwith high, reliability. There is much evidence It is likely dinary supervisoryratings contain substantial error. discarded because that many promisingselection devices have been they were "validated"against poor criteria of success.

We believe that wehave found at least apartial way out of We this dilemma in thetechnique we call the"assessment center." management to agree on a developed this method byfirst asking higher distinguish between number of characteristicswhich would usually and definition and re- good and poor managers. After much discussion Among them were definition we wound upwith a list of25 qualities. such things as oralcommunication skill,decisiveness, leadership continue the list ability, independence,etc. You could probably yourselves and come prettyclose to what we have.

After these qualities wereagreed upon, we devisedtech- to these characteristics. niques which wouldelicit behavior relevant pencil tests, but These techniques includedinterviews and paper and Usually there are threeof more important arethe "simulations." administrative task, known asthe In- these. One is an individual been used in the Basket, a technique,incidentally, which has also exercises- - study of schooladministration. The other two are group discussion. The a miniaturebusiness game and a leaderless group candidates at a total assessment runsfor three days, and twelve time are assessed.

Assessment staffs arecarefully trained toadminister the

130 the 25 managementcharacter- techniques, observebehavior, and rate is rated andthoroughly discussed at a istics. After a candidate the candidate in oneof four cate- staff meeting,the staff places "acceptable," "less thanaccept- gories: "more thanacceptable," Staffs show a highdegree of agreement able," and"unacceptable." in this finaljudgment. operated in the Our firstassessment centerwhich actually We now have over50 centers selection of managementopened in 1958. have been throughthem. This trowth and about25,000 men and women any high level decreebut because has taken placenot because of observed the processand judged themethod to local line management in comparatively expensive, ahighly valuable tool be, even though that the assess- of better management. I might note the selection They are, by and ment staffs arenot professionalpsychologists. most of whom havenot attended large, second-levelmanagement people, college. develop an assessment cen- I would liketo propose that you ability. The qualitieslooked for ter for theevaluation of teaching and the assessment exer- would, of course,not be the same as ours but I have nodoubt that it cises would mostlikely be different, methods. would be quite easyto arrive athighly satisfactory theory, be used inthe Such an assessmentcenter could, in but there mightbe insurmountable actual selectionof new teachers, of candidates,most of practical difficulties. The great number evaluated at the sametime of the year,would make whom have to be during the Assessment might,however, be done staffing impossible. probationary period. final year inschool or duringthe new teacher's completely on the useof assessment as I don't wantto close the door aselection method. recommendation is,however, that theassessment My definite validate the research criterionagainst which to center be used as a used, in the ac- indices which areused, or might be techniques and would agree, ineffect, situation. The profession tual employment evidence of good, at the centerwould be taken as that performance research could then mediocre, or poorteaching ability,and selection the hope of earlyand great benefit. get going inearnest and with protest- in this room arealready inwardly I know that many is whether You're saying thatthe criterionof good teaching ing. years after taught are effectiveand happy citizens the children of any numberof they're out ofschool, and youthink, I am sure, less than someultimate cri- reasons why youwouldn't buy anything might as well stophaving terion like that.Well, if you won't, you because you aren'tgoing conferences on teacherselection research You must agree on somereliable to get anyproductive researchdone. is practical toget. The assessment and convincingcriterion which it center can providesuch a criterion.

131 Discussion

Jay Greene: Thank you very much Douglas. I have always been an admirer of the ingenuity and the originalityof people in business, especially since an experience I had about a year ortwo ago in connection with someeditorial work. They were talking about the book used in elementary school., Look, Dick, Look. All of you are familiar with that, andthey wanted to modernize it. There was a meeting of someprofessional advisors, and they decided to take the book, Look, Dick, Look, and modernize it by spellingit back- wards. And if you spell Look, Dick, Look backwards, you get anin- teresting title, Kool, KidKool. Since that time, I've been an admirer of the ingenuityoxbusiness people. We have some time, now, for questions, and we wouldbe delighted to refer them to our two speakers. Yes, Al.

Albert Schiff:The question I would like to ask Drs. Baker and Bray is a question relating to theselection of people at the time of initial appointment. At the present time, you were talking primarily of management people; these are peoplealready in positions with your Company, who are working up to managementpositions. For example, teacher candidates are graduates ofcolleges and univer- sities.When you go recruiting in colleges and universities,look- ing for engineers or chemists or business-admajors, do you put them through a testing procedure?Or do you say, he's a graduate of MIT or a graduate of so and so;in other words, what do you do for people at the professional level who are not in youremploy?

Douglas Bray: Yes, we do test college graduates from even the best colleges before employment although wedon't attempt to put the potential recruit through an assessment center.We could not possibly staff up to assess all of our collegerecruits on a pre- employment basis.We are now, however, putting many candidatesfor sales employment through a pre-employment assessmentcenter.

Paul Baker: I would answer the same question in terms of Standard Oil. We do not universally test youngprofessional people. In certain functional categories they aretested prior to employ- ment. In the total sense of all of the peoplehired during the year in the United States, we would be testingabout half, prior to em- ployment, with specialized testing instruments, not amassive test - ing battery, but more or less a screening test. We do, however, anticipate, in the not too d2:.stant future, almostcomplete testing of young professional people.

Milton Gold: I was particularly interested in Dr.Bray's suggestion there that use might be made of somekind of assessment in the student's undergraduate work and inthe first year of teach- ing and I recognize some kind of relationship hereto Dr. Rivlin's

132 remark last nightthat the rating at thetime of initial employment device which is followed might rather beconsidered as a screening has certain advan- up in thetenure period. I think that teaching relationship with the tages over industryin terms of a closer and that it's undergraduate preparationthan is true in industry, for example, to have a very possible for the NewYork City schools, with colleges in theMetropolitan area, in terms close relationship And that it of efforts to do someevaluation prior tograduation. tLis testing of thestudent might be possible,then, to spread out than on a one-day and to get a broadersample than is possible written test plus aninterview and any otherkinds of performance suggestion here of ob- that are required, sothat maybe Dr. Bray's far broader servation in the teachereducation days would give a sample once thecriteria have beenestablished. Baker, first, the Fred Williams: I wanted to ask Dr. interviewers whom you nature of the trainingthat is given to the the spoke about and thenask Dr. Bray if hewould also comment on nature of the traininggiven to the staff ofthe assessment centers.

Paul Baker: The nature of thetraining of, well, let me just give you as a casein point, I'm sure, andI do this because for example, as our I'm sure you'll recognizethe name Anne Roe, She's had a lot of principal consultant ininterviewer training. experience in interviewingand a lot of experiencein teaching and is struc- in the field ofpsychology. And her development program tured around first astudy of all of theother kinds of information prior to an in- that one has availableat the time of interview or interviewer not to terview, such, so thatprimarily you teach an individual that are attempt to make judgmentsabout facts about an more readilyobtained some place else,for example, intelligence. trying to assessthe Why should an interviewerwaste a lot of time test will do a intelligence of an individualwhen a twenty-minute much better, moreaccurate job. which one cannot so But related tointelligence, d thing the degree to which readily assess with a paperand pencil device is intelligence one is goingto use that intelligence, orhow well that is, the train- is applied in apractical sense. So the interviewing is appro- ing is one of anintellectual sort thatis a study of what actual interviewing, priate in an interview,followed up by training, of the mistakes that tape recorded and fedback and an examination mistakes; and it's an in- one makes and anattempt to correct these volved process and afairly involving process. training, using the At the presenttime, we're doing some extending the sametechnique in little video recorder,which is just that you'll find in terms of the technologyavailable, and I think video recorder intraining the future ratherwidespread use of the

133 get an imme- exercises, and particularlyof this type, where you can well as hearing, diate playback and seeyourself, you know, and as in the train- you can seeand hear the kinds ofmistakes that you make would you like ing. Does that answer yourquestion sufficiently or to get into some of thedetails?

Douglas Bray: The training of theassessment staff occupies recommend three weeks, andthe as much asthree weeks; generally, we techniques, of training amounts to, well,learning how to run the to develop a com- course, butthat's minimal. The important thing is meaning of these25 charac- mon understandingof the definition, the teristics, writing reports., sothat others can see what you saw exercise, through the report, becausenot every observer sees every members in the obviously, writing reportsto convey to the other about the group whathappened, so that they canalso make inferences three- behavior and arrive atthe ratings. The last part of this for the staff to week training is adry-run group, a practice group try out their wingsbefore they meet the realcandidates.

Thomas Bransford:I'm prepared tobelieve that Standard the best in the world, Oil has a very fineselection program, maybe to the operationsthat but I am suspicious, orlet's say curious, as However, I will lead to a predictivevalidity coefficient of.75. instance, that if I relatethe can imagine some. I can suggest, for proportion of admittedmistakes in hiring tothe total number hired, in ten, "Yes, boys, we and that I detectand admit one mistake shortcut phi coef- shouldn't have hiredhim." Then I compute a from the per- ficient by subtractingthe percentage of mistakes minus .10 and I come centage of satisfactorynew-hires, then I get .90 synthetic coefficient of.80. Hokever, if I follow up re- up with a kinds jects or if I correlate somescores onselection tests with any afterwards, and I con- of ratings or any kindof objective criteria coefficient sider that the practicalceiling on any one correlation of the test and of is going to be theproduct of the reliabilities the criteria, I justcan't expect predictivevalidity coefficients much above .60, evenunder very favorableconditions. the kind of ques- Paul Baker: That's a complex question, and I would send them tion I used to getfrom my graduate students or no andthen back and say, "Youreduce that to one I can answer yes come back I'll talk to youabout it.t1 Most of the time they'd never with the same question. in front You're asking, essentially,how do I dare stand up claim a validitycoefficient of .75 andhow do we of an audience and Obviously, we're do it?And I don't know howdeeply to go into this. has discussed itin concerned with thecriterion problem, and Doug have devoted probably moreeffort some of hisearlier comments, and we the criterion problemthan we have tothe selection to the study of As you problem or the predictionproblem, predictioninstruments.

134 yourself point out, you musthave a reliable criterion measurebefore you can getanything to correlate withit, but having found a reli- able criterion measurein a statistical sense, say, aneffectiveness in some kind of performance on ajob, then it's relativelysimple to pick up a handful ofinstruments that will scan themental abilities and some temperamentfactors and get a very goodcoefficient.

What we do in general andin our concurrent validitystudies and this is is to approach the problemof the criterion, not directly, continuing dialogue with our an argumentthat we have, it's almost a superiors in the organization.How do we know that you arepredicting success, and whatis that, really, becauseeverybody recognizes that success in businessis a complex concept; successis composed of lots of different things, and we say,well we don't really, butwhat we look at and what we're tryingto predict and for purposesof research, let's accept the fact that welook to the intervening variablesof present and past rewardsfor successful behavioi" in arather gross organizational sense.

In a typical study, andthese have been done many, many times, and for many, many yearswe've been doing this, we pick up variables relating to height,relative position in theorganization, salary history, evaluationhistories, functional histories,and so studies, we had 72 bits of forth. For example, in one of our major criterion information, thingsabout people that reflected,in some sense, successfulbehavior. This matrix was factored andthe first components had to be adjustedjust a little bit to throw itthrough the age factor, so that itwould be correlated zero with ageand, using this as a first principalcomponent, we look at the structure, does it make sense? If it makes a lot of sense,this then becomes a criterion number, such that you canget a score for eachindividual on this criterionvariable and then you lookat people who get big scores and everybodywill agree, yes, if we couldget more people like that, we'd be happy. And this becomes a criterionnumber which then becomes, with a high degreeof a kind of statisticalreliabil- ity if you will buy theconcept of the length of thevector as in some wayreflecting the reliability,becomes a vector variable which we can predict.

Other procedures that we'veused have been the canonical variate. It's pretty much the same sense,that is you have a matrix of criterion information, amatrix of predictiveinformation, and you, those of younot familiar with the canonicalmodel, you essen- tially factor both of thesematrices and then fit the two spaces together to maximize thecorrelation between them.

I've done a couple of studieswhere I've factored the cri- variates and I've done terion matrix. I've calculated the canonical a thing which I call iterative multipleregression,,and this is striking a trial vector on one side and regressing one set of vari- ables against that and then taking the regressed variable and re- gressing the other against that until the thing stabilizes. I haven't worked out all the analytics of this yet, but I know it con- verges not to a first component and ndt to a canonical variate, but to, in a sense, a more practical variate. And again, we come up with a criterion number which, people will agree, yes, that, the people in this bunch here with high scores are the kind of people we like, and they're successful people, and these others are not so successful people. This becomes the thing that we can then compare with all of our predictive variables.

Although I claim only validities in the .70's, in many cases, and these are cross-validated in all cases, in some cases right now we're doing a study where we have six basic groups which, and that's cross-validation with a vengeance, if you will, and not concurrent validity but long range, well five-year validities. I expect to get, again, validities, in the high .70's here. In some studies, particularly when we have a well-researched battery of in- struments and we take it into a foreign language in a foreign coun- try, we find that almost invariably we do a better job there, the validity is higher than it is in the United States, for a variety of reasons; you've got a much wider range of talent, to start with, and our foreign friends, being less sophisticated in these areas, do ex- actly what we tell them to do, so we get very, very good clean data, and they work very hard at producing good data for us.Does that tend to answer the question?

Thomas Bransford: In a way, as you did in your paper and you alluded to in your speech, the idea that you don't regard the selection as a one-time, ten-minute process

Paul Baker: No.

Thomas Bransford: ...but as an accumulation of data over a long period of time. You also said that you have the advantage of multiple placement...

Paul Baker:. Yes.

Thomas Bransford: ... soI assume that part of your success is that you use a lot of information that we'd ordinarily call cri- terion information, and say, well, yes, maybe we made a mistake in his initial placement, but, gee, now, now we know what he can do, what he's good for, and before

Paul Baker: No.

136 Thomas Bransford: ... youget through, you don't have too many people you can't usesomewhere, somehow.

Paul Baker: We attempt very strenuously to avoidthe con- tamination of predictors. We are looking at,always the basic ques- tion, what is it that we can determine aheadof time, without any contamination of the criterion variables with thepredictor vari- ables? If you would examine our research indetail, I think you would appreciate the fact that we doavoid this contamination.

One of the reasons we're doing thisfive-year follow-up study now on this rather massive bunch ofdata involving some 3,000 cases is that we've beenusing these data in the developmental proc- ess of a number of thesepeople and we're just at the point now where the data that we have areuncontaminated, but they won't be very much longer. It's going to be getting at - in other words - things have happened to people as a resultof the predictions that we have made. It's happened at a very high level in the organiza- tion, so that the contamination has been kindof indirect. We will never be able to usethese data in the future for any kind of statistical studies; in other words, our validityis going to go way up, because ofdirect contamination.

William Brown:I'd like to go back to Dr. Schiff's ques- tion and ask a little more from Dr. Bray oninitial screening and testing, after campus recruitment. The situation. I'd like to stress here is, you're a recruiter from a largecity school system, you're critically short on teachers, you're at a universityplacement of- fice and you're interviewing candidates andmost likely in the next three or four rooms are interviewers fromsurrounding attractive suburban communities. These interviewers offer contracts,sometimes based on a fifteen-minute interview.As a representative of a large city, and remember the teachers areselecting the city, and it's very difficult to recruitthem, what testing would you do in that connection?

Douglas Bray: If you are really in a situation inwhich you have very littleopportunity to select at employment, which is the one you have outlined, then I guess Iwould not try to select at employment. I would try to select by attritionduring the first year on the job.We have found, however, that interposing ashort adjunct to selection, such as a mentalability test, does not inter- fere with recruitment. It may be, in fact, that making a job a little harder to get may cause more peopleto be interested in get- ting it. Even if some school systems cannot affordthe luxury of selection at the moment, I still believe thatresearch into how to select better teachers is still of greatimportance.

Rufus Browning:We recently had a change in our rules,

137 We now permit a which recognizesthis factor, justbrought up. without a passing scorein teacher to become aprobationary teacher proviso that theteacher the National TeacherExamination, with the the National Teacher coming aboard sign acontract to take and pass Failing to pass the Examination during theprobationary period. automatic terminationat the test during theprobationary period is recognition of the pointjust end of the period. But this is in brought up. out that the staffingof Perry Kalick:I'd like to point constitutes time, andthat time the A.T. and T.assessment centers is money... We're wasting your money, asastockholder? 1 Douglas Bray: question of the worthof Perry Kalick: No. There's no potentially a spending $300 to$500 per candidate, if i acandidate. Large $100,000100,000 to$200,000, or more, investment fraction of the,perhaps, city school systemsspend only a small might spend. I would like $300 to $500 thatbusiness and industry undergraduate to follow up on Dr.Gold's comment onassessing the I've been supervising training of a teacherprior to entrance. Harlem, in the typesof student teachers fortwo years in East beginning teachers areappointed. schools to whicheight out of ten of supervisor ob- The basic requirementis that the college student teacher fourtimes a semester. I have functioned serve the have devoted a con- "resident" in thissituation in that I more as a teachers; siderable amount oftime to the trainingof these student teacher about twentytimes in i.e., I haveobserved each student morning observationswith the varying situations,including two full cooperating teacher outof the room. have undergonethis I have found thatstudent teachers who in reality aremostly ratedconsiderably above extensive experience that beset a be- average intheir ability to copewith the problems principals of the ginning teacher in adifficult school, by the they are permanentlyappointed. I go along with schools to which However, in assessment centersimulation approach. A.T. and T.'s opportunity to assess a the college situationthere should be an performance prior tothe upper or evenlower student's teaching teaching per- early indicationof a student's senior year. We need an in formance so that we cantailor subsequentteaching experiences weaknesses, or perhaps accordance with astudent's strengths and to student teaching. For have a basis fornot admitting a student teaching may beadequate; for some students onesemester of student students should never others, an entire yearis necessary; and some the formal studentteaching course. I'm suggesting have gotten into phase of that the collegehas a prime role inthe initial screening the teacher selection process.

138 Jay Greene: Thank you. I'm ,afraid that the time allotted to our panel has been used up and I do wantto thank Douglas and their background in industry. We * Paul for their presentation from are fortunatein having around the table people with variedexperi- ence and .background, and wehope that this panel represents the be- ginning of a sharing of ideas, a give-and-takein discussion, in which all ofuswill'be prepared to, perhaps, make changes in some of our basic'thinkia, and perhaps, chart new channels or newef- forts. This is ondu5f the most difficult things todo, to change basic thinking because everybody believes that what he'sdoing and the way he is doing is the best ww. We hope thatthe two speakers here from industry will continue to share theirthinking with us in all the problems that the panels willbring to us, and we hope that you'll participate as freely and as honestly and asdelightfully as you have this morning. Thank you.

139 ,111111,11......

in Civil Service Session No. Personnel Selection Practices -procedures currently utilized -validity of selectionprocedures -relevancy of Civil Serviceselection procedures to selection of teachers

Chairman: Samuel Streicher, Boardof Examiners

Bernard Berger, Departmentof Personnel, The Cityof New York Thomas L. Brarisford, NewYork State Civil ServiceDepartment Raymond Jacobson, U. S.Civil Service Commission

Paul Denn: Could we ask you now tosit down, please, to re- reminded me of an old storyabout our commence. Someone in the lobby doctors bury theirs, lawyersbill mistakes. He pointed out that theirs, and we promote ours. At this point, let'sturn now to the Ird practices that obtainin the various branchesof Civil Service. like to introduce acolleague, Dr. SamuelStreicher, who'll chair this panel. Dr. Streicher.

Samuel Streicher:I think that one reasonfor meeting far opportunity to relax andlisten to away fromhome is that we have an problems are very much what the other fellowsuffers, find that his We also get a like ours, and from itderive a partial consolation. chance to relax in attitudeto the point wherewe're willing to en- listen to the tertain another fellow'spoint of view, and even to there may arise way he goesabout doing things. Afid in this process one's own practices, the assump- a stillsmall voice that questions to tions on which the practices arebuilt, and, later on, urges us is all entertain some modificationof them. This self-reappraisal of personnel in the more important inconnection with the selection Federal, State or local Civil Service jobs,whether they are at the level, whether they arein teaching or innon-teaching areas.

To make the transitionfrom industry to teachingunder from industry to Civil Service isquite a leap. You first must move of personnel there is no teaching in areas inwhich in the selection of control other than a very vagueone. Thus, the Superintendent Schools of a givencommunity will determinefor himself how he'll supportive services, etc., select teachers,principals, directors of and if he assertsthat he has no examinationbut merely interviews has received his somebody, this person, asHarry Gilbert points out, not examination, though hedoesn't know it. The Superintendent is accountable to anybodybut himself and his Boardof Education; and, fortunately for him, theBoard of Education doesn'thold him ac- countable, in the usual sense;if he made mistakes,well, that's too 35 or more years bad. The mistakes willeventually wash out after accommodating and quits orotherwise of service, unlessthe person is separates himselffrom service earlier.

140 6j

When you come to Civil Service, withFederal or State and City laws setting boundaries, andwhen it's the policy of the people to make your decisions reviewable byothers (usually by courts or arises to create elabo- 5 commissioners of education), the temptation rate machinery, to become efficient inits operation, and then to find it downright uncomfortable to make changesin procedure and agonizing to change underlying assumptionsand general design. There's the temptation to continue whatwe've been doing, and to be- come victims of our ownprocedures. That's another reason why get- ting together here, not only with representativesof industry but also with representatives of CivilService groups at different lev- els will help each of us.

I assume that when we talk aboutthe selection of teachers, we are talking in genericterms; thus, teachers would include super- visors, administrators, directors ofspecial service branches.Some reference was made earlier today and latelast night to the selec- tion of people at the upper reaches.Second, we include within our consideration of the selection processthe initial screening as to competence and the appointment policiesthat may or may not facili- tate employment of the better peopleand may keep many a fine person from even making application. I believe that we also implythat re- tention policies determining theconditions under which people work, particularly with regard to morale(so badly abused a word), will come within thepurview of our discussions. It will be easy for any one of us to defendhimself and say, "That's the law." Well, first of all, we're not so'sure that weknow what is the law; even those of us who are in the know, are sureof one thing: we don't know the law fully; and second, laws canbe changed, for they merely represent the policy of the people at a givenstage and the policy may change

Thus, as we proceed to examineourselves we may think of the Biblical dictum, net us comeand reason together," or more nearly literally translated, "Comelet us become wise together.!' The latter connotes a receptivity, amutual impact, and a hope that the spirit of exchanging, ofwillingness to listen, will drive away any temptation atdefensiveness.

The three speakers for todayhave consented, under thedie- tation of the Chairman, to limitthemselves to about ten minutes of presentation. They will not read their papers,though they prepared papers, then we'll openthe meeting to discussion, not merely 0 questions addressed to the speakersbut to a free-for-all, a tearing apart of the ideas presented. The first speaker will be Mr. ernard Berger, who is Director ofTraining in the Career DevelopmentDivi- sion of the New York City Departmentof. Personnel. Formerlyhe was Principal Personnel Examiner of the Bureau ofExaminations of the New York City Department of Personnel. Mr. Berger.

Bernard Berger: The selection of teachers and the other, per- school system generallyinvolve approximately, positions in the in the about 100 differentkinds of positions; haps a maximum of positions - generally, City, Stateand Federal, the Civil Service We run kinds of positions -number in thethousands. the different laborer - the gamut ofpositions from thoseof cleaner, porter, is about 40 percent higher thanthat of a be- whose starting salary which require, as a ginning teacher inNew York City -to positions of experience.So minimum, a Ph.D. and acertain number of years responsibilities in selec- have a very widevariety of you see, we Department of The selectionmethods used in theNew York City tion. that are used inschool Personnel don't vary verymuch from those selection in administrativeselection, or in the teacher selection, described earlier today. methods used in privateindustry that were with all of the kindsof tests We have attemptedand experimented by ourselves andby others in that have beendevised and developed able to serve theCity competent- trying to selectthose who will be We find that wehave the problemsthat the ly, perhapscreatively. much dif- organizations have: they are not very teacher selection have to experiment We've come to therealization that we ferent. methods of experimenting and we havechanged many of our and we are than we quickly in shortage areas,of taking less obtaining people possible resource to normally expect, andof consulting with every appropriate selectionmethod make a betterselection method - a more staffing needs of theCity. I'm not that will fillthe needs, the do employ. I going to talk indetail about thekinds of tests we procedures, just want to speakabout the validityof our selection assignment given to usin prepar- which was one ofthe areas in the ing for thisconference. which is repro- (Mr. Berger now readhis position paper duced on page26.) is Dr. Thomas L.Brans- Samuel Streicher:Our next speaker State Department ofCivil ford, Director ofExaminations, New York of the TestDevelopment Dr. Bransfordformerly was Chief Service. has been United States CivilService Commission and Section of the Psycho- President of thePersonnel Divisionof the New York State President of thePsychological logical Associationand currently is Dr. Bransford. Association ofNortheastern New York. do things and how Thomas Bransford:We can tell how we do things is for ourpurposes, andstill miss the good the way we I with respect toimprovement of teacherselection. mark completely distributed to that my prepared paperwould have been had expected only to the other Now I find ithas been distributed everybody. The and that we do nothave time to read italoud. panel members things the framework inwhich we do ourwork, the various paper tells know about how well weare we try todo, our chagrinat how little we to teacher selection. First, I doing it, andquestions its relevance beyond it to speculate onthe will skim the paperand then I will go things I think are wrongwith teacher selection.

142 If we were talking only about the comparison of various kinds of Civil Service approaches to Civil Service problems,I would have a much simpler topic. I know and could explain some differ- ences tetween the State system and the Federal system, and the City system and I could explain some differences in theAmerican Civil Service system and various foreign Civil Service systems,many of which are based on accidental differences in law,differences in cul- ture, differences in kinds of problems.

Basically, it's almost impossible to summarize whatwe do. We're very much like the octopus, whowas asked how in the world he managed eight legs, and, of course, hewas unable to move, pondering that problem.We are responsible for holding 4500 separatecompeti- tions, separate examinations in thecourse of a year. These are for some 3000 classes of positions in the State service, and 10,000 dif- ferently named positions in municipal services, whichprobably repre- sent only several hundred really different classes. Our main con- cern has been how not only to meet the objectives ofa public system, which is open to review and challenge bymany individuals and inter- ested groups, but how simply to keep ahead of sucha tremendous work program.

Historically, the American Civil Servicewas based on a so-called competitive system, in which itwas assumed that public jobs were in great demand and thatmany people were competing for the choice plums.Under such a set of conditions, very elaborate evaluative techniques might be appropriate, butas Superintendent Brown from Los Angeles pointed out in his comment in theprevious session, one of the facts of life is thatwe are in an entirely dif- ferent kind of labor market.We have great scarcity, and our prob- lem really isn't the refinement ofour evaluation techniques, which could be much more refined ifwe had more resources and more time. Under present conditions the competition is notamong candidates for jobs but among employers for quality personnel.

There is an element in American Civil Service which differs a little tenni foreign Civil Service systems and from industrialsys- tems in that the cbntral evaluationagency does not complete the selection process.It nao the .;ask of comingup with a panel or a list of qualified candidates ina one-dimensional sequence, supposed- ly binding and relevant to fillingany indivj.dual position in that general position class. One safeguard we have, and it is really not so much a matter of any recognized inadequacy of our testingprogram, but a matter of constitutional law. In most Civil Service systems in the United States, under the American theory ofchecks and balances, the Civil Service system is regardedas the creature of the legisla- ture whereas the appointment process is regardedas the province of the executive branch. Attempts, in Civil Service practices and rules, to limit selection to the single highest ranking eligible

1143 of have run against thisConstitutional doctrine, the separation powers, so thatmost Civil Service Agencies nowallow an appointing officer, a representative of theexecutive branch, a choice from a limited number of the eligiblesranked highest for a particular class of jobs. Many jurisdictions have arule of three. This rule of three is under attack, onthe one side, by one set of pressure groups, and onthe other by other pressure groups. I'm reminded of Luther Gulick's aphorism"Opposing pressures keep public officials upright." Our employee groups and ourminority groups, generally, would prefer a much morerigid system, whereby there was nochoice by appointing officers.Appointing officers often feelthat they should be allowed even morelatitude in selection from aneligible list according to their appraisalof the candidates in relationto their needs.

Mr. Berger has opened upthis subject, so I guessits fair to pursue it, that thepromotion racket in the NewYork City the Sanitation Department'sappointments did not reflect at all on evaluating agency, but on theappointing officer's choice among groups of three. Minority groups and reform groupsoften have taken the position that this choiceof three, or proposedextension to five, does give an appointingofficer a chance to play variouskinds of favoritism with the classtie, or with the politicalaffiliation, or in this case,something much more crass.

However, we can discuss theclimate, and the conditions and the problems of evaluatingpeople for the public service gen- erally and still miss the markcompletely with respect tothe teach- study made by ing profession. I'm reminded of "10,000 Careers," a Dr. Flanagan's colleague, Dr.Robert Thorndike, in which 10,000 people who'd been testedin the Air Force SelectionProgram were followed up in civilian life. Surprisingly, those who ended up as teachers were characterizedby low verbal and quantitative scores in validating on aptitudetests. If we follow the reasoning we use interest inventories to find morepeople like those who arealready in a profession, we mightselect for teacher training,candidates with low scores on verbaland quantitative aptitudetests. But should we be bound by the kindsof people we now have inthe teach- of people who are ing professions? Perhaps we have a real shortage interested in the conditionsof employment that applyto teachers. I would surmise that NewYork City has more problemswith the black- board jungle image, whetherit's a fact or not, as adeterrent to teacher selection than withpossible inadequacies in theselection jobs can be as secure as instruments. Until New York City teaching the suburban schools are,it's a losing race asSuperintendent Brown has pointed out to have moretime-consuming and more elaborateselec- We tion procedures. What we must do is a bitof social planning. at- must assess teaching potential,make the teaching profession more tractive to young people, andguide people with thepotential for

11414 When we have donethese teaching success intothis profession. further refinements inthe evalu- things, there willbe more point to ation process. I want to makeit To avoidmisunderstanding, however, selection methods have clear that I don't meanto say that careful is just as place only in asurplus labor market;I think the reverse qualified people, fully When we have ashortage of fully true. of qualified in the senseof meeting all ourdesired requirements determine aptitude education and experience, westill have a need to likely prospects intotraining for on-the-jobtraining and to direct Mr. Berger talksabout, sidestep- programs,possibly the type that certification requirements. When we in the pub- ping or postponing worth hiring lic service got awayfrom the idea thatthe only person another employer inthe same type was somebodywho's been let go by numbers of candidatesbut of work, we not onlygreatly increased the materially improvedthe quality. education courses is My personal experiencewith required in college one wasin limited and unfavorable. Of 2 C's I received in the EducationDepartment. an educationalpsychology course taught obsolete andconflicted The psychologicalcontent of this course was psychology courses I wastak- with more moderninformation from the What were the right answersin ing in the PsychologyDepartment. Department were notrecognized in theEducation De- the Psychology did poorly I used the PsychologyDepartment answers and partment. The other C in my examination in theEducation Course. on the final The course college work was based ontwo tests I can neverforget. and was taught byEdwin Ray was in thepsychology of philosophy and later Presidentof the Guthrie, a leader inpsychological theory The course coveredthe psycho- American PsychologicalAssociation. philosophers from Platoand Aris- logical content ofthe classical The first part ofthe test totle up throughDescartes and Leibnitz. by author of 17 passagesand since we had required identification to at- studied the works of17 philosophers,the students tended The correct answer was tribute each passage to adifferent author. examination was Plato 17 times. The other part ofthe final course true/false test in which we wereto leave alone thestatements a those that were that were true andto cross out theitem numbers of statements false butthe false. Most of us markedabout half the key was all statementstrue. is Mr. RaymondJacob- Samuel Streicher: Our third speaker Director, Bureau ofRecruiting and Examining,United son, Deputy Formerly, he was a States Civil ServiceCommission, in Washington. Commission, member of the Divisionof Standards ofthe Civil Service in Washington.Mr. Jacobson. like to use thelimited Raymond Jacobson:I think I would

145 discussion to get under way, time available inorder to permit the just to touch on three orfour areas, not toreally summarize my One thing I would liketo whole paper again,which you will get. Service, in attempt- point to, in connectionwith the Federal Civil jurisdictions (be- ing to distinguish itfrom other Civil Service although there are a great cause I thinkthere are some differences, similarities) is that the FederalCivil Service Law has one many general piece of tremendous advantage in thatit is a very broad and of this leg- legislation. In fact, we'reconvinced that the authors though I've islation, back in1883, were extremely wise men, even his stay in of Mr. Pendleton'sdoing anything else in never heard very Congress. In this area, theywrote a law which was really a basically this is the broad statement of generalpublic policy and legislation we still areable to live with. It has made things easier in many ways, Ithink, than in somestate and local systems, where they have beenharnessed with much morerigid legislation. which is basically We do have a broadstatement of public policy, to see to it that that the purpose of theCivil Service System is the details and the pro- jobs are filled onthe basis of merit, that determined by rules cedures by which thiswill be accomplished are by the Civil Service issued by the Presidentand regulations issued flexibility over the years; soour Commission. This has given us think, over the last50 system has changed,rather remarkably, I Our own con- years, even moreremarkably, perhaps, inthe last 20. objective have cepts of what was neededto accomplish this basic been sharpened andchanged.

We also have anotheradvantage, relatingparticularly to Civil Service System to Tom's comments aboutthe relationship of the Government, the legislative andexecutive branches. In the Federal. creature of the Presidency. the Civil ServiceSystem is clearly a years ago, This was not quite soclear when Tom was with us, some of, attitudes that had grown not because oflegislation, but because dis- up in termsof our relationshipswith the Congress and the In the interest of the President, overthe years, with thesystem. Now, last 20 years, therehas been an enormousshift in attitude. System, in all we clearlyhave a situation wherethe Civil Service examining and of its manifestations(not only in its recruiting and selection program) is an armof the Chief Executiveand functions Congress. as an armof the Chief Executive,not as an arm of the That doesn't mean we'renot subject to Congressionalappropriation but no more than any and Congressionalcontrol; of course, we are, other executive branchagents.

Let me turn fromthese differences to abrief characteriza- in which we go tion of a couple ofprincipal points about the way selection. about our business asit relates to theproblem of teacher obViously are conducting We obviously have a verylarge system and we examinations in the same waythat the New York Stateand New York

1146 of positions.We are the largest City systems arefor wide varieties different sub-systems, single employer inthe country.We have many think the parts ofthe system really, within ourtotal system, but I we're talking about atthis that are most relevantto the subject parts of the systemwhich relate to therecruit- Conference are the of and selection processesthat go to the filling ment, examination, and technical level jobs inprofessional administrative our entry typists and the categories. So let me dismissthe stenographers, the plumbers andthe welders, postal clerks, thesheet metal workers, selection which isdirected primarily and so on. Let's talk about themselves throughhigher educa- at those peoplewho have prepared of going aboutthis job. We have basicallytwo different ways tion. it's obviously far morethan I don't want toover-simplify, because categorize our methods astwo. We two, but broadlyspeaking, we can 20,000 to 25,000 people bring into the FederalCivil Service about and master's every year atthe college entrylevel, the bachelor's bring them in againto a wide varietyof jobs, in- degree level. We employment cluding, by the way, someteachers. This is not a large but we have in ourBureau of category in theFederal Civil Service, of classroom teachers Indian Affairs a verysignificant population secondary level. So we have some at the elementaryand some at the that in just alittle bit. experience in this area,and I'll touch on basically we use twodiffer- But in this whole rangeof categories, For about half ofthese jobs, we use anexamination ent techniques. Out of this we known as The FederalService EntranceExamination. and 13,000 people a year.We test between select between 10,000 in 150,000 people each academic year,and this results 100,000 and it's a pro- this number ofselections. This is a massive program, abilities testing, muchalong the 'lines gram thatinvolves general it involves anevaluation of that have beentalked about here and education and experience. in our law that is By the way, wehave one restriction local jurisdictions troublesome, I should say,that most State and is we may notrequire education, and industry donot have, and that except for professionaland scientific as aminimum requirement, administrative typesof positions, wehave positions. So, for many not have collegedegrees even to allow peopleto compete who do in which we wouldlike to en- though that mightbe the desirable way professional jobs, such asteachers, engi- ter people. But for our and so on, we dohave positive educa- neers,physicists, economists, We provide formost occupationswhich we have tion requirements. battery we identified as not inshort supply andfor which the test Exam as the method appropriate, the FederalService Entrance use is their We give everybodythis same test. We evaluate to come in. to what kinds of prep- backgrounds and codethem in great detail as selectively use thislist of eli- aration they have,and then we the Agencies. For example, gibles, depending onthe job needs of abilities. We usethe verbalabilities some jobsrequire high verbal test score for selective use of the list. If there's a need for someone with specifictraining, we'd like to bring someone into an agency where the job demandsrequire some business administration or accounting background, not necessarily professional accounting. We can look for people with, say,12 hours in accounting, as an indica- tion of interest. Agencies recruit heavily for a particular job. They identify people in whom they're interested in thecolleges and we have ways bywhich, in a perfectly legitimate use of merit sys- tem procedures, we generally can find ourselves ableto reach the people that are really in demand because they will perform welland they have a record that is appropriate for the job.

Now, in a large number of other categories, and this in- cludes primarily the engineering and scientific categories wehave found, unlike A.T. and T., that we haven't been able to get away with written testing. Here we have followed a procedure which is working extremely. well. It's not getting us all we need. We have not yet found anybody who gets all they need of top qualityphysi- cists, for example, or mathematicians, or engineers, butwe've done pretty well with this. It's a device by which our recruiter, who is an agency man, whois recruiting for his program, can recruit in the colleges where he can find the kind of talent he's looking for, and as a licensed rater,who can act for the Civil Service Board of Ex- aminers, and rate the person on the spot, in terms of hisfaculty evaluations, his interview, and his college record, and can give this person a rating and offer him anappointment, if he's authorized by his agency to do so. Frequently, the delay here is more of a secur- ity delay, because so much of our needs are in the DefenseDepartment and they decide they've got to check people out prettythoroughly be- fore they make an offer of an appointment, so we have adelay there, but this is not a delay through the recruiting and examiningsystem. This system works well in the shortage areas. It involves considera- tion of the background of the individual, consideringeverything they have done and everything we can find out from theircollege records. In most of these categories, we do have aneducational requirement, so we're dealing with onlycollege seniors, or graduate students.

Relating this to teachers; we have a somewhat similar sys- tem in operation for the Bureau of Indian Affairs inits teacher re- cruitment program, in that we do have a device bywhich we give the teacher recruiters in the Bureau of Indian Affairsauthority to make appointments. They work through our Inter-Agency Board in Albuquer- que and they operatethroughout the country in the recruitment proc- ess. They have a tough job because just as you have atough job filling jobs in East Harlem, we have a tough jobfilling jobs in the Hopi Reservation in New Mexico and Arizona.Let me tell you it is not easy out there and in fact many of the problems are verysimilar, in that there's a cultural difference. Some problems are worse, in that there's isolation that is extremely serious andextremely tough

148 schools; teachers, who willteach in small to live with,for these 70 miles away teachers and a verydifficult situation, three or four snakes, 70 miles infested onlywith jackrabbits and on abumpy road filling station,and small town,which has the awayfrom the nearest tough job these places. So we have a I'm notkidding, I've seen while we haven't recruit on thecollege campus, selling. But when we more than youhave, well as we'dlike to here, any been doing as We think this yearweregoing to we've been doingreasonably well. Ba- think we'veimproved ourmethods. do a littlebetter because we of deal of weightto the evaluations sically, we aregiving a great of these people.We administer no the colleges,in the evaluation Examinations as partof our We're using theNational Teacher tests. people on thespot. Being evaluation and we areevaluating these sort of take careof the paperwork able to makejob offers, we because paper workhas This is aninteresting process, afterwards. for a long time.We have, in killed us inthe CivilService system that reconcile ourselvesto the fact shortage categories,learned tc matter of gettingthe recordsstraight paper workreally is just a whom we possibly interestin the job, later becauseeverybody we can to have a job if we canpossibly get them would liketo have, can too muchattention report for duty. So we don't pay accept it and clean it up as wego along,and we to the paperwork, as I say, we the primary in the way ofwhat we consider don't allowthat to get I hope thatthis procedure, or job, gettingthese jobsfilled. useful in discussed aroundhere will be other ideasthat have been and tomorrow. Thank you. further discussionsduring today Discussion for discussionin the Samuel Streicher:The floor is open form of questionsorcomments. question whetherit Mr. Bergerraised the Ralph Walter: into con- specialized training,to take this is necessaryto have preparation, be- real problemwith college sideration. Now this is a students for pressurefor us to prepare causethere's increasing for thedisadvantaged, like urbanschools, schools specialized fields of view ofselec- disturbed, and so on. From the point emotionally in this preferto would you peoplewho are involved tion of teachers, possible, whowould with as broad ageneral training as have someone when they are onthe job, orwould you receive aspecialized training of specializedtraining first? prefer them tohave some degree Would you want Anyone careto comment? Samuel Streicher: special- with regard towhat's meant by to elaborate onthe question period? ized trainingin the pre-service this is fairly commonin many of Ralph Walter: I presume in urban schools for trainingteachers to teach yourcolleges, this pressure

149 and some schools havedeveloped a rather elaborate program,where they get special training inthis field. Is this essential or is it better to have a teacher whois generally qualified andthen gets some specialexperience and in-service trainingfor urban schools.

Samuel Streicher:Any of the people engaged in programs of teacher training care tocomment? Please.

George Redfern: I would like to try to answerthe qves- special- tion. As a former personneldirector, I would have welcomed ized training for theinner-city because the marketresearch showed 60 per cent or more of the vacancies tobe filled were in that area, and I must say that mostschool systems have struggledto have their in-service programs adequate tore-train. There's another very prac- tical problem, that in thehighly competitive market, it'sacademic sometimes to say to do it inthe in-service method. If you don't, you can't hirethem from your competitorsanyhow. So, therefore, the elusiveness of many ofthese candidates and thecompetitive mar- ket make it an academicquestion as to whether you canhave a real, honest-to-goodness in-service program, soto answer your question, I wish we could have somespecialized training, as much aspossible. It would facilitate - andI recognize your problem.

Harry Gilbert: I'm Harry Gilbert, PennsylvaniaState Uni- versity, Brooklyn, New York. I'd like to comment on a coupleof seductive points that were madeearlier. We're never going to get despite all of our announcedin- away from somevalue judgments, terest in hard research. But two points were madethat I think are going to call for some valueconsiderations: one was Mr.Berger's that we should not make testspublic in the Board ofExaminers, for example, which means you can re-usethem and refine them, getting better and better tests. The second was that weshould hire any col- lege graduate and then trainhim to become a teacher onthe job. Now both of these sound veryattractive on the surface,but I want to comment on some of their hiddenmeanings.

In the first place, theBoard of Examiners I knowis unani- you're hiring in the pub- mous on this,takes the position that when lic service, you have aresponsibility to tell anybodywho is re- jected exactly why he is rejected. If you are going to keep your tests secret and refine them,etc., a person who is denied ajob, cannot look at that test, knowexactly which items he hasfailed, and file an appeal, ifhe thinks he has been improperlyselected. We take the valueconsideration that in the publicservice any ap- plicant has this right of seeingwhy he's rejected and a rightof great deal harder, not just a appeal. So I think it makes the job a little harder, but I think it's a necessarypart of examining in the public service.

150 Secondly, this business of hiring anycollege graduate and then making him a teacher comes tothe heart of what I think is a real derogation of the teacher in oursociety. I work now for a real hard-boiled Dean in the College ofEducation at Penn State, Abraham Vander Meer. He used a figure of speech I'd liketo present to you. He says, when I hire an architectto build my house, I'd like to have someone who isconversant with Camus, who knows thecontributions that a Matisse has made to ouraesthetic taste, knows all about some of the avant garde musicians, etc.,but first of all he wants an ar- chitect who knows thermodynamics, andmakes sure he puts into his house the properties that will makeit resist the weather properly. Then he can have all the othercharacteristics. The same thing ap- plies to a teacher; if you think ateacher has to be one who knows children, knows how to present thesubject matter which he knows thoroughly, this calls for a prettyhighly developed integrated series of preparation. When you say let's take anybody, sincewe're in a shortage, you're downgradingthe need for this total prepara- tion, and in fact, you may be doingyourself an injustice. I think Doug Bray made that point. Don't sell yourself too short. In the long run, if you stick to the standardsthat you require, that in itself makes it attractive. So I'm going to keep yelling, even though one side of me is very muchdevoted to research, to the need, too, that we don't discard plainhard thinking and coming up with some values that wethink are indispensable.

Raymond Jacobson:On this issue of the publicityof tests, I just want to indicate that I agreewith you that in a public sys- tem any applicant has the rightto reconsideration and appeal, has the right to know why they areineligible, if they are, or why they got a 72 instead of an83. We in the Federal Civil Service,and I think this is true of most CivilService jurisdictions throughout the country, state and local, do notfeel that in order to meet that ob- ligation, it is necessary to publish ourtests and to throw them into the kind of controversy that Ithink we get into that way.We've had no trouble with theCourts. We get an appellant who wantsto find out how he did in the examination,the whole test rolled out infront of him.We tell them in which way they wereweak, in which way they failed, we don't go over it questionby question, and I think this public obligation can be met withoutregard to this kind of unneces- sary openness.

Bernard Berger: Dr. Gilbert knows thatup to July of this year we publishedmost of our examinations. In fact, the Board of Examiners I think adopted our policyof publishing examinations some- time ago, but we have come to therealization that we're not doing ourselves a service as a selection agencyin constantly pouring out our energies andwriting new equivalent examinationswhich are not equivalent. We probably have as elaborate anappeals procedure as any jurisdiction has. It is quite likely that we havebeen involved

151 in more litigation than most other jurisdictions, so we are particu- larly cautious in our test and appeals procedure to avoid -1.itigation as much as possible. We do provide, for example, every candidate with a blank numbered sheet where the candidate, on a multiple choice kind of examination, can list his answer as he goes along, and take that home with him.We publish the key answers, the tentative key answers, so that almost immediately a candidate can tell whether he has passed or not; secondly, we allow candidates to see the questions for which they were -marked wrong -- only those -- and the candidate has an opportunity then and there to appeal or to come back again with some reference material to support his answer.We also know that in our test-wise population in New York City, candidates, if they feel that there is a true grievance that should be rectified, and if we have opposed their contentions, they go to the Courts. There are many lawyers in New York City who are willing to take these cases on some contingency basis. In this way the problem of appeal is pretty well handled. This is true, even more so, in the State Civil Service Department where a candidate can on occasion take an expert along with him to look at the question. So the appeal proce- dure is still there.

Thomas Bransford:May I speak to this -- I hate to leave to our most recent convert the job of proselytizing the Board of Ex- aminers on extending the useful life of good tests.

We have been able for years to play one-upmanship with our colleagues in the New York City Personnel Department by saying that if the City Civil Service hit upon any good questions, they would im- mediately throw them away.We can't use that jibe any more. The State's examining program involves many candidates from New York City as well as upstate.We do have a system which, in some ways, is more rigid, more secure than the New York City system.It's true that we do allow consultants to accompany candidates in reviewing some of our tests. Some of our tests have been exempted from any review. A clerical review of answer sheets and key answers without access to questions is generally allowed. In some examinations we allow can- didates who are willing to travel and make special arrangements to appear at special centers the week after a test has been held to re- view the tests and tentative answers under secure conditions to point out what they believe are errors. Some of our candidates are urging us to follow the New York City practice because they are not sure what answers they gave, and don't know whether or not to object to the tentative key answer.We feel that one of the virtues of our system is that objections are not merely self-serving. Another vir- tue is that the exposure of the test questions is more limited in time and in place. In examinations where we allow review of both the test questions and the scored answer sheets, some candidates com- plain because we do not allow them to take tests home for extended research. I would like to join Mr. Berger in urging those whom Dr.

152 stampeded by Board of Examinersnot to be Gilbert leftbehind at the from constructivechanges in the Civil Serviceemployee press away and reusingtheir best test policy and procedurefor safeguarding questions. I meant toattack you, Harry Gilbert: Tom, I apologize; too. can't have it It just seemsto me that you Gertrude Unsert which means Either you keep yourtests completely secure, both ways. the kind ofprocedure which that nobody may seethem, that is, use administering scholarshipexaminations, the State Departmentuses in is put into anenvelope, anc whereby each bookletis numbered and the test may not seethem or you allow even thepeople administering As soon as youallow any formof ap- some formof appealsprocedure. security has beenviolated. One peals procedure,to that extent, and the other oneat Item 50 and so on. person maylook at Item 10 I don't think youcan straddlethis People can putthese together. You can deceiveyourself that issue and pursue amiddle course. you're really not. you're keepingthem secure, but I'd like to tie someloose ends together Jules Kolodny: of the speak- reactions. For example, one and then throwit out for there ought to be a who it was,mentioned that ers, Idon't remember I think it was and then asubsequent person, broad recruitment area, they give in talked about theteaching examinations Mr. Jacobson, The NationalTeacher connection with theBureau of IndianAffairs. for a broaderrecruitment area. Examinations wouldmeet the demand evaluation speaker, for example,spoke about the A third that perhaps we prior to jobplacement. I would urge of training Traditionally, the kind ofreouirements we have. ought to re-examine American college held in lowestesteem on the the courses that are require. By and the ones thatexaminers in teaching campuses are highest esteem. the courses that arenot held in the large, these are re-examine and talk wonder whether wecouldn't really Now, I would required in termsof kind of coursesthat might be in terms of the be things that are and let someof the "how" courses urban sociology while on the job. I college but arelearned in service not taken in professions in termsof how to know that solittle is donein other but a body ofreal knowledgein subject matter handle situations, Law schoolswill devote a must be learnedin the law, forexample. schools of educationtend to devote minimal amountof time, whereas know the "how to do," butpeople may not a maximumamount of time on supposed to teach. subject matterwhich they're and a re-evaluation If you have abroader recruitment area require, you mightconsider givingthe of the kind oftraining you

153-a National Teacher Examinations asa Civil Service hurdle. You would have to consider what Dr. Streicher alludedto, namely, that the job must be made very attractive.We must make the position such that we want people to move into jobs which will have holdingpower. That's a matter which the superintendentsand Boards of Education have to concern themselves with rather thanselection officers.Once that is done, you may be able to havea very substantially high cut- off point on the National Teachers Examinations. Our particular com- munity may take people whoscore in the top docile, and that's our cut-off point because the conditions inour school system warrant our being this restrictive. On top of that, you must tie it in with the type of thing Dr. Rivlin indicated andsome other speakers have al- luded to, namely, there should bean on-the-job training program. I'd like to think of itas an internship program. The medical pro- fession, for example, talks ofan internship program; we, on the other hand, go through whatwe think is a very careful selection process, and then we throw the beginning teachers intoa full pro- gram, close the door, and he's all alone witha group of youngsters, never having done anything like this before. I should imagine that if we tie some of these loose ends togetherand make a complete ',gestalt'? in a broader recruitmentarea, using the National Teachers Examinations, with very substantially high cut-off points,and have a type of internship program where the teacher teachesa shorter day and has enough time to observe his colleagues andto talk to his col- leagues and then is evaluated at the end ofa given period of time while he's still a probationer, you'remore likely, rather than less likely, to get the quality personnel you're lookingfor.

Harold Mitzel: There's still one loose end in Mr. Kolodny's remarks, and that is the modern teacher educationcurriculum. It is not true that the teacher education curriculum isloaded with methods courses, although that may have been true someyears ago, but I'd in- vite him to take a look ata teacher education program and he would find a minimum of courses which he describesas ',how ton courses.

Milton Gold: I think we have to distinguish here between things that we are looking for. In terms of preparing teachers for the inner-city, we discovered at Hunter that the mostmeaningful things that we do are to provide students with first-handexperience in the situations, so that they're not afraid of it, thatthey can see if they can adapt to it, and that they can cope with the situa- tion that they have to face.

We also know that, in looking for the prospective teacher, we're looking for a person who hasa commitment to teaching and, in the case of New York City, who hasa commitment to work in what may not be the most pleasant teaching situation, in terms ofabsence of pressure, tensions and so on. These things are quite ata variance with the kinds of things that we normally test for, andit isn't a

l53-b test security, question of whetherthere's any test security or no the question is how do youfind this commitment,how do you provide Are we asking theexaminers, a programof experience andtraining? kind of examining and to now, to stepout of the role of a narrow partake in the problemsof pre-service andin-service education?

In line with whatMr. Kolodny mentioned afew moments ago, with one of theinteresting things we aredoing is in cooperation in Central the Board of Education,working with some 25 teachers that they Bronx schools, wherethey are given a reduced program, so other and to work withpeople may have sometime to confer with each who are providing somekind of supervisory oracademic assistance. On the one handthere are the cognitivethings that Mr. Kolodny has it; on the other mentioned, and nobodydisparages the importance of system often doesn't hand, the importanceof survival in the school top decile of a test have much to do withwhether a person is in the that's cognitive in nature. something and to Raymond Jacobson:I just want to clarify just add a point which mayhelp with this area. The clarification recruitment of teachers is that I didn't wantto imply that in our require Nation- in the Indian schoolsthat we rely exclusively' on or is this (and it al Teacher Examinations.What we have done here indus- seems to me someof the comments we hadthis morning from our that this is the try people lead inthis direction, or indicate direction that might bevaluable for the profession as awhole), we're looking for in a what we've done isthat we've identified what in a series of teacher in an Indianschool. We've identified this front of me, but there are some- job elements. I don't have them in thing like six or seven; oneof them, of course, isknowledge of apply that methodol- educational methodology;another is ability to ogy, and so on.

We have used a varietyof alternative measuringdevices to example, a satisfy, to consider thesedifferent requirements; for will have some value in say- National TeacherExamination Test score and we ing what somebody knowsabout whatever that test measures, don't want to throw it away, wewant to accept it ifthe person has demonstrate this taken it, but we don'trequire it, because they can by completion of knowledge in another way.They can demonstrate it recommendations in work with high classstanding and high teacher is set up a series of education programs. So that what we've done th9 job, that are needed the important elements,the key elements in for job success, andwe've had a whole varietyof measures against in this variety these. Now, it so happensthat we haven't included We will accept of measures a writtentest that we administer. the graduate others. For example, we'reinterested in performance on record exam, which they may ormay not havetaken. If they've taken the total record of the it, we want to knowabout it, it's part of

154 sched- individual. We provide,in what we call anunassembled rating education, for a whole ule, that is evaluationof experience and series of alternatives. who are them- The evaluations aremade by the panel members teachers, principals,people in the education pro- selves supervisory agreed on this gram of theBureau of IndianAffairs, and they've all We've had an extensivetraining program withthem, rating procedure. Civil Serv- understand each other onthis. They operate as the so we employees, ice Agent, eventhough they are notCivil Service Commission what they do is usethis whole and they conductthe evaluation, and numerical score for battery of information,to decide on an actual each applicant. this caution. We Murray Rockowitz: I'd just like to make if there is a teacherthat talk in terms ofteacher selection, as a four or five we're after. I'd like to cut acrossthe comments I think that's people this morning. It is a problem ofcriterion. Dr. Gold alludedto basic, but the questionis - criterion forwhat? inner-city, and I thinkthat programs, teachertraining for the kind of base which rep- what's necessary for usto determine is some Then after resents for Mr. Jacobsonentry into the publicservice. try to wed the par- you've made thedetermination of this base, you task involved.Now, ticular qualificationsof the entrant to the criteria for the various somehow, research has toset up a number of I think that we kinds of demands made onteachers in various areas. the qualities in the would agree, for example.almost a priori, that teacher necessary to teach abright child aredifferent from the best in the under- qualities that are necessaryto bring out the either privileged youngster whohasn't been able tobe identified as do bright or not brightbecause we don't havethe instruments to to all our considera- this. I think that thiscaution ought to apply of the research tions here, and itshould probably influence some tomorrow morning perhaps. lines of attack thatwill be delineated to the National That's one of the reasonswhy I take exception because, unless they can serveto determine the Teacher Examinations the varia- basic minimum fromwhich we start toexamine to meet all the schools in theurban tions on the themethat are demanded by put the imprimatur on an centers in the worldtoday, we're going to work out in many de- individual and then findthat he just doesn't manding situationsin the urban centersof our time. Art Benson, whom Ihave Jay Greene: I want to say for National heard talk on thissubject very much,that he regards followed by other Teacher Examinations as ascreening device to be to make clear thatwhat we types of screening,but I should also want other places is actually have been talking aboutin New York City and selection, where we sayat screening device. It is not test a simple is a the end of this procedure, wenow haveselected somebody who good teacher. City we are lessprone to say Strangely enough,in New York methods. The informal somebody who usesinformal selection that than ten-minute interviewwith into a collegeand have a selector may go decision mentally,if at his record,and make a anapplicant, look who is sympatheticto chil- that he haspicked somebody not verbally, to be a successfulmember how to teach,who is going dren, who knows staff.. We don'tbe- and a successfulmember of the of the community, sort of situation. lieve that you cango thatfar in that have a back- We believethat a teacherof English should English, and so wehave anot-too-difficult ground in thesubject of similar to NationalTeacher Examinations', screening instrument, these would-beteach- national reputationto see whether which has a Strangely enough, manyof minimum knowledgeof English. ers have a that, on a basisof about 1 to In thesetests, you find them don't. after completingcollege as 100 per cent, someget 20, 15, 30, Therefore, this is anecessarypreliminary screening. English majors. of teach- knowledge offundamental methods Secondly, heshould have a classroom andproceed toteach. ing, enough towalk into a interview, we don'taim interview.In our We also have an teacher, is a warm,capable, wonderful to judgewhether the person well withchildren, orwhether he's whether he's goingto get along No, it's asimple screeningdevice going to motivatethem properly. to com- that a teachershould be able based on thismajor premise, all that is orally. That's about municate withreasonable clarity And I think,using valuejudgments being measuredin the interview. about and notresearch, it isgenerally that HarryGilbert talked often is the vehicle forteaching very agreed in theprofession that is going to the teacher'sspeech. If the teacher ateacher's voice, There, again, wehave the teach in amuddled way thatis unclear. thoughtsprofessionally, with minimum screeninglevel, to convey personality isconcerned, only reasonable clarity,and so far as show up in aninterview may be those fundamentalthings that may What we'retalking about now, commented uponin the interview. de- been urging, aminimal screening really, is what someof you have feel is essentialfor a per- with valuejudgments, we can vice which, After this, letthe administra- son whois to step into aclassroom. have been do- period. That's all we tors take overin a probationary ing, and that'sall we'retalking about. four more minutes. Any Samuel Streicher:We have about further comments orquestions? briefly One of thethings that was very Bernard Berger: I deliberatelyavoided mentioned by Mr.Kolodny wasfholding power." think it'ssomething that in mypresentation, but I mentioning this discussing selectionmethods be- merits considerationby any group determination ofappropriateness of cause noevaluation and no

1.56 selection methods and recruitment of teachers is worth anythingun- less you can hold onto or retain the teachers. This, of course, opens up a host of problems which go beyond an attractive salary. The provision of in-service training, the necessity for cooperation among the supervisors of teachers, in their dealing with a new teach- er, the entire work situation, the promise that the job holds in terms of satisfaction for the individual teacher, and so on, and very relevant problems. I don't think that you can possibly do any kind of evaluation, unless you speak in relation to turnover in the school and to retention of teachers.

And by the way, in our giving the more general kind of en- trance examination for professional positions in New York City, im- plicit is a minimum of a one-year trainee program or an internship, where most of the time is devoted to formal training and a series of rotation of assignments, in order to first, allow the individual to determine if this is really his place; and second, to determine whether we want to keep that person.All of these positions carry a one-year term, prior to a permanent appointment, which also includes a probationary period.

Fred Williams:I have one question I would like to ask Dr. Greene in the light of what he said. I'd like you to recall, Dr. Greene, the Chairman announced approximately 3-1/2 minutes ago that we only had four minutes, so you have 20 seconds to answer.What do you see is the particular advantage of our testing procedures at the entrance level?Over the National Testing Examinations, if at all?

Jay Greene: Just three or four.Art says his is a screen- ing. It doesn't include any oral phases but we do have an oral. In some areas, as health education, art, and so on, we have a performance test as well, and then what was mentioned before. We have felt obli- gated to release our questions to people, so that they might learn where their errors were, and improve; whereas the National Teacher Examinations have security measures which prevent the releasing of the tests. These are the major reasons.

Samuel Streicher:May the Chair be permitted a brief per- oration? We have placed on the table the many considerations that should be brought to bear on the question of the selection of educa- tional personnel.We've barely mentioned all of the important con- siderations. The knotty problems that grow out of these matters will be discussed later, I hope, in greater depth.Those who planned this program were really wiser than they knew. They brought on an inter- disciplinary interchange from business right through education. If I had the time, I would have told you of some of the experiences of mine in East Africa, where Standard Oil of New Jersey had a representa- tive who did recruiting. He said to me, "What can I do when the President of the Republic says, 'You must employ 25,000 of my citizens

157 in your various gasoline station, orelse" ?" It was just a condi- tion, don't you know, in the climateof employment that may bowl over all otherconsiderations.

We've run the gamut from the initialinformal selection in the teacher training institutionright through the last stage of dub- bing a person as permanentlyappointed and entitled to various tenure rights. Shelley said, HWe look before andaft, and pine for what is not." Later on, we'll try to satisfy someof the pining by asking for what should be better selection processes,and what else we must do to find out about them.

Paul Denn: Thank you, ladies and gentlemen. The meeting is adjourned.

158 Procedures for Selection Session No. 4:Desirable Policies and of Teachers

Chairman:Arthur Klein, Board ofExaminers

William B. Brown, LosAngeles City Schools George Redfern,American Associationof School Administrators Albert Schiff, DetroitPublic Schools

this morning and wehave Paul Denn:We had a full session is one which another one thisafternoon. This afternoon's session and procedures in the will focus directly onthe problem of policies of them selection of teachersthemselves. We have two panels, one the dealing with thepolicies and proceduresfor teacher selection, such procedures. other with criteria andthe problems of validating colleague, Arthur Klein. The first ofthese ,will be chaired by my point of view of most Arthur Klein: I think I express the that thus far, our of the people whom Ihave spoken to when I say There conference has been a verystimulating and provocative one. answered by my number of questions thatI should like to see are a else who own boardduring the next fewmonths, and I'm sure everyone selection will takehome is engaged in personnelwork or in teacher deal with locally. with him some questionsthat he will want to desirable proce- In discussingteacher selection and the fortunate today in having dures for selectingteachers, we are very with these practices three men who have beenintimately associated Associate SuperintendentWil- and proceduresin three large cities. in charge of personnelin liam Brown of LosAngeles, George Redfern, Secretary of the Cincinnati for many yearsand now the Associate of School Administrators,and Albert Schiff,who American Association I know that our is Director ofPersonnel in theschools of Detroit. Superintendent, Theodore Lang,who is with us now,will be own Deputy these gentlemen very muchinterested in hearing ofthe ways in which city in the country. have met the problemsthat beset every large We'll begin withAssociate SuperintendentBrown. and thanks to our William Brown:Thank you, Mr. Chairman, from Los Angeles.We're glad New York hostsfor including two of us fresh air for a change, to be here in theCatskills, have some real morning's and attend this veryworthwhile conference. I felt this and comments gave us discussion, the presentations,and the questions I hope earnest- very basicguidelines for the selectionof teachers. will be sent to us. It is ly that the fullreport on this conference development of ideasfor the preparation a landmarkin the study and Angeles can getfederal financing of teachers. I hope some day Los about that. I think it for a similar conference. I'm very serious would do us good in theWest.

159 kinds of teaching assignments in ourschool systems. One or two ex- amples have given me greatencouragement. The University of Califor- nia at Los Angeles now requires everystudent teacher to have half or all of hisstudent teaching in a poverty area in ourschool sys- tem. This has been one of the mostheartening things that has hap- pened to our system in a longtime. It's really helping our recruit- ment. I cite that merely as oneexample of the things that are tak- ing place increasingly betweenthe Los Angeles City Schools andthe twelve surrounding colleges anduniversities which are our primary, and one of our best, sourcesof new teachers.

I've put on the wall a chart ofthe five phases of the re- cruitment and selection of careerteachers. I hope these five phases reflect some desirable policies andprocedures.

The first phase is the recruitment program. There ;ire the regular programs: 1) California campus recruitment--Ourstaff visits approximately 35 colleges anduniversities in the state, two, three, or four times a year,depending upon need. 2) Out-of-state recruit- ment now includes visitations to over200 colleges and universities. The special recruitment programs areused increasingly in terms of shortages and inner-city problems. One program involves internships. This is a program whereby a college oruniversity, working closely with a district, invites matureadults (housewives, members ofother professions, and the like) to returnto college but not to the cam- pus. These are baccalaureate degreegraduates who have had no teacher training. They are invited to prepare to beteachers by focusing teacher training, not onlystudent teaching but courses in professional education, in a school in adisadvantaged area.We have 180 enrolled in one such program.

Talent search is another program. This involves direct re- cruitment of baccalaureate graduates,which was necessary last summer to meet the elementary shortage. Operation Return is still another. There are always teachers in thecommunity who can be drawn back into the profession.

The newest recruitment programthat we've added, and if we don't add one every few months I'msurprised, is recruitment in Europe.We have found that there areapproximately 700 teachers re- turning each year from the armedservices dependents schools. Many of these have separated from theschool systems in which they taught before they went to Europe. Through an arrangement with the Univer- sity of Southern California, the LosAngeles City Schools will inter- view in Europe for recruits for this program.We have gone to Hawaii Mr. Baldwin can tell you of hisexperiences there in seeking teachers.

Recruitment is a very difficult business. The competition is keen. We have to search out andproceed in every way we can to help our school system get superiorteachers. That, briefly, is our

161 recruitment program. As I tried to emphasize earlier, that phase of the program will not succeed in itsobjective unless there is a very strong alliancewith the teacher education institutions. There are several forcesat work trying to get our school system to take over its own teachertraining. That is not our business. Our busi- ness is staffing theschools and teaching--not teacher training. At least one member of our State Boardof Education would like to have Los Angeles relieve this shortageby general hiring of baccalaureate degree graduates, and training them to beteachers through the re- sources of the school system.We have to do some of this in critical emergencies, but we want to keep such hiring down asmuch' as possible.

The second column of the chart outlinesthe first appraisal. This is the interview and contract offer bythe recruiter. The re- cruiter may see the candidate in New York City,Chicago, anywhere in the U. S.; he may see the candidate on a campusin California.We go just as far as we can toward a contractoffer at the time of inter- view. Personal qualifications are always appraised,and if all the college records required for appraisal purposes areavailable, an offer of a contract may be made at that time. On our local campuses, where the records are complete for our purpose, wenot only offer the contracts, but the candidates, if theyaccept, talk immediately with our Assignment Administrators regardingthe schools in Los An= geles where they may teach. Phase One, the recruitment, is the reaching out, the searching for teachers; andPhase Two is the inter- view, drawing them into the school systemif they are evaluated as )ping qualified for one year of service, the initialemployment.

The next phase is the tryout year; that isthe third column on the chart. This tryout year is extremely important,and I was very much interested in Dr.Bray's response this morning, when he em- phasized the attrition in a tryout year. This is exactly what is happening in our district. We know that, with the kind of initial appraisal we have to make, there isgoing to be a definite percentage of -the new teachers who can't make it, whoaren't qualified for regu- lar service.Approximately 9 to 10 per cent of our newconditional or provisional teachersreceive no better than average ratings,with the probability that a very substantialpercentage of this group will not make it.They are encouraged to resign, and many ofthem do resign from the school system.Now I might emphasize that during this tryout year we see just as great anobligation on the part of our staff as on the partof the new teacher. The teacher must prove, himself; he must get ready for the formalexamination.We must help him to know the Los Angeles school system,its philosophy, its cur- riculum, its teaching methods. This is extremely important, I feel, as a selection device,the tgo-vay street during this conditional year of service. It's a probationary year under State law. We can't counteract that, but the condition for continuing oninto a second and third year of probation is successfulpassing of the formal ex- amination. That is phase four, the most important one inmaintain- ing the high standards of our school system.

162 All elementary teacher candidates must pass anEnglish Usage Test to continue on to other phases of theexamination. An objective test covering basic subject matter areas,methods, tech- niques and philosophy, has a weight of 30 per cent.An evaluation of training and experience by a committee has a weight of40 per cent. A second committee evaluates personal characteristics.

Now running all through this, there is a centralstrand of importance as far as we are able to ascertain from ourcorrelation studies. The central strand basically starts with the strengthof the recruitment program, whether the recruiter makessound judgments in terms of qualifications to enter service,whether we unearth well- qualified candidates. If we can't, the whole process falls down. The judgment of the principal during thetryout year, as to whether the teacher is performing up to standard, asreflected in the per- formance report, becomes the most important factorin the formal examination as far as our studies have indicated. Now what we have here is a dual system of selection, an initialselection and then a formal screening during the tryout year.In the past, a number of years ago, we had just theformal examination system. It didn't produce enough people to staff our schools. Our operating divisions, elementary, secondary and college, stronglyfavor the dual system. They want the Personnel Division to get goodpeople, after initial screening, into the school system. They want to try these people out, help them get started, and be perhaps themost important factor in determining whether they stay on andbecome career teachers. So this is the heart of the selection program.I think it is desirable for our system. Whether it is good for any other districts, I have no idea. This double screening system makes a heavierload in proc- essing, but there are some clear indications thatit is working well for our school system. The fifth and final phase is the second and third year of probationary service which providesthe final apprais- al before the teacher enters upon a careerjob. Very few teachers are terminated duringthese two years of service.

Briefly, that's the way we select in Los Angeles, I've learned some things this morning that havealready given me clues as to wherethis procedure can be strengthened. I won't go into those now. I found particularly helpful some excellentpoints on teacher education and the great importanceof a closer working re- lationship in terms of recruiting early in the college career,in the freshman, sophomore and junior years, andgetting better records in the college as a sounder basis for ourrecruiters to appraise these young pelple before we decide whetherthey should serve with us.

We have a number of problems. One is the necessity for the dual screening.Another involves the credential problems which the state imposes on us. Los Angeles requires 5 years of preparation

163 and trying to keep a for all teachers,elementary and secondary, critical shortage is five-year standard inthe present period of requesting the State to extremely difficult.We have considered give us our own credentialstructure. Four We have problems intraining provisionalteachers. elementary shortage. hundred were hired inmid-August to relieve an intensive training in our We had two weeksin which to give them enough. We have masterteachers helping them, schools--that wasn't of them but some of them arenot succeeding.We hope the majority will make the grade. with our surround- A third problemis that of competition the good new How a large schoolsystem can hold onto ing districts. rate for new teachers teachers it gets, Idon't know. Our turnover We know that salaryis an importantfactor both in is fairly high. district has to recruiting and holdingteachers.We believe our in California, just as a have one of thehighest teacher's salaries selection program. Our minimum now,with a bacca- phase of a good We laureate degree and noexperience, is$6,220 for the first year. in a state for alarge feel that salary mustbe at or near the top district in poverty areas,to city, which has adefinite part of its retain high qualityteachers. the picture at the There are somefavorable factors in increasing in our areaand present time. College enrollments are We believe in building upthe prestige we areencouraged by this. recruitment job and gettingout of teaching, doingthe right kind of recruiter for the LosAngeles into our colleges more. I find, as a that I have to go in a City Schools, that oneof the first places The president must college or university isthe president'soffice. teacher education be in on the organizingof the college to help people for the jobs wehave available. It can't be prepare young of placed upon the Deanof Education alone. He needs the support the president, andall the faculty. encouraging. The innovations inteacher training are very Many new and I believe thisis a vital part ofdesirable selection. and universities. promising programs areevolving in our colleges teachers to serv- We should plan forthe return of former of good staffing I think the only way wewill meet the problem ice. teaching in is to think of every youngwoman whoenters elementary know she won't staytoo long Los Angeles as apossible returnee. We then she will be outfive in most cases, probablytwo to six years; return to service. to ten years to raise afamily; and then she may teachers than When these womenreturn, they aretypically stronger when they left us.

1611.. The compensatory education programs, and we in California fortunately have several State programs that are helping out, are beginning to make teaching in the inner-city more attractive, by lowered norms, increased remedial instruction, and specialized as- sistance to the pupils in various fields.Something is happening here in the United States in our large cities that I believe is go- ing to be an important factor in drawing strong teachers to the central city and encouraging them to stay there. I go into the classroom in school after school, and see a teacher with a small group of children around her learning the elements ofEnglish. This is heartening and our teachers are enjoying this kind of service.

We have had excellent community response to recruitment appeals in the face of shortages. I believe that our communities can work much more closely with us than we have asked them to do in the past.When we reached the middle of August this past summer and found that we were 800 teachers short, we had already gone through all of our existing recruitment programs. There were no more teach- ers available.We went to television; we went to radio; we went to the newspapers. Nine hundred applications came in, and 600 were employed. This was one of the most heartening things that has happened in our school system.

Mr. Chairman, I think that very briefly tells you what we have been trying to do and the direction in which we're working. I haven't gone into details on the parts of our examination program. I believe we heard some very important things this morning on that.

I want to repeat that this has been a most worthwhile ex- perience for me, and I want to thank all of you for the opportunity to share with you in this conference. Thank you very much.

Arthur Klein: Thank you, Dr. Brown. I'm sure that there will be some questions later after we've had an opportunity to hear from the two speakers who will follow you. Dr. Redfern, now in Washington, formerly in Cincinnati, will tell us about his personnel experiences in the selection of teachers in Cincinnati.

George Redfern: Thank you, Dr. Klein.Members of the panel and colleagues of the conference. May I say, at the outset, that even though I am now associated with the American Association of School Administrators, I am still interested in and will be work- ing with large city problems. I am certain that one of the prime large city problems is likely to continue to be that of teacher se- lection. The only difference in my present situation is that I'm not obliged to worry about meeting that September teacher employment deadline when school opens. I can also look at the teacher selec- tion problem from a somewhat more academic point of view.

165 large as LosAngeles, group ofcities, not as I represent a written examinationsnor thankful, whichneither use for which I'm Cincinnati, Ibelieve, services ofboards ofexaminers. have the Some of thefollowingconditions may betypical of thesecities. characterize theirselection procedures. process, basically apreliminary screening 1. Selection is scores the establishmentof eligibility 2. In Cincinnati, frequently required,and and rankings are eligibility scores orrankings. Appointment ismade from 3. orrankings are Candidates havingthe highest scores given priority. four position papershows, there are In Cincinnati, asmy grouping must be candidates inthe highest eligibilitygroupings and in the nexthighest grouping. appointedprior to those I'm considering,are ob- Not all cities,in the category for candidatesfor selection liged to computeeligibility scores however, it hasbeen arequirement since In Cincinnati, advent of purposes. World War II. Even with the the depressiondays before the Board of in the lasttwenty years, the severeteacher shortage selection to remove oralter the basic Education hasnot seen fit in ac- relaxation hasbeen permitted requirements eventhough some than normallywould bepermitted. cepting candidateswith lower scores Nor- objective is toselect from among. Still the chief 1800 ap- system processesfrom 1500 to mally, theCincinnati school total about600 The number ofvacancies will plications annually. 100 duringthe September openingof school and a year(500 for the two-thirds re- one-third areemployed and school year). Thus about procedures used. jected throughthe selection academic record, selection factorsused are: The three It would befoolish references, andoral interview. professional factors are validand precise for me toclaim thatthese three teachers employed areof the highest enough to assurethat all possible quality. transcript of is judged byanalyzing the Academic record student teachingrecord. particular emphasisupon the credits with measureof aca- is used asthe significant The gradepoint average demic proficiency. (usually five innumber) are used Professionalreferences unreliability of criterion.Recognizing the as thesecond selection forms pro- the references arerequired on generalrecommendations, be scored. This permits school system sothat they can vided by the Care isexercised tomake sure better comparisonsamongcandidates.

166 that references are providedby those who have haddirect supervisory position to make contact with the applicantand presumably are in a relevant judgments as tothe competency of thecandidate. Nor- The third selectioncriterion is the oralinterview. mally, it is a groupinterview and conductedwherever possible at of applicants Board of Educationheadquarters. A gratifying number though ex- are ableto come to Cincinnatifor the oral interview even people em- pense money cannotbe provided. About 50 per cent of all ployed come from outsideOhio, which represents aconsiderable load interviewed of interviews.Most of theseout-of-town candidates are on Saturday byinterviewing teams organizedby the personnel division.

The three selectioncriteria are weighted. Academic record cent; and the counts 30 per cent;professional references, 20 per oral interview,50 per cent.

As I understandit, my position paper hasnot been put in over-simplification rather than your hands,but I'm going to risk bore you by reading the paperwhich gives more detailregarding the Cincinnati selection process.

The selection proceduresused in Cincinnati are notunique -- observations they aren't unusual. However, I'd like to make some about the type of teacherselection which they represent. As 1. Selection has to be madewithin a framework of reality. So it Dr. Brown has said,that reality is severe inlarge cities. the demand that Los is in Cincinnati where wehave only a fraction of Angeles has. in the 2. Another reality is thenegative image of teaching to change that image inthe large cities. Despite persistent effort attractive. minds of candidates, theystill find suburban schools more schools can and do attract In Cincinnati, forexample, the suburban over half ofall the graduates ofthe University of Cincinnati. employed in the Formerly, more thantwo-thirds of its graduates were Cincinnati Public Schools. assign 3. Even though universitiesin or near large cities may students to do theirstudent teaching in theinner-city schools, this does not guaranteethat the graduate willseek employment in observation. the city. Our experience inCincinnati confirms this

matter of high demand andlow supply 4. Another reality is the especially in critical teaching areas. You know the familiar ones.

the lack of good positionspecifications . 5. A fifth reality is This which are needed inrecruitment and especially inplacement.

167 to correct, how- is a problem within thecapacity of the school system ever. There may be many specifications whicheach vacancy requires. If principals fail to specifythese requirements, recruitment and selection is made more difficult.

6. Another very significant reality isthe wish of the candi- date to know where his placementwill be at the time of employment. Most large cities have notfound an effective way to assure a spe- cific placement at time of employment,especially during the spring recruitment season. This is a handicap of serious proportionsin large city teacher selection.

7. An additional reality is thatthere are two general classi- fications of applicant. One is the interim teacher; the other,the career teacher. The former has short range interests; thelatter, a career expectancy. Much that has been said this morning by ourbusi- ness and industrypanelists has been more relevant for the career than the interim teacher.

8. An eighth reality concerns money. In Cincinnati, we have a salarypolicy that attempts to keep teachersalaries competitive. Salaries are fixed at the 75thpercentile of the 41 largest cities. While this would seem to establish afavorable competitive position, it cannot overcome the manyadvantages--monetary and non- monetary -- which competitive suburban communities possess.

9. A final reality is therather unique requirement that em- ployment in school systems peakssharply at one time during the year; namely, inSeptember with the opening of school. Many other professions are able to smoothe out theiremployment needs.Hy con- tention is that since all school systemshave this peak September employment problem, more mistakes are madein teacher selection than if employment needs could be moreequally distributed throughout the school year.

In the time that remains, Iwish to make some more obser- vations about the non-examinationtype of selection procedures. After listening to Dr. Mitzel andDr. Medley, I admit that Idontt feel very comfortable about the adequacyof the Cincinnati teacher selection criteria. Admitting the weaknesses of the criteria,I can only say that we are obliged to livewith them for the present at least.

I am convinced, however,that the criteria can be refined and should be. The following steps may be useful inthis refinement process:

1. Better position requirements mus, prepared by principals describing the expectations of theinner-city teacher. With a

168 clearer understanding of what isexpected of the teacher, selection may be made moreeffective.

2. Each selection criterion shouldbe carefully evaluated.An attempt might be made to correlateeach with a success criterion which possibly could beascertained from the appraisal recordsof the teacher during the probationaryperiod. If the selection cri- terion correlates favorably withthe teaching success criterion, a greater degree of confidence canbe manifested in its effectiveness.

3. Professional references might be"field tested', by a sam- pling process. For example, a random sample ofwritten references might be verified by telephonecalls to the individuals whomade the references. The purpose would be to seeif a conversation would yield additional or differentinformation than was indicated onthe original reference form.

1. The oral interview could bescrutinized. Types of inter- views might be compared, i.e.,variations might be made in the com- position of interviewing teams to seeif one combination appeared more effectivethan another.

5. The priority given the variouseligibility groups might be checked for validity. For example, scores achievedin the highest priority grouping might becorrelated with the teaching successcri- terion achieved during theprobationary period. It might well be established that high priority groupcandidates do not consistently turn out to be high teaching successteachers. If, on the other hand, they do, it would tendto verify the logic of appointingfirst from the top eligibility grouprather than the next highest group.

There are probably other waysto evaluate the selection criteria. The point is that refinementsprobably can and should be made in the procedures.

In conclusion, I sincerelyhope that this conference may advancethe cause o2 better selectionof teachers for the large cities. We need better ways toexchange and disseminate information and We needmore dialogue betweenschool personnel administrators closer con- membersof boards of examiners.We likewise should have tacts with those, at the universitylevel, who are preparing teach- ers. We need the help of thosein the universities who have ex- pertise in research design who canassist school personnel people and board of examinerspersonnel do a more effectivejob of evalu- ating selection techniques.

I hope that out ofthis conference may come abetter mechanism for exchange anddissemination of teacherselection data American Association of and information.I also hope that if the

169 School Administrators, as anorganization, can be useful in helping large city personnel administratorsfulfill their roles better that it may have the opportunity ofdoing so. Thank you very much.

Arthur Klein: Our third speaker, Dr. Albert Schiffof Detroit.

Albert Schiff:Mr. Chairman, colleagues - no cityschool system, worth its salt, wants to lowerteacher standards, but the needs of the community are immediateproblems and cannot wait for long range plans.Teacher selection has always been a calculated risk, but we can minimize this risk sothat even the probationary period of which we've spoken in some of oursessions is a period of strengthening the teacher and not another screeningdevice. We all know too well the incalculableharm an incompetent primary teacher can do during his first yearof teaching to a group of youngsters just learning to read. This is unforgivable.

It has been ten years since I'veentered the personnel field. About 600 to 700 new teachers were allthat the city of De- troit needed to meet its needs in1956. We now search for 2000 teachers every year, to help meet the needsof the growing community, to lower pupil-teacher ratio, to keepboth the Federal programs as well as our local school programsoperating and to offer all the other services that we've always said weneeded. Even with funds available, we are short of personnel, and2000 teachers are getting rougher and rougher to fitd.

What we have to do - what we must do -is re-examine the supply lines of teachers. Present training facilities are numeri- cally and qualitatively inadequate tomeet the needs of education today. They must be reorganized, they must bereformed, they must be augmented by new types of programs,and in many cases new types of institutions. We must make a nation-wide analysisof teacher training and the utilization of teachers, sothat every level of education, elementary, secondary and college, canbe assured of a steady flaw of high quality instructors. This is the crux of the problem that big city schools face.

For the most part, large city schoolsystems have set up screening devices and techniques to weedout the less qualified and the less desirable teacher candidateswho have already been gradu- ated from teacher training institutions. Boards of Examiners and other personnel officials, and Detroit isincluded, set up programs of testing and interviewing and examinationof credentials, at great expenditure of funds (I just heard yesterdaythat the Board of Ex- aminers in New York spends$1,500,000 on thi0 to insure that their school systems do not employ teachers who can giveless than quality education to their students. This is deplorable especially at a

170 time when staffs of this naturecould be devoting their time and their energy in working out newerand better ways of selecting teach- ers for thepresent and for the future.No profession has such a rate of turnover and mortality asthe teaching profession. It thus becomes apparent to many of us that wehave to find the techniques of selection that will identify thecandidate who has a commitment to teaching and will subsequentlyremain in the field for a life- time career.

Selection procedures as they pertainto personnel adminis- trators begin with the college senior and,although our selection procedures in Detroit are perhaps asadequate as most large city school systems, I question thecontinuing need for teacher selection by personnel administrators ofschool systems when the selective process for futureteachers should begin much earlier thanthe col- lege student who is approachinggraduation. In Detroit we're often hurled this challenge, 'Why can't wecompete with the suburbs in terms of quality education, class size,salary, etc. ?" Let's not fool ourselves--the sooner we face thisproblem realistically and indicate to all concerned that largecity school systems, such. as Detroit, New York, Chicago, and others arenot suburban areas, that our teachers mustreflect the ideals and the characteristicswhich are necessaryfor effective teaching in a large industrialand cos- mopolitan city. Then and only then, can we come to gripswith cer- tain basic procedures to befollowed in the selection of teachers for the fUture.

In Detroit, over the past few years,the Personnel Divi- sion has sponsored clubs for futureteachers, and I don't want to minimize this. In 1957 at the beginning of ourefforts, we had 12 high school clubs serving 300 members.We now have almost 5000 pupils participating in the161 school units.An average of 25 to 30 members is the usual size of these groups,although some clubs range as few, atthe last count, from 8 members in oneschool to 130 in another.Even in our society, as mobile as itis in terms of changes teachers may make during theirlifetime, the greatest source of future teachers for any large cityrests with the high school stu- dents in the classrooms of that city. I hate to use an old cliche, but it's so true, that the "teachersof tomorrow are in our high schools today," and if you go too far awayin recruiting teachers - at least we find this to be true -the less chance we have to get teachers to come to Detroit.

There's not much solace in finding that schoolsystems are competing for the pitifully few highlyqualified, competent teachers available. I am also weary of thesewonderful, beautiful recruit -. ment brochures of how to attract teachersto our respective commu- nities. These techniques may be very wellfor business, but I often wonder if it is desirable for educators.

171 The selection ofprospective teachers should becontinued through the college anduniversity level, beginning atthe freshman with year. School systems shouldprovide a co-ordinator to work teacher training collegesin their areas. The aim should be to help Isn't it a para- the prospective teacherand suggest shortage areas. dox that we still have asurplus of social studiesteachers and be- in cause of thatsurplus we're having themteach math and other areas which we have criticalshortages? Let us providechild study oppor- get them tunities through classroomvisitations, teacher conferences, youth jobs as camp counselors, secureemployment in year-round In brief, we haveto groups, aidthem in materials for class papers. and education re- extend the servicesprovided by teacher education be achieved search divisions to theend that greater cooperation can Bill between the school systemand the teacher traininginstitution. Brown, George Redfern,and I all agree thatthis close cooperation with teacher traininginstitutions is importantand imperative, and the internship program canbe an extraordinarilysuccessful training device leading to a realfunctional partnership betweenteacher edu- Let's look favorably cation institutions andpublic school systems. upon the useof Federal funds to help usin a search for quality teachers for the future.Dr. Brownell, formerSuperintendent of permis- Schools in Detroit, who isattending this conference, gave us sion to modify our teacherstandards last year.We had an experi- mental group of seventycandidates enroll at WayneState University college credit in who were post-degreecandidates with little or no careful education. They worked with peopleout in the field under supervision and along withstaff of the teachertraining institution. We still have overforty of these peopleenrolled now and this is their second semester.While this may appear tobe a small number, There we've gained forty teachersby means which provedsuccessful. must be other programsthat are equally valuable.

Special needs of ametropolitan area, and thisof course I do not need to call to yourattention, demand raciallyintegrated to meet the needs school staffs and trainingof prospective teachers teacher train- of deprived or disadvantagedyouth. We do call upon for teaching about whom ing institutions notto recommend candidates competence, per- there are any questionsof general intelligence or sonality, or other factorswhich preclude successin teaching. to assume that these School personneladministrators have a right candidates are ready tontacklen the rigors ofthe classroom if they have been or are giventhe aid of adequatesupervisory and administra- school systems will needto modify tive staff. I believe large city to alleviate the rigid teacher recruitmentand selection procedures while teacher shortage by adoptionof temporary stop gap measures problems. still attacking theproblem of effective long range for work- Dr. Brownell laidthe groundwork this past year institution in our area. ing out plans withthe teacher training

172 Task Force I is Five Task Forces areat work at the presenttime. devoting its time tothe development ofteacher internship programs students--and the developmentof for undergraduatesand post-degree post-degree and/or M.A.T. programsin teacher specially designed working in the education (campus andoff-campus). Task Force II is for the prepa- area ofexnansion and modificationof current programs Task Force III isworking ration of teachersfor inner-cityschools. for the preparation in the developmentof specially designed programs aides (campus andoff- of teacher assistantsand para-professional policies and proce- Task Force IV iscurrently developing campus). post-degree students dures for the recruitmentof undergraduate and its time to the for teacher education. Task Force V is devoting follow-up studies, ac- organization of researchteams for conducting of teacher education tion research,evaluation and assessment programs.

It is the plan ofthese Task Forces tohave some recom- mendations in operationfor the February, 1967,school semester.

Thank you very much.

Discussion

position papers that were Arthur Klein:When I read the you've just heard, I was submitted by the threegentlemen from whom approach that they had delighted with theforthright and realistic administrators in the to the great problemsconfronting personnel when I heard one ofthe large cities. I was amused, of course, familiar with in the speakers repeat thequestion that all of us are "Why can't you competewith the suburbs?"Well, of large cities: for example, course, youpeople can't supplythe answer in Cincinnati, In New York City, because you haven't anorganized testing program. of the answer: "It's the people who askthe question are always sure your darnexaminations that keep themout." you'd like to ask I know that youhave questions that You may recall that these gentlemen. One or two commentsfirst. which has been underscored Harry Rivlin pointedout the importance, by some of our speakersthis afternoon, ofthe selection process examination proper that begins in thecolleges. Then there is the. is the probationary by the selection agencyand thereafter there period of There is a continuingselection process over a period. forcefully the years, butI was glad to hear Dr.Schiff voice very untrained or undesirable thought that one can'tsubject children to teachers, merely because we canfall back on theprobationary period Dr. Rivlin left withFred for the eliminationof those teachers. Williams a message tobe read to us. Fred.

Fred Williams:Thank you Mr. Chairman. Dr. Rivlin ap- parently felt that outof the conversationhe had last night there

173 may have been somemisinterpretations and wantedthe opportunity to sure what the con- clarify his point. f wasn't present, so I'm not fusion was, but I'm surethis will be of benefitto all who are here. I'll read this as he gaveit to me.

"The keepers of the gate arekey people in any school sys- procedure, but I tem. I do not question theneed for a selection hope we can find better waysof selection. I think research money evaluating perform- will be invested betterin improving our ways of the improvement of the ex- ance on thejob than by concentrating on into the system.Xhere amination proceduresfor determining entrants this is only a screen- must be selection forinitial appointments, but promising ing test. Those who are passedshould be 'accepted as risks' and should be evaluatedduring their period ofprobationary service before being grantedtenure. Provision for evaluating prior experience is importantfor two reasons: one, previousexperience and two, such evalua- may be a usefulpart of the screening process; tion can be part of arecruitment process.

"Why shouldn't the Boardof Education raidother school and principals to systems by invitingunusually successful teachers find the join New York's schoolsystem? Can the Board of Examiners of perform- time and personnel resourcesto focus on the evaluation time-consuming chore of conduct- ance, if itrelieves itself of the by availing ing, administering, andrating written examinations How can we get itself of such prefessionaltest agencies as E.T.S? other school systems good experienced teachersand principals from For example, can we givethe and yet be fair,objective and honest? examinations privately to thosewho have been evaluatedin their job must continue to in their own homesettings? The Board of Examiners honestly, re- be the keeper of the gate,selecting impartially and has a re- gardless of race, creed orpolitical power, but it also first set of sponsibility for evaluatingthose who get through the gates." addressed to our three Arthur Klein: And now questions speakers.

Ralph Walter:Dr. Schiff stressedthe need for the col- As far as the ex- leges to cooperate withthe city school systems. is another side perience of the studentsis concerned, I think there in junior to that coin.We've just gone throughthis experience: time year, we have1000 juniors who go outand spend part of their We in city schools andpart of their time insuburban schools. could not found great cooperation inthe suburban schools, but we cities place our youngsters inthe city schools. Two of the largest in the state turned usdown on this, and Ithink if something could be done to induce them toaccept, it would behelpfUl.

rth that's hard tobelieve. Albert Schiff: Someone here said City, and Dr. that that's going tohappen in New York I don't think So you tell thebe- Lang has justsaid he'll take themin New York. River is fordable. nighted people in NewJersey that the Hudson this question toDr. Jules Kolodny: I'd like to direct in-state to severalhun- Brown. You referred tovisits being made out-of-state as well,to several hundredcolleges. dred colleges, and visits in connec- You've talked ofvisits to Hawaii;you've talked of recruitment among thebacca- tion with the Armyschools, returnees, professions, etc. May I ask laureate graduateswho were in other assigned to thistask; and these two questions: how many people are what is the size ofthe budget appropriation? question, there aretwo William Brown: In answer to your about 15 professionalrecruiters forces involved. There is a force of that, many schoolprincipals are who work year-round. In addition to two, three, or fourweeks de- relieved of theirassignments for one, Supervisors from theoperating pending upon thelength of the trip. Division budget forrecruit- divisions are used. So, the Personnel the salaries of the15 regular staff memberswith ment would include of the salary of approximately$15,000, and the salaries an average estimated principals and thesupervisors. Wayland (Baldwin), you to use the schoolpersonnel. once how muchit cost the district What would that cometo? those figures at thepresent Wayland Baldwin: I don't have including examination com- In our wholeselection procedure, time. principals, super- mittees where wemake generous useof our school also in the recruitment visors, vice principals,and teachers, and I would estimatethat effort, we use about athousand people a year. days for each one, onthe aver- this would averageabout three or four into several This is a district-wideenterprise and runs age. give you any exactfigures. hundred thousanddollars, but I can't Two Charles Cogen: I would like tomake three comments. nature, one of apositive of them are more orless of a negative nature. have been some A good dealhas been said, andI think there submitted by thesespeakers, about therole of the fine suggestions school systems and so on. teachers' college andcooperating with the consideration that wehgve to bear in mindis the in- Now I think one lot about, adequacy of manyteachers' colleges.We hear an awful in my experience--studentswho come into the and we see--I've seen realistic classroom who had neverheard of a lessonplan and other foreign to the subjectmat- things in our teachingwhich are a little who teach much, much ter of many facultiesof teachers' colleges too theoretically.

175 Secondly, another negativefactor is that we haveto be role of teacher evalua- concerned about thesupervisors in this whole been mentioned atall in any tion and screening.Now this has hardly this time; namely, thefact of the discussionthat we've had up to incompetent to do the jobof that supervisorsfrequently are very assisting the teacher or mayhave personalities thatreact unfavor- laissez-faire attitude. There are all kindsof atti- ably or have a against plywing in tudes on the parts ofsupervisors which militate bring forth a goodproduct, to the teacher thatwhich is needed to much consideration. and this is somethingthat I think bears very candidate with How do you take accountof that in comparing one another when they'vehad different kinds ofsupervision?

Finally, a more positivenote: the question hasrecurred session but at othersessions, over and overagain, not only at this recruit and retain of the problem of theinner-city and how we can of the teachers for these difficultschools. This is probably one I think, in that connec- crucial problems of theday in education. mention- tion, we ought tomention--and I don't seehow we can avoid into the ing--the fact that we needto pour tremendous resources the teachers and to schools in order to makethe job attractive to to refer only brieflyto the hold them. In that connection I want I have visited a More EffectiveSchools program in NewYork City. firsthand, and I've heardmuch couple of them and Ihave seen them I won't go intodetail more aboutthem second- andthird-hand. there- - since I think mostof you are familiarwith the situation Los Ange- the saturation ofservices, and so on. I understand that what is happening les sent an investigatorto New York City to see with extremely glowing in the "more effectiveschools" and came back will solve our dis- reports, and said thatthis is the thing that knowledge,. Los Angeleshas advantaged schoolproblem; and yet, to my not introduced that program. sitting with Theodore Lang:I haven't hadthe privilege of evening, so I do nothave the benefit you thismorning and yesterday with you. It is alzo of your earlierdiscussions. It is good to be have problems comforting to know thatother large metropolitan areas those almost identical to ours. As each of youspoke this afternoon, In the spirit of of us from New Yorknodded our heads inagreement. of the panelists who learning, I have questionsto direct to a number spoke here today.

But, before I askthose questions I wouldlike to recommend, to because no one spokeof it, the techniqueof manpower projection early enough date so determine personnel needsfor September at an school that there can bebetter preparationthan is the case when a made such system first knowsof its criticalshortage in August. We able September 1966 projectionslast February. As a result, we were approximately 2,000. (This to anticipate ateacher shortage of

176 deficit stems from the samefactors that account for yourshortages, funds for expanding primarily, the availabilityof Federal and state compensatory education.) Thereupon, we trainedapproximately 2,000 college graduates duringthe summer by enlistingthe assistance of conducting an inten- one of our bestteacher training colleges and six to sive teacher training program.The program extended from seven weeks, givingthe participants, who had noprevious pedagogic preparation, six or eightcredits of professionalbackground for secondary school and elementaryschool, respectively, sothat they Of could be brought into oursystem as teachers inSeptember. course, they werenot adequately trained evenwith that degree of approach than first recog- preparation. However, it is a much better nizing your problem in August orSeptember, and then havingto re- without cruit and place in the classroombaccalaureate degree holders to de- any professionalgrounding. So, I would strongly urge you to determine velop the technique ofprojecting your manpower needs in.your systems than the probable shortages. This should be simpler of substitute teach- in our organizationbecause we have a huge group ers (about 30 percent of the total) and ourmethod of assigning these substitutes, in distinctionto the appointmentprocedure that applies to regularteachers, is such that we arenot able to compile accurate data on theinflow and outgo of substitutes.

Now, on the specificquestions, in order tobetter under- stand each of yourpresentations - To Dr. Brown: I would be inter- ested in knowing, of allthe contracts that youmake with teachers, what percentage resultfrom contract offers madeat campus inter- anticipated views?And my second questionto you concerns the teachers who fail your failure rate: What is the proportion of of initial teach- formal examination afterhaving completed one year ing with satisfactoryperformance?

William Brown:Before I answer yourquestion, I would We feel, like to underscore theimportance of manpowerprojections. project well ahead, from a recruitmentstandpoint, that if we cannot fore- we cannot dothe best recruitment job. Also, we have a serious casting difficulty at thepresent time because wedo not know for certain that a new State Lawin California will bein operation in needs by 350 our districtfor February. This law would increase our obstacles in additional elementary teachers. There are some serious applying to the State Boardof Education and meetingthe provisions ahead of the law. I concur strongly in theimportance of projecting insofar as that's feasible.

Theodore Lang: The second question wasexpected failure - of those who have the one yearof initial teachingexperience, who satis- take your formalexamination, but limitingit to those who had factory performancereports, what percentagefail?

William Brown:Those who were strong oroutstanding?

177 Theodore Lang:No, those who weresatisfactory. cent, Dr. Lang, would William Brown:Approximately 15 per be about as close asI could come onthat.

Thank you. The question I havefor Dr. Theodore Lang: approximately 50 per Redfern is on theoral interviews: You say that those who cannot come cent take the oralinterviews. What happens to which is a weightedpart of in to headquartersfor the interview, yourexamination? The campus interviews,in practically all George Redfern: is on- instances, are madeby a team and theoral interview then that about 85 ducted on the campusby the team.Now, I would say level, would have atwo-team per cent ofthe people at the campus interview, but there are someinstances of only oneinterview. contract offers atthe Theodore Lang:Do you make many campus?

George Redfern: Wemake no contractoffers at campus, anybody a contract even though weneed them badly.We will not give he processes theapplication. at the time ofthe interview, until Dr. Schiff, Theodore Lang: The question toDr. Schiff: serious problems isthe nature of theinitial assignment one of our in New York I know it variesfrom school to school of the teacher. stories in Head- depending upon theprincipal. However, we hear many including those who quarters of beginning,inexperienced teachers, who were gi.c,en the took our summerintensive preparation program, toughest class in theschool for theirfirst-year assignment--the teacher has difficultyin kind of class that evenan experienced such initial assignmentsin your handling. How do you control system? the question as best Albert Schiff: I'd be glad to answer there may be some spe- I can, but Idid suggest tothe chairman that mainly affect NewYork, Chi- cific problem areasthat I think would discuss in a smaller cago,Philadelphia, etc.,that we may want to group session. and the Board Basically, the policyof the Superintendeut place in to try insofar aspossible to alternately of Education, was teacher and an experi- where vacanciesexist a beginning a school system of Now in reality,of course, you have a enced teacher. known as a voluntary and involuntarytransfers. We have a policy three-year probationaryperiod, careertransfer. At the end of the transfer in the Detroitschools.We every teacheris subject to third year a highersalary in- also give a teacherat the end of the crement instead ofthe normal salaryincrement.

178 Theodore Lang:I can clarify the question. I think the problem is not so much the school of assignmentas the class within the school to which the new teacher is assigned. I feel that there are many good classes in disadvantaged areas whicha beginning teach- er could handle adequately and there are very bad classes in privi- leged neighborhoods, with whicha beginning teacher would have dif- ficulty. It is the character of the initial assignmentwhich I am interested in learning to control because principalswill take the experienced teacher and graduallymove him to the better classes, with the residual problem classes being left forthe beginning teacher.

Arthur Klein: I'd like to turn the meeting back to the chairman, but I've been urgently requested by thesetwo gentlemen to try to arrange with Mr. Dennis approvala rescheduling of a dis- cussion period between 8:30 and 9:30 tonight in thisroom of repre- sentatives of the large city school systems to discussgreater co- ordination and more study in depth of the problemsthat confront the large city.

179 Session No. 5: Criteria: Problems in Validating Teacher Selection Policies and Procedures

Chairman: Paul Denn, Board of Examiners

David G. Ryans, University of Hawaii Donald Medley, Educational Testing Service Harold Mittel, Pennsylvania State University

Paul Denn: Most of us are in agreement thatwe must make some professional attempt to select, in an objective and validway, those whom we want to teach our children. I think when we go off into a discussion of projected needs of personnel,inner- and outer- city rivalry, teacher shortages andso on, we tend to forget that the individual teacher, into whose classroomwe're going to send our own child, is the one to whom we are entrusting the fate of that child. The fact that there areenormous problems to be encountered when we embark on such selection does notdetract from our obliga- tion to make the effort. The problems may be complex and difficult but I, for one, believe that theyare not insuperable. Like a second-place transportation agency,we must try harder. One major key, I believe, lies in research, in depth,more intensive research and, perhaps, more scientific research, to establishvalid criteria for selection and to develop the instrumentsthat are needed.

We are fortunate in having three researchmen who have been working very extensively in this field.I'd like to introduce them and have them present their views in turnnow. First, I'd like to introduce someone who is known, I believe,as the Dean of Educa- tional Research. He was president of the American Educational Re- search Association. He's the author ofa standard work on charac- teristics of teachers and, at present, he's theDirector of the Edu- cational Research Center of the University ofHawaii, a most desir- able place to be. Dr. David Ryans.

David Ryans: Coming before you following the mostgra- cious and flatteringly generous introduction, andknowing full well I do not live up to my billing, I feelvery humble--and also very reluctant to open my mouth at all. I should have stayed back at my place in the audience and restedon the laurels so kindly bestowed.

There are other reasons formy feeling somewhat reluctant to pursue the topic around which I built theposition paper submitted to those responsible forour meeting. One reason is that I fear what I may have to say will sound rather academic tosome of youand most of us think presentations of that sortare likely to be much more ab- stract than practical. I hope my remarks can be completely sound, but I also hope they can be acceptedas practical in nature. To me, what I have intended to talk about does notseem academic; I sincerely

180 hope it will not appear so to you. For I do indeed agree with those of you who have pointed out that in thepublic schools we must oper- ate in a context of reality. But, as has been amply demonstrated by some of the personsparticipating in this meeting, sound research can be accomplished in the public schoolreal world; we can operate in the context of reality and still specifyteaching criteria, develop procedures for predicting those criteria,and test those predictors against our agreed-upon criteria.

Someone once said Josiah Royce followed aprocedure in lec- turing in which he first told hislisteners what he was going to say, then told them what he had to say, andfinally told them what he had saiu. I will not be as brash as to try toemulate Josiah Royce, even to th9 extent of patterning myoutline after his formula. But I would like to begin (particularly since youdo not have copies of my prepared paper) by giving an overviewof what I want to say and what I tried to cover in my prepared paper. I want to say a few words about what we mean when we talkabout criteria and the criterion probiem--and why it is important for teacherselection; and I want to say something about teacherselection as I think it should be planned in light of agreed-upon criteria;and I had intended to talk briefly about some of the problems we facein trying to designate criteria and obtain estimates ofcriteria. (Note that I am not talk- ing about teacher selection procedures, assuch, at the moment, but about the criterion problem andestimates of criteria of teaching against which we may judge the extentto which teacher selection pro- cedures are working.) My major position will be one Ihave taken a number of times in discussions of thissort--that the criterion prob- lem is at the same time the mostimportant and the most neglected of the problems of teacher selection. This is true, I think, whether we are concerned withcontent validity, or predictive validity, or whatever approach we take to the problemof validation.

I am going to read my prepared paperin part, inserting comments occasionally, and skippingabout quite freely. I hope the excerpts will not have too muchdiscontinuity.

A good deal of confusion appearsto exist with regard to just what we mean when the termcriterion, or criteria, is employed.

A criterion is simply astandard or bench-mark use: to pro- vide a frame of reference for judging orevaluating something. It may be thought o1 as amodel against which comparisons may Tv...made. Usually criteria evolve from commonagreement about acceptable stand- ards--regulatory boards for insurance,public utilities, banking, contracting, and such operate with a set ofagreed-upon standards (criteria) as a model.In many circumstances (I think Ishould say in all circumstances) criteria arearbitrary and relative to values that are held to be important by someparticular group of persons at

81 hope it will not appear so to you. For I do indeed agree with those of you who have pointed out that in thepublic schools we must oper- ate in a context of reality. But, as has been amply demonstrated by some of the personsparticipating in this meeting, sound research can be accomplished in the public school realworld; we can operate in the context of reality and still specifyteaching criteria, develop procedures for predicting thosecriteria, and test those predictors against our agreed-upon criteria.

Someone once said Josiah Roycefollowed a procedure in lec- turing in which he first told hislisteners what he was going to say, then told them what he had to say, andfinally told them what he had saiu. I will not be as brash as totry to emulate Josiah Royce, even to the extent of patterning myoutline after his formula. But I would like to begin(particularly since you do not have copies of my prepared paper) by giving anoverview of what I want to say andwhat I tried to cover in may prepared paper. I want to say a few words about what we mean when we talkabout criteria and the criterion problem--and why it is important forteacher selection; and I want to say something about teacherselection as I think it should be planned in light of agreed-upon criteria;and I had intended to talk bxiefly about some of the problems weface in trying to designate criteria and obtain estimates of criteria. (Note that I am not talk- ing about teacher selection procedures, assuch, at the moment, but about the criterion problem and estimatesof criteria of teaching against which we may judge the extentto which teacher selection pro- cedures are working.) My major positionwill be one I have taken a number of times in discussions of thissort--that the criterion prob- lem is at the same time the mostimportant and the most neglected of the problems of teacher selection. This is true, I think, whether we are concernedwith content validity, or predictivevalidity, or whatever approach we take to the problemof validation.

I am going to read my prepared paperin part, inserting comments occasionally, and skippingabout quite freely. I hope the excerpts will not have too muchdiscontinuity.

A good deal of confusion appearsto exist with regard to just what we mean when the termcriterion, or criteria, is employed.

A criterion is simply astandard or bench-mark used to pro- vide a frame of reference forjudging or evaluating something. It may be thoughtof as a model against which comparisons maybe made. Usually criteria evolve from commonagreement about acceptable stand- ards--regulatory boards for insurance,public utilities, banking, contracting, and such operate with a setof agreed-upon standards (criteria) as a model.In many circumstances (I think Ishould say in all circumstances) criteria arearbitrary and relative to values that are held to be important by someparticular group of persons at

181 at some particular time andplace. Indeed, the matter of "values" and "value systems" is basicto the consideration of criteria.

Now in relation to teacherselection, just as preferences and opinions (values) ofindividuals vary with regard to the compe- tencies and behaviors expected ofteachers, so the criteria against which teacher selectionprocedures should be compared will often vary (at leastin certain features) from community tocommunity; and validity studies of teacher selectionnearly always require replica- tion adapted to the varying conditions.

In taking this position thatcriteria are determined by value contexts that differ amongschools and communities, I am imply- ing that the first step in theconsideration of criteria against which to judge a teacher selection programmust be to determine the expectations that are held locally withregard to teaching and teach- er behavior. The extent to which there may be concensusabout major issues, the greater the assurancewith which a school administration or its educationalresearchers may approach the designationof cri- teria and their components, and thegreater the possibility of con- ducting meaningful validitystudieth of teacher selection procedures.

I shall return to therelationship between criteria and value systems later, but I firstwould like to comment further on the nature of criteriaand the frequentneglect of, and confusions about, considerations relating to criteria.

Here I do not restrict my remarksto studies of teacher se- lection and research on teacherbehavior; they are no more vulner- able than a great deal ofresearch in the behavioral scienceswhere the problem of the dependentvariable, or the criterion, has been ne- glected. From years of reading researchreports and research propos- als, I conclude that manyotherwise elegantly designed researches- - well designed from the standpointof sampling, control of the experi- mental variable and other independentvariables, data analysis (and often involving real ingenuity ofapproach) - -have, almost as an after-thought it seems, settled upon someavailable instrument, or perhaps hastily thrown together some test,inventory, or other ob- servational technique without greatregard to its validity and re- liability--proceeding on the assumptionthat such an instrument satisfactorily reflected, and provided usefulestimates of, the cri- terion behavior. (A friend of mine sometimes refers to thissort of thing as the "law of the instrument"and points out it is appli- cable in many of our activities. He illustrates his "law of the instrument" by saying if you give a baby ahammer, the baby will find something to hit with it; or, as itmight be applied to the personnel field, if we have an available test orinstrument of some sort, we will find some place to use it--oftenwithout regard to what the pur- pose of our work maybe.) This happens to be a pet peeve ofmine- - the fact that so many investigators seemto neglect or give too little behavior) attention to controlof the dependentvariable (criterion problem from the very and that instead ofconsidering this important to give it only cursoryat- beginning of theirresearch, they appear tention some time laterin the investigation. had to do I must move along. The second part of my paper Perspec- with a rather vaguesection titled,"Teac'her Selection in to take the problemof In this sectionwhat I tried to do was tive." procedures we usually ap- the curriculum developerand think of the I then tried prove incarefully carried outcurriculum development. the same general con- to look at teacherselection research within to certain objectives,fol- text of developingprocedures to conform estimates of the attain- lowed by evaluatingthe procedures against constructor ment of the selectedobjectives. Just as the curriculum out those objectives properly first sets upobjectives, then spells materials, and theneval- operationally, then developsthe curriculum defined ob- uates the curriculummaterials againstthe operationally jectives, it seems to methe logical processfor teacher selection those personnel is to set upthe criteria ofteaching, spell out behaviors, developpredictor criteria objectivelyin terms of teacher to the criterionbe- procedures that arehypothesized to be related materials against the haviors, and finallyevaluate the predictor to spell out in someten steps the criteria. In my paper I try operations one mightappropriately employ. These indade: of the value (1) Selection and der...7nationof general aspects they relate to system framework ofthe school/community as here to the agreed-uponquali- teacher behavior. I am referring in a particular ties that are desired, orexpected, of teachers particular kinds ofteaching situations. (Note place and in is again that this processof arriving atcriteria necessarily individuals or groupsof subjective and a matterof the values criteria individuals may possessin common.When we designate We we proceedfrom a context of anaccepted value system. opinions, view teacher behaviorin light of a setof attitudes, sorts of teacherbehavior we and viewpointsthat reflect the kinds of behavior wedisapprove approve andprefer and also the To the extent any groupof persons and find unacceptable. biases share in common certainexpectancies, preferences, or teacher behavior maybe about teachers andteaching, criteria of defined for that particulargroup.) properties of teacherclass- (2) Identification of observable that may be relatedto the specifiedoperationally room behavior cataloging of teacher described criteria(i.e., the descriptive characteristics and behaviorsthat occur in theclassroom).

Identification of kindsof situations inwhich the agreed- (3) occur--and in whichthey upon valuedteachers behaviors!' may may beobserved and assessed.

183 (i.e., description in terms of ac- (4) Operational description tual teacher behaviors) of theagreed-upon valued behaviors that are to comprise the criteria ofteacher behavior.

(5) Selection of methods of estimatingthe operationally (i.e., behaviorally) described valued behaviors.This is the problem of instrumentation relative to thecriterion behavior and ob- taining assessments of the criterion behaviors. In assessing some aspect orcharacteristic of the criterion behavior of teachers we are trying to estimate the extentto which that de- fined characteristic is manifest by some teacher.

(6) Development of selection instruments andprocedures that are hypothesizedto yield estimates that will reflectthe opera- tionally described teacher behaviors(criterion behaviors)- - which, in turn, are assumed to reflectthe value framework of the school and the community served.

(7) Assembly of data yielded by theteacher selection instru- ments and procedures noted in Step6 above.

(8) Assembly of data yielded by theprocedures used to estimate the criterion behaviors--Step4 above.

between estimates of the be- (9) Analysis of relationships haviorally defined criterion behaviorand the estimates of teacher characteristics used in theteacher selection procedure.

(10) Evaluation of the teacher selectionprocedures by the drawing of inferences about the validityof those procedures for predicting the criterionbehaviors designated in Steps 1 through 3 above.

The next section of my paper dealswith "Common Confusions in Dealing with Aspects of theCriterion Problem." I think I shall skip most of this, but I do want tomention the importance (even though it may seem academic) ofdistinguishing between criteria and "estimates of criteria." I used two specific examples in my paper to try to illustrate the distinction, onerelating to "classroom management" and the other to "communicationof knowledge." In the case of "communicationof content" I suggested a fewof many cri- terion behaviors that might bedesignated (e.g., clarity of presenta- tion, subject matter depth, etc.),all of these having to do with definition of sub-behaviors involved inthe behavior we might call "classroom communication." In light of the criterion behaviorsthus designated, I then suggested severaldifferent ways we might go about estimating such teacherbehaviors--estimating the criteria to get the benchmarks against which we mightjudge our teacher selection procedures if we chose. Thus, we might employ observationprocedures to rate samples of teachercommunication behaviors or we might make

1814 frequency counts; or instead of resorting to a sample of the cri- terion behavior of the teacher 22E se, we might choose to view the criterion in terms of known or assumed correlates of "teacher com- munication of knowledge" revealed from measurements of the pupils.

The next section of my paper dealt with "Some Considera- tions in the Designation and Estimation of Criteria," and in this discussion I was primarily concerned with the need for trying to avoid bias (i.e., criterion bias) that frequently affects both the definition of criteria and also the development of measures or measuring instruments that yield estimates of the criteria. I will not dwell on this, but I think we should remember at. least two very important sources of criterion bias, one having to do with the ex- clusiveness of our criteria (and the estimates we later develop to reflect those criteria) and the inclusiveness of our criteria and criterion estimates.We do not want to include anything in our definition or estimation procedure that is not really a part of teaching behavior (that may contaminate our thinking), nor do we want to exclude anything that is really a part of teaching behavior. We could give many examples of criterion bias with respect to teacher selection procedures.

In subsequent sections of my paper I commented briefly upon approaches to criterion definition and also approaches to ob- taining criterion data.

May I remind you again that I have been talking almost en- tirely about the designation of criteria and of the development of estimates of criterion behavior in teaching--not about teacher selec- tion procedures. I have pointed out that ideally this should pre- cede the development and/or choice of teacher selection procedures and devices and that such teacher selection procedures as might be adopted would be ones presumably geared to the criteria. And in the ten-step paradigm it was not until steps 7, 8, 9, and 10 that we got along with research directed at determining whether or not our selec- tion procedures appeared to be doing what they were intended to do.

In practice this is not likely to be the way things work out. Very often we decide upon certain teacher selection instru- ments or procedures with only a vague idea of what criteria we are concerned with. To be sure the selection of devices has often been accomplished on the basis of considerable experience and wisdom grow- ing out of experience. But it is after the use of the instruments that we finally begin to wonder whether by chance they may be measur- ing the criteria we find acceptable in the community and school sys- tem. So, very often we enter my paradigm at about step 6 and then go back and retroactively try to think, "Well now, what was it we really wanted to be getting at with these devices ?" Perhaps this is justi- fied, or at least it is what we have to do in our "context of reality," the line, even ifit is done inretrospect, we but some place along of teach- need to try todesignate exactly whatit is that we expect ers and that wehope our predictors maypredict. remarks, I feel a strong As I come tothe conclusion of my this much toohurriedly, sense ofinadequacy. I know I have gone over several days just totalk about how and I wish wecould sit down for might go about outlining astudy in a particularschool system we of criteria and involvethe which would startwith the designation procedures matching of predictordata provided byteacher selection behaviors of teachers ap- against operationallydefined criterion Perhaps some day we cando this. proved in the particularcommunity.

May I read the lasthundred words or so of my paper.

As is the casewith most of youpresent, I have given a great deal of thoughtto the problem ofthe criterion, particularly relates to teacherbehavior and to theproblem of validity as it identify and study of teacherselection devices. I find it easy to with which we are recognize many of theproblems and difficulties descriptions of the cri- faced in trying todevelop satisfactory teachers and techniqueswhich will yield valid terion behavior of I recog- estimates of thecriterion behaviorinvolved in teaching. of criterion behavior nize the sources ofbias in the description of the estimatesyielded and the conditionsmaking for invalidity behavior. But I am ad- by different methodsof assessing criterion involved in obtainingcri- mittedly frustrated bythe difficulties and, on terion data which are, onone hand,inclusive and complete I know it is not the other, exclusiveand free of contamination. particularly when we mustfrequently easy to lickthese problems, where we have beenusing cer- conduct validitystudies in situations devices that wereselected on an a priori tain teacher selection descrip- basis without thebenefit of guidanceof adequate criterion faced with the problemof tions, and now, afterthe fact, we are estimates of criteriondescrip- providing proceduresthat will yield to test our selectiondata. I do not think the tions against which recognize, as situation is animpossible one, but Icannot help but considera- I think most of usmust, that we arefaced with practical methods tions which force usto compromise andemploy make-shift studies of the quality we that preclude thecarrying out of validity would like. not unfamiliar with Paul Denn: The next two panelists are They both were withthe the problems of educationin large cities. City for a periodof time. Board of HigherEducation in New York is now head of theTeacher Donald Medley, whois the next panelist, Testing Service inPrince- Behavior Research Groupof the Educational ton.

186 The dis- Donalc' Medley(raising microphone a fewinches): cussion is going to go onat a little higher levelfrom now on--in one sense, anyhow. I don't know whether Ilike to be here discussing problems which, as Paul hasmentioned, Harold Mitzel andI have al- ready spent several years onwhile we were doing researchin teaching that we left in defeat! in New York City.Maybe this is an admission

(Dr. Medley now read hisposition paper which isreproduced on page95.)

As I've beensitting here, I've beenthinking that this proposal is couched interms of the presentstructure of a teacher's and then the career where ateacher goes to school so many years training stops and sheteaches so many years. Apparently, the theory If you can just is that at some point ateacher becomes competent. get him to that point,then you have it made. I think if we thought through- of teaching and teachingcompetences as something that grows out the entire career,and if we thought ofthe selection problem as identifying steps, when ateacher has learned all that acollege can teach him, let's take himinto the school system, andwhen he has learned all that we canteach him in the schoolsystem, let's make Now if I had my way, up our mindwhether we want to keep him ornot. and what happens this distinction betweenwhat happens in a college Something like in the school systemwouldn't be nearly asclear-cut. Cities Con- what Harry Rivlin suggestedin his paper at the Great ference and what I thinkhe's trying to implementat Fordham where the transition from anundergraduate pre-servicetraining to in- gradual and it service training performanceis more or less much more the individual. is much more individuallytailored to the status of it out because you So in thinking aboutthis proposal, don't throw think it won't work underthe present system.After all, we're en- think might titled to talk about whatought to be as well as what we be.

Paul Denn: The next panelist, HaroldMitzel, is Assistant Dean of Research atPennsylvania State University.

Harold Mitzel:I'm reminded of themilitary convoy situa- travels at a constant rateof 35 tion. You know, a military convoy commander has set miles per hour, at leastthat's what the convoy that the when he takes off inthe lead jeep. But it always happens drives at a constant speedof guy who's onthe rear end of the convoy going 83 miles per hour just inorder to keep up. So if I seem to be catch up with this 83 miles per hour, it'sbecause I'm trying to Dave Ryans and convoy. It's a pleasure toshare another forum with with them in makingpresenta- Don Medley. I've had the joy of working like this tions in the literature,and an appearance before a group significant is another interestingadventure for me. I think it is the same that we all prepared our papersindependently, but some of threads or themes do runthrough them.

187 (Dr. Mitzel now read his position paperwhich is reproduced on page 99.)

Discussion

Paul Denn:Would you like to ask some questions, ormake some comment? Whowould like to begin?

Douglas Bray: I'd like to make a contentious comment. I feel a great contrast between the severeand immediate practical prob- lems in places like Detroit, LosAngeles, and New York and the formal- istic academic research suggested bythis panel. These two worlds are just so far apartthat very little progress will be made insolv- ing problems of teacher selection if weinsist on the kind of purity which has held back progress in thisfield for so many years. I really feel that something far merequickly workable is called for.

David Ryans: I don't think it's nearly asimpractical as it sounds. No. I can really get kind of worked upabout this be- cause you know we talkabout all of these practical problems that face us, and we forget about the factthat we can sample, as Harold was saying and as Don wassaying here. We can get samples of teach- ers and we can workwith them, and this doesn't mean that wehave to do this with everybody.We can carry on a parallel processof, call it expediency if you will, so far asteacher selection is concerned, at least this is my contention and oneof research which does have some foundation. You see, I could be contentious and say aslong as you keep on doing thingsthe way that you're doing now, you don't know what the dickens is happening and neverwill because you don't have any criteria to start with; you neverhave anything to test them against.

Douglas Bray: I heard someone, I think it was HaroldHunt, say that there are alot of "izes" that the administratorhas to use: supervise, organize, and finally, compromise -with the last of these perhaps being the most important. I think we can do some compromis- ing and at the same time achieve a goodpart of our goal here.

Samuel Brownell: It seems to me that in our discussionof teacher selection we're not so far apart aswhat might be suggested. Obviously, we need instruments.We need various ways to know how to put the teacher in the best place. We don't always have the chance to do that in a school system, so wedo need help in that.

If we are in a small school system,and I used to be a superintendent of a small school system(that was before I went to Detroit) and I knew that I had to fill a vacancyin a third grade in a particular school,the criteria of the kind of teacherneeded could be pretty clear because I knew theadministrator. I knew I had a stable situation.

188 When we get to the matter of selecting people fora large city school system, I think we have to recognize the fact that what we're trying to do is to find people whocan come into a school sys- tem, not a particular school.We're looking for people who arego- ing to make a career, who can be placed inone school or another. In other words, we're looking for people who are flexible, who have the general basic qualifications, and thenwe can fit into many types of situations. lktrthermore, we have to recognize the fact that people change. A teacher that we select today isn't the same as the teacher next year, nor ten years from now, nor twenty years from now. What we're trying to do in selecting teachers fora large school system is to find the people we ought to take into the large organization to work.

It seems to me that the emphasis on the selectionprocess has to be to protect the children against those whoare obviously incompetent, who haven't the characteristics thatwe can take the chance on. We're trying to do the same thing the life insurance doc- tor does--to get out the poorest risks.We may be wrong occasion- ally, just as they occasionally select poor risks. I'm just simply saying that I think if we take that into account, the fact that teachers are flexible and they do learn and theygrow and they change--then we're not quite as far aparton this as it was sug- gested because we have two different kinds of things we're trying to do, and we need two different kinds of processing.

Paul Denn: Sometimes some of us want more than that when our own children are involved. Dr. Baldwin?

Wayland Baldwin: At one time I felt impatient when I listened to the "researchers."I wondered if it would ever be pos- sible for researchers and practitioners to speak thesame language. Some two years ago I attended a s-day conference held in Tempe, Arizona. The topic for discussion was "Bridging the Gap Between Re- search and Practice." Before the five days had ended, researchers and practitioners were speaking the same language. I gained much from this conference. Particularly I was convinced that there- search expert has much to offer to the personnel practitioner. It is important that we get together. It is not possible to bridge this gap in a 2-day conference.We need much more time. I should like to recommend a 6-week seminar for selected leaders in the field of educational research and school personnel administration.

Paul Denn: Well, we can make a start, and let's try to do that because it's most important for us. Dr. Darland, you had your hand up.

David Darland: I'm merely going to say that I think you are closer to the truch of where you ought to be in terms of what

189 this meeting is aboutthan any time since I've arrived,and I heard Dr. Bray very clearly supportthe idea of criteria.Well, where did they get the one, thecriteria that you use--did you justreach out and pull it out of the air?

Jay Greene:I've a question, and I agreewith Dave be- cause I sensehere some closeness, a questionto the three people who just spoke. Aren't there differentcriteria and different in- struments at different levelsthat should be perfected toachieve special purposes? For example, the firstselection comes when a student is selected in thecollege to go into teachertraining or criteria that may be teacher education. Now at that level there are different than the criteriathat you want to describe later on. There should be criteria forputting them into this programand cer- tain instruments that oughtto be set up at that time forthat pur- Med- pose. If we did that, we would bemeasuring achievement, as Dr. ley points out. Then the next step comesafter he has graduated from this program and he wantsto be a beginning teacher.Here you want some criteria relatedto the beginning teacherphase, and you want to try to perfectcertain instruments limited onlyto the be- ginning teacher phase, because youcan't measure the other aspects. Here again, you'd be measuringhis achievement at that pointwhich Dr. Medley says is morepossible than getting intoprediction.

In the third level, afterhe has taught for a year andhad Then some teachingexperience, you're down to the realjob itself. you want to determine somedifferent criteria and at that time per- fect some instruments,perhaps the teaching test youtalked about, which could be introducedperhaps for the first time. You can't use it earlier and here, too, you aremeasuring on that level--achieve- ment rather thanprediction--which may come in thetotal situation. My question is: "Is it possible that we can use someof the instru- ments that you wouldreject because they're notsuitable for predic- tion but they are more suitablealong the way and to developdif- ferent sets of criteria?"

Paul Denn: Harold, would you want totake that first be- cause you hadpresented some steps in the process.

Donald Medley: I have a short answer, yes. It seems to me what Drs.Bray and Brownell say sortof hooked up in my mind. When Dr. Brownell talkedabout what we're really tryingto do, we're trying to identify teacherswho can go into a large citysystem and be assigned any place in thesystem and succeed there, beflexible. Flexibility has been added to allthe things that Harold talksabout between the job we're and suggests. I think the gap here is really trying to do and the toolswe're trying to do it with. We're trying to pick this kind ofsuper-teacher with some kind of apaper-and- pencil test.

190 I remember someone used to say to me,if you.want to tell what a cowboy is going to do when het- drunk,find out what he did the last time he was drunk.And I think that we've just got to...if you want to attack thisproblem now, and you don't want to run away from it, you've got to get that teacher as close tothat situation he's going to be in as you can and see how he does in it.

Now we researchers are supposed to be attackingthe basic problem and finding out enough about the teaching process so we can help you, but it's going to be a while. It's going to be a long while and in the meantime, that's about the bestadvice that I think we can give you from thisside. But I do think that's what Dr. Greene has said several times, and everytime I want to get up and agree with him.

In the meantime, we can weed out some people,the people who never learned how to speak clearly enoughto be heard in the class and the people who are obviously in such poorcontrol of their own behavior that youcould see that they couldn't achieve in a class- room. We could still do something like that, but look atwhat' these people have now without bothering to prove thatthis is correlated with teacher effectiveness or not. One thing I know from all the re- search I've done and read, and one conclusionthat's most sturdy is that there's no way to tell how a person .is going toperform in the classroom, outside of the classroom; there's no testyet that will do it. I've got high hopes for simulations, but that'sin the future, as you know.

David Ryans: I just wanted to respond to say I agree also with what Dr. Greene said and make two comments. Talking about pre- diction research in any field, we often talk about, orwe've just roughly classified and people have talked about this for years,im- mediate criteria and intermediate criteria andultimate criteria that are further along. For example, what the medical doctor is ac- tually doing after he's out in practice 20 years as comparedto pre- dicting his grades in medical school which would be immediate.

I also wanted to add that in trying to predict, notcri- teria of teacher effectiveness but certain criteria that Ichose to identify and try to predict, the further I went back, thatis, did the predictive testing back in college days, the.lowerthe correla- tions got with the criteria measures that we obtained whenthey were actually teaching. On the other hand, we could get some prettyhigh cross validity coefficients bytaking two different samples and could get some very high cross validity coefficientswith teachers who were on the job and at the present stage oftheir performance. I'm just agreeing, that's all.

Perry Kalick: I think that we can all agree that the pay- off is in the actual classroom performance of theteacher. The

191 Board of Examiners is aware ofthis too in their giving of the per- formance and classroom teaching tests. However, the fact that these tests are expensive and relativelyquite time-consuming propositions probably accounts for their increasingdisappearance from the teacher selection process. In addition, a candidatets performance in a class of middle class children mighthardly be indicative of his per- formance in the difficult class to whichhe will be appointed. I think the practicality of the useof the teaching test rests upon the question of how many samplesof teaching performance are neces- sary and the questionof whether there are enough funds available for the tests. Furthermore, there is the problem of logistics; i.e., where are you going to givethe examination, and how many qualified people do you have readily availableto make the assess- ments?

Dr. Brownell talks about not taking achance. Can we af- ford to take a chance, particularly in viewof the damage that is done in the first year of teaching?In the teacher selection proc- ess we have not yetbeen able to be predictive of teacher hostility in the classroom. Jersild would say that almost a classical analy- sis of a teacher is necessary to find outwhether or not a teacher is going to do real damage, verballyand/or otherwise. In view of the cost of the latter, perhaps we should makepre-service teaching observation and the probationary period part ofthe selection process.

Douglas Bray: These comments sort of jell one of my ob- servations to what I thought they were saying.. I think the real gain is to be made by knocking off thebottom end of the scale. If you can knock out theone-third of the candidates who are manifestlylack- ing, you would make a substantial practicalgain. You don't need these elaborate research designs to achievethis. One other comment on the criteria: many people seem tothink that observation in the classroom would provide the best criteriajust as in business people believe in training the supervisor to ratejob performance. I'm not so sure, however, thatsimulation does not yield better criteria. You can not only have a highly trainedobserver group, but you can standardize the performance situation.

David Darland: There may be another factor we should be considering. In business and industry whenthey find that the struc- ture is in the way of function, theychange the structure. And they are much freer to do so. However, education is going to be re- organized. But I sense that statements being madehere on the evalua- tion of teachers are based on the theory thatthe self-contained classroom is going to continue. Moreover, maybe the major cities of the United States are just simplyunmanageable not only from the point of view of education but with respect to manyproblems.

Paul Denn: Thank you, ladies and gentlemen. The meeting is adjourned until 8:30 p.m.

192 Session No. 6: Open Discussion

Chairman: Paul Denn, Board ofExaminers

Paul Denn:Mr. Charles Cogen, Presidentof the American Federation of Teachers, hashad a tremendous amountof experience with respect to largeurban education, certainlybecause the largest part of his followingis located in these areas,and he is acquainted I've asked him to then at firsthand with anumber of these problems. get the ball rollingfor us by telling us afew things about his ex- periences in large urbaneducation and some of theproblems as he sees them. Charles Cogen.

Charles Cogen:Thank you, Paul. As a host and also as a former colleague, Mr. Dennasked, as we were enteringthe room here, comments, I sup- whether I would be willingand ready to make a few pose representingthe teacher's point ofview--how the teacher sees this problem of urban schoolsand along with it, of course,the staffing of the urban schools.

Now, I've had myexperiences in New York City, as youall know, as a teacher for many yearsand also as a lowerechelon super- visor, as a departmentchairman. But I've also come intocontact have been in with many teachers and others,such as professors who touch with the problem intheir own areas throughoutthe country.

I think it is not somuch a matter only of sizeof the school system that makes NewYork City so differentfrom other the geo- cities. In New York by the verymagnitude of its problem, graphic situation, the ethnicproblems, and so on, I thinkthe prob- lem becomes a qualitative one aswell as a quantitative one.

Now from the point ofview of the teacher. We are talking situation right at now about thosewho are thrown into a classroom the start of their careers;the teacher comes in, inmost cases, is fair to without any real practicalguidance, and is, I think it made earlier in the say, very muchin a fog. One of the comments I evening applies very muchhere. There is not enough guidancegiven tide him over this to the beginning teacher toset him at ease, to the critical stage. That confronts everyteacher but in particular teacher in the inner-cityschool.

And you have the problemsrelated to the disadvantaged of cul- children. The teacher comes acrossthese problems, not only tural deprivations, but alsoof the under-privilegedfrom the point discipline that of view of emotional handicaps;also the problems of teacher the teacher has toconfront. This is something that every And so I has to face and is verymuch concerned about, as youknow. overcoming the diffi- feel it is imperative thatthere be some way of culties that the teacherhas to confront whenhe comes into a setting of this kind. 193 Now I mentioned earlier in theevening the More Effective Schools program. I feel that something along those lines isimpera- tive from the point of view ofrecruiting and retaining the teacher. We must have a setting wherethe teacher is able to give individual attention. I walked through ,a school the other day, oneof the "more effective schools," and you would hardlybelieve that there were any real problems existent there amongthe children. The teacher has few children to hendl'il she has all kindsof assistance--teachers aides; teachers in training who really give somehelp, individual help; sup- plies of kinds that they never haveelsewhere. These are things that help the teacher to overcome the greatinitial handicap which they suffer when they get into adisadvantaged school. And of course there is the problem of class size.

I would say that just about asimportant as anything else is this matter of large classsize. This is especially significant in the school with disadvantagedchildren. In such a school, where you have also largeclass sizes, the problem is aggravatedto a much greater degree. And so long as we do not adequatelyface this prob- lem of class size, and also theproblem which I haven't mentioned yet, of salaries, you're notgoing to get anywhere from the pointof view of recruitment and holding onto the teacher. I was at the con- ference on the education of thedisadvantaged a few months ago at the White House. There were all kinds of proposals that weremade at the panels--remedial reading andungraded classes, and so on and so forth, but youhardly heard anything at all, except frommyself and v colleague from the AmericanFederation of Teachers, on the two big questions of salary and classsize. School systems are try- ing to run away from them, but Idon't see how you can escape them.

And finally--not finally by wayof covering the subject by any means, but fromthe point of view of the timelimitation that has been imposed and that I wantto impose upon myselfI think the problem of adequate supervision isextremely important. We need supervision that is democratic, that ishelpful, and constructive; and I do not use these terms inthe loose sort of way that is often used in college training classes. You really have to have a super- visor who is on the job with theteacher. I think one of the things that the Board of Education mightconsider in any city would be for a supervisorto do some teaching as well assupervising, just the same as a departmentchairman does in the New York Cityschools. The supervisor needs to keep hisfinger in the pie, so to speak, to realize what the problems are, tobe really working on a colleague basis with his teachers rather than on abasis of superior towards inferior.

Now I know there are manyother problems, but I think these are the major ones thatteachers throughout the countryin the large cities are thinking aboutand worrying about'. Of course, you

194 duty free lunch have the matters ofnon-teaching chores and the periods which aregradually being taken careof by coPective bar- of the fature, the gaining procedures. I think this is the wave cities and has collective bargaining thathas begun in the major and ad- gone into thesmaller communities aswell, where teachers problems and get to a com- ministrators togetherwork through their imposed upon the mon understanding,rather than having decisions through from both teachers from abovewithout any clear thinking things that I think we sides of the table. These are some of the problems of recruiting ought to keep in mindwhen we consider the and retainingteachers. point of view of the Fred Williams:Mr. Cogen gave the problem that the teacherfaces in the school ofthe disadvantaged and to give the pointof view I would just like tobe presumptuous enough with respect to someof of the problem thedisadvantaged child faces develop the kind ofrapport the teachers we havewho are not able to necessary to do aneffective job teaching thatchild. to the Thus, when you mentionineducability with respect the problem takes child as one of theproblems, I would say that successful in giving place in reverse also. Sometimes we are not teacher to function suc- the teacher whateveris necessary for that heart of what I want tomention- - cessfully. This leads me to the earlier in the day, that is, those elementsthat were alluded to for success in theinner- and which areconsidered to be essential suggested that we shouldlook to the city. A couple of speakers because some seemed placement of the teacherin a specific situation This capacity is not one better able to functionin the inner-city. if this is some- of the elements that ismeasured and I'm wondering What are these particu- thing to which we shouldturn our attention. measuring or are wemeasuring lar characteristics?Are they worth significant than some of them or can we measurethem?Are they more them that are racially the things we aremeasuring?Are there any of associated? necessary corre- I don't want tosuggest that there is any characteristics but, as a lation between colorand these particular society in which wehave general premise, hasthe nature of the minority group teacher lived created a conditionwhich would allow a characteristics for functioningin the inner-city to a to have these minor- greater extent than wouldbe true of scmeonewho is not of the the degree to ity group?This leads then to thenext items, namely, successful in obtainingminority which large citieshave been and are effort is being madein staff group members onstaff, and how much integration. two questions in may Paul Denn:Well, Fred, you raised teacher with a mind. One is, can we train amiddle class teacher, a implications middle class background,regardless of any racial

195 nationalities and races because there aremiddle class people of all is to teach the childrenin the inner-city. And the other question raised from an implicationI got from what you say, can ateacher brought up in a minorityenvironment teach in a middleclass area? Would What I'm suggestingis that I think that both arepossible. you agree orwould you disagree?

Fred Williams: Paul, yes, I thinkthat both are possible. head start because he What I am saying is Iwonder if someone has a youngsters, either through has lived a life moreclosely akin to these Is such a economic circumstances orbecause of racialidentification. characteristics which would person more likelyto have the particular of rapport. I'm make him more successfulat least in the development and so on. not talking now aboutmethodology or content knowledge This is the point thatI'm raising.

Paul Denn: I give the firstspeaker the prerogative.

Charles Cogen. Just one second. I think at one point I referred to the group asbeing 4neducable; I reallydidn't mean it stand under the givencondi- that way. Ineducable from where they tions but certainly notfrom a sound psychologicaland wholesome underprivileged and this iswhat point of view. There is then the orientation to the setting I meant, theyhaven't gotten their proper that they are put in and wehave to give them thiscompensatory education, head start, and so on.

Harry Gilbert:It was because werealized there was going to be a good deal ofattention to the problemsof the culturally dif- ferent that we invitedto this conference asocial psychologist and a after long deliberation, so we sociologist. Each one turned us down sociolo- were stuck atthe end and didn't havethe representation of comments in gists or social psychologistshere. I want to make a few that we're doing a very this area, however.I feel very strongly great disservice in ourlabels of the culturallydifferent as cultur- racist, actually, in ally deprived, etc.,and we're being downright assuming a kind ofhomogeneity among these groups. times, A very simple studythat I've alluded to many, many conducted by our New YorkCity Board of Education, seemsto me a Justman's simple very great big eye-opener. I'm talking about Joe study of pupils he calledstable and mobile. I'd like to take a minute or two to tell youabout this because Ithink it nails down homogeneous group. this great point of thinkingwe're dealing with a

Justman took the youngstersin completely Negroschools in He found Harlem and BedfordStuyvesant who were inthe sixth grade. that about half of themhad been in the very sameschool since the third grade, so he hadrecords on these youngstersfrom third to

196 sixth grade in the sameschool. He found that the mean IQ forthis group, whichhe called the stable groupbecause they were in the same school, was about 100.8 in thethird grade and about102.4 in the sixth grade, something we liketo call a difference that issignifi- cant statistically butoperationally not much, different. But for the other youngsters, the otherhalf, who were in the sixth gradebut had transferred one, two, three or moretimes from third to sixth grade, he found that the initial IQ was98 something and then 94 in the sixth grade. When he analyzed that further,he found that the kids who were transferred four or moretimes had gone from somethinglike 88 to 84. The point is that the moretransfers, the lower the IQ initially and the lower it becomesin the sixth grade. He repeated this study with achievementtests and got the very same results.

Now this is a simplestatement of finding. These are facts; he found that these kidswhom he called stable had an IQ andachieve- ment just like the rest ofthe city, but the kids who weremobile started lower and kept gettingprogressively lower. Now when you take this whole group and put ittogether, you get the common finding that has been repeated over and overagain, in other words...

Paul Denn: Harry, if I may interrupt...If youhad taken the statistics only for thesixth grade, without analyzing themout or going back, itwould look as if the whole grouphad regressed.

Harry Gilbert:And this is the common point thatis re- peated over and over again,that the youngsters in the schoolsin the slums start out lower andkeep dropping lower. I think there's a massing effect that occursthere, and I submit it's a very racist kind of massing and somethingthat should not be done.Now I'm cer- tainly for our trying to see whatteachers are successful for what children, but I think we had betterbe very careful of our analyzing the youngsters and not treatingthem as though they are a homogeneous group. This is a blind alley.

When Charley Cogen was talkingabout what we need, he men- tioned all these things in the moreeffective schools, and a plague on that name,this is giving tt a label andit's saying it's success- ful before it even is evaluated. I'm sick of calling projects "Higher Horizons" and "MoreEffective Schools" before we'veevaluated them. By golly, you and I know that someof these schools are more effective than others, and I knowthat for some of the kids,the schools have been effective,for some they are not, and I'm alittle tired of the way we've compressedand homogenized all theresults to- gether. What I'm suggesting is that thedimensions of this problem are sc very deepand so very complicated.

I know something about thesemobile kids because I was a school psychologist, and I'vebeen in their schools and theirhomes. I don't I think the schoolsdon't know how to treat thesechildren.

197 care what kind ofteacher you're going to get, youcannot substitute for the dire effects of the deepsocial pathology. This has a long history, economics, political,social and there are a lot of reasons that are societal as well as personaland familial.

But if we're going to expectthe schools to be able to treat these youngsters, then I thinkwe're starting out with a bar- gain we'll never be able tofulfill. I think we better get readyto face the fact that we need somereal social dynamics that areof an order that is unacceptable yet. That means taking these kids away from the home, and I've triedthis. I have been in court on a, ne- glect charge in a home wheretherd was drunkenness, and the kids were absolutely terribly neglected. A smart lawyer made a monkeyof me when he had the mother enter courtin a nice black dress with awhite handkerchief and a lot of tears. You can't take kids away from mothers, and any smart lawyer canmake any foolish psychologistlook even morefoolish.

This is why I say we've gotto do some things that are daring and different. I am greatly concerned aboutall of our pro- grams, our headstarts or whatever we do wherewe're trying things for large groups withoutreally being analytical anddiagnostic about what we're doing forwhom. Excuse me; this is a topic on which I feel very greatly, and Ican't help expressing it.

Paul Denn: You started by saying that we seemto have com- mon problemsall over the country whereverwehave a large urban cen- ter and let's see whether ornot that's so. Let me ask, for example, some of the peoplefrom around the country, what do youconcede to be your most serious problem asfar as urban education isconcerned in your area?Let's start with Detroit. Al, would you?

Albert Schiff:I have listened with greatinterest to Fi.ed Williams and also CharlesCogen's statement of problems asthey view them. I'm not going to go into HarryGilbert's now because Harry is dealing with a verysmall select sample of thethousands of youngsters we have in ourclassroom. In Detroit, the immediateprob- lem we have is staffingclassrooms with able, competentteachers. This is the problem.We have only two sourcesof teachers, (1) the beginning teacher, (2) theexperienced teacher. If we have vacancies in the inner-city that wehave not been able to staff,the Negro com- munity would say, "See, how can wehave quality education when you don't even give us a classroomteacher ?V

It becomes a racial issue aswell as a quality education issue. Out in the peripheral city area,if we can't find a math teacher, the community may acceptit as a shortage of mathteachers, but that same shortage inthe inner-city has otherovertones. There is no question that theNegro community can show that wehave more substitutes filling regularpositions in the inner-city than wehave

198 difference what in the peripheral areas. And it doesn't make any logical explanations you cangive, such as "teachersstill have the In the final right to select the placeswhere they want to teach." analysis you can say,"Here is your assignment,"but they still have In Detroit, we place a rightto accept or rejectthat assignment. teachers under certainpolicies and proceduresand Ted Lang raised 4 in the question about principals. Principals have little to say indicate to a placement of teachers inDetroit. All a principal can placement officer is his needfor a math teacher. He cannot tell us, to my school and I "I know somebody whois asking for a transfer want you to give himthait assignment."

Detroit operates itsassignment policy--bothinitial ap- staff pointments and transfers--inaccordance with the "balanced well-balanced concept." This means every attemptis made to have a teachers, race and staff as it pertains tobeginning and experienced Detroit has an annual racial census. The last census in Octo- sex. We ber showed that allschools in Detroit areracially integrated. currently have 31 per centtotal instructional staffwho are Negro. This is why I look withgreat trepidation uponthe theory that Ne- groes can bestperform only in the areawhere they have a better knowledge of the culture ofthe Negro.Now, I don't know whether can't have real in- this is ghat Fred had inmind--but I am sure you tegration of staff by limitingthe placement of Negroesonly in Negro schools.

The majority of Negroteachers are still inpredominantly experienced teachers to move Negro areas. Our problem is to get more chapters of the American into the idner-city--andI hope the local Federation of Teachersand the National EducationAssociation will is the time has come help us in this area. What I am really saying education, must now whenteacher organizations,who have a stake in convince their members tohelp in the inner-citywhere experienced teachers are needed. Perhaps, a trial basisfor a period of two teacher years with theunderstanding that if itdoesn't work out, the can be returnedto his former school. New York tried voluntary transfers and only a halfdozen people or so werewilling to do this.

In Detroit a conditionof promotion to anadministrative position is experience inthe inner-city. This works all right for of the the few seeking promotion,but it still doesn't take care larger critical needto really staff the schoolswith experienced teachers in greater numbers.

Now, a word aboutactual placement. Here is a school that That is not a truly has two Negro teachersout of a staff of forty. integrated school.Not when 30 per centof your teachers are Negro. lists- - There is now a vacancyin that school.We have eligibility We have two sources: a socialstudies vacancy exists inthat school.

199 (1) a transfer list of peoplewho have asked for transfers, or(2) Negro in a beginningteacher list. We make the judgment to place a that school. In order to give this schoolracial balance we go to our transferlist until we reach the first Negrowho has asked for a transfer to that area and weoffer it to him. Or we go down our eli- gibility list for beginningteachers and we pick the first Negroto fill that spot.No if's, and's, or but's aboutit. This is the policy of the Board of Educationto achieve a balanced staff.Racial balance is one of the factors inplacement and you're not goingto get racial integrationif you side step these issues.

I think it's importantthat the teacher organizationshelp us. They want to help, but not whenit's at the cost of teacher morale. When they start talking aboutwhat's good for the system and about the need for smallerclass size and other supportive serv- ices, our reply should be thesekids are without teachers nowand they are in your school system. Can't we get the experiencedteach- ers to go tothese schools and help us duringthis emergency, be- cause we can getall the beginning teachers wewant to go into the more privileged areas. This is an easy thing todo.

I think I've spoken enough. I feel so strongly aboutthis because people talk about whatthey want to do but when thechips are down,nobody is willing to take thestand and the personnel administrator is in the middleof it; the principal, God blesshim, is in the middle of it; andthe teacher is certainly in themiddle of it.

Harry. Gilbert: How do you know the race of ateacher?

Albert Schiff:Although we do not code teachersby race, almost every teacher whom youemploy comes in for interview or as- transcripts from some south- signment.Now we know that if you hve ern universities orthe teacher had attended asegregated school, we can assume we arenot mistaken about race. Sure, we miss sometimes and often we make mistakes--we'vesent some white teachers when we really thought they were Negroteachers--but we were right in about 90 per cent of the cases. By the way, Harry, thereis nothing in the law that prevents the codingof teachers after employment. In other words, race cannot be a conditionof employment, but we cancode our teachers any time afteremployment. We don't do it in Detroit. Many of these people willbe seeking promotion later on so evenif the law provides that we maycode them, we feel it prudentnot to do so.

Paul Denn: Okay. Let's give Fred achance to get in on this.

Fred Williams:I just want to respond to apoint that you raised, Al.

200 I'm not in favor of the all black schools, staff and pupils or anything else, nor am I suggesting that only the Negro can teach a Negro child by any means.What I am saying is that because of the kind of society we have had, unfortunately, it may be that a Negro teacher, generally, may have a greater likelihood of developing rap- port, and if this is an important characteristic, then we ought to try to tap it to the extent possible in getting this characteristic into our schools and taking advantage of it.

Paul Denn: Fred, I go for what Al said, namely, if we spread them around the schools. I think it's just as important for the white youngster to know the Negro teachers.

Fred Williams: Believe me, I've devoted much effort to this essential aspect and it is why I raised the corollary question of staff integration.

Paul Denn: As one of my friends used to say, "Let's not waste time in heated agreement. Jules?

Jules Kolodny: There are se many things kicking around here, I don't know where to begin; but, I'm going to start with the fellow at my right.

I would like to remind you, Al, that when you say you have an emergency today and cannot organize a program of smaller classes naw because it will require five or ten years to develop, I would re- mind you that the Federations with which I am familiar tried to get you ready for today five years ago and ten years ago; and you didn't listen.When I say "you," I don't mean Detroit; I mean New York, Los Angeles, etc. People suddenly get excited about Watts when the thing explodes. But the Los Angeles Federation of Teachers has been talking about inadequate education there for more than a decade, so that it isn't quite fair to say that teachers aren't prepared to go with you, when you have allowed the situation to deteriorate.

I would like to go to the question of race and faculty in -. tegration. I sat down with a committee headed by Fred Williams and a few others. We worked out a plan by which our Central Placement Bureau would try to identify Negroes by virtue of the fact that we know we have ghettoes and even if you have a Bachelor's Degree, you are more likely to live in the ghetto or on the periphery of the ghetto as it moves north or south into a white community.And I think Ted would agree with me that the procedure worked fairly well. I don't think we had 100 per cent success; but we were able to iden- tify Negroes by virtue of their address. This would include the graduates of southern colleges and the city college graduate, as well as the New York University graduate who lived at a certain ad- dress which was in a deeply populated Negro community.

201 ilmive.arirmimppoinnorrappinggimesw

changing the courses Now, earlier in theday I talked about somebody took me totask and that are requiredfor teachers, and Let me give you anidea of'what said I wasn't abreastof the times. back to the typeof thing that Tedhas I mean. I think this comes simple illustrationwith- been talking about. I'm going to use a very child is taught from in the Puerto Ricancommunity. A Puerto Rican take him to taskfor infancy on that whenhis elders scold him or Then he comes to New misbehavior, he shouldhide his head inshame. the teacher York, lives in aPuerto Rican sectionof our city and what does the average He holds his headlow in shame, and scolds him. "When I'm talking to white middle classteacher say to such achild? This is a cultural you, I want youto look me straightin the face." with the best ofintentions dynamic which iscompletely misunderstood simple illustrationof the kind by the teacher. I use this as a very in teacher traininginstitutes of program I wouldlike to see embodied sociology, psychology,minority in terms of groupdynamics, cultural of the sort Idescribed. culture, history andunderstanding of things thanges in preparation,despite the I still thinkthere's room for fact I was told thatI'm not abreastof the times. classes Let's go on alittle further. Wetalked of smaller when we quite that simple. Perhaps, if ten years ago and life isn't But, when you talk talked of smallerclasses, we'd be readyin 1966. in 1966, the answergiven is that wedon't have of smaller classes Now if don't have the capitalbudget to do this. the space and we have a you would,by making yourclasses smaller, you had the space, teach- problem because you'dneed considerably more greater personnel to do is go We recognize thisproblem. Perhaps what we ought ers. relationship of maybe into another kindof thinking. Try to keep a that one is free torelax, one is three teachers andtwo classes, so out for individualat- free to pull three orfour or six youngsters Because we are dealingwith a tention during partsof the day, etc. aspect of the problem,the reduction of totality as you affect one But, impinge upon thelack of supply forpersonnel. class size, you We ought to give that doesn't mean wehave to throw in the sponge. devices. thought to newdevices and I thinkthat there are new schools Let me go on tothe kind of teacherwho is sent into Teachers are membersof the human race in the disadvantagedareas. disabilities andhostilities and and they sufferfrom all the same percentage of teacherswho are racists;how hatreds. There is a small Board of Examinersis hard to tell;but they did; and they got by the schools But, the rest ofthem are not. They go into the I meet them. enthusiasm; but they best of intentionsand with real with the very skills and they don't don't have theknowledge and theydon't have the pointed out thatthey're not have the tools, andCharley Cogen has I think it behooves usto getting the kind ofhelp that they need. ought to look at JOMSof the supervisors ask, "Why?"Then, maybe we are formerteachers, and we've licensed.After all, supervisors

202 whatever inadequacies they have aredistributed in the same manner. Supervisors forget that they have been promoted from amongthe teach- ers; they try toput a halo over their heads as though they don't display this equal distribution of bell-shapedcompetency. They think that they are all equally competent. We have to look into schools in Harlem, for example, and the Harlemsthroughout the coun- try and lbok at the holding power of faculties. Why is it when you have two schools, six blocks apart, oneprincipal has a holding power and his seniority range of his facultyis that they stay there 11 to 114 years? And why is it that the, next fellow, every Septemberhas to start with a brand new faculty?The two schools have the same student body, are in the same socio-economic area, sameparents, and the same curriculum.

I think that we ought to look intothis aspect of the problem. What is it that makes one person successfulin this area and the next fellow unsuccessful?When the teacher gets into this situation, not only doesn't he get the helphe needs, but he probably has a class that is difficult because ofpatterns of behavior result- ing from retardation that already exists. The incidents of disci- pline are higher in this area and theteacher,, therefore, falls into one or another category. He either throws in the sponage and says, "I've had it" after a short time, or he takes alook at the school and doesn't even show on the first Monday. Some of them will go on the Monday, stay Monday and Tuesday, and they never reappear on Wednesday. Some of them are more adaptable; they figure out apeda- gogic device by which they can somehow orother control the disci- pline by the use of some sort of materials; but,it's not a true learning situation. The kids are managing not to kick up and then we wonder why, at the end of the year, there's readingretardation. The reading retardation is there because there has beenschooling, but no learning during that entire year.

There is the really professional teacher who wantsto do a good job but he really is inadequate because he has neverlearned the necessary skills from his, college training.Nor has he gotten it from his principal. It may very well be that the principal doesn't know how himself. It may well be that the principal starts withthe attitude that a lot of these' kids are ineducableand that if you just hold them in line, "that's the best and themost we can expect from you." I've heard this from some principals around town.What I'm really asking for here is that we begin to lookat the college pro- gram; that we begin tolook at what's happening in the schools; what's happening with respect to the kind of supervision weget.

We must see a distinction between the "mobilepopulation" and the "stable population. ".It's shameful, but the permanent record card has a tremendous story to tell at anytime, including the number of times a child has moved when he comesto a particular teacher in

203 he stays longer What has happenedto his scores as the fifth grade. fourth that they haveleveled off at aplateau at the in school is of this data in amuch bet- grade or thereabouts. We could make use Unfortunately, ourpsychologists ter way if wehad the facilities. then Deans ofcol- go and becomemembers of Boardsof Rcaminers and psychologists; but Ithink this is leges; so, wedon't have enough an area wecould look into. pathology that HarryGilbert What I am sayingis that the percentage of theto- about is presentin a much larger was talking It's a very sig- than you're willingto conceive. tal population the population the population andit's the part of nificant part of the kids who are with the normallearning process of that interferes ordinarily and who really arestable and would have not as mobile can develop I think we oughtto determine whether we done well. of getting parental con- residential centers. Because the question indicated, thereought to be the sent is verydifficult, as Harry time day schools,starting at 8 kind of programthat might be long the child to hishome at 6 o'clock in themorning and returning value in this kindof o'clock in the evening. There is a tremendous should have the properclinical attachments program. Such a setting designed to give it ahome effect. to the schooland be properly and examine theHome Economics When you go into amodern high school There is absolutely no Department, you see alovely apartment. into this kind of this kind of plancouldn't be built reason why I'd like to see It costs money;it costs a lotof money. center. and principals superintendents ofschools, chairmenof departments, for more money forthe schools asCharley Cogen fighting as hard Instead, they has fought and as someof the teachershave fought. for salaries. place theiremphasis on an index of the parentswho What I would liketo see is an amalgam rightfully so) sincetheir children arenot are sodisgruntled (and professional staff,marching to- learning, togetherwith the entire suffering from a very severesocial gether and indicatingthat we are fantastic sums of money. Now, I don't want disease which requires I think into auxiliary,after-school services. this money to go lots of money and you're wasting yourtime. Uncle Sam appropriates how to read betweenthree and it goes into tryingto teach the kid 8 or 9 years ofschooling between four in the afternoon,when for 7, This is very niceadded income for 9 and 3 hehasn't learned. would be disgruntledif they thought I was teachers.My membership educationally, it has taking their extraincome away fromthem; but, get money.fromUncle Sam, let's little value. If you're going to program that werequire and not put it into thekind of day school after-school activities, waste it in theseperipheral auxiliary, which look good on paper,but don't do thejob! around the Paul Denn: Thanks a lot. I'd like-to get How about Chicago,Los Angeles, country.We got as far asDetroit. Washington?

2014 Wayland Baldwin:I'd like to say something about Los Angeles.

Paul Denn: Would you? Not a speech--just some of your problems, or

Wayland Baldwin: Frequent references were made to this pro- gram as conducted in NewYork City. I expressed a personal opinion to the effect that I would like to see a MoreEffective Schools pro- gram for all boys and girls in LosAngeles and in New York. The pro- gram as conducted in New Yorkincludes less than 1 per cent of the students in disadvantaged areas.With our limited finances we cannot implement this type of program without jeopardizingthe educational program of the rest of the800,000 boys and girls.

Paul Denn: How about other areas of the country, Phila- delphia, Chicago?

Richard Sanders:I am. Dick Sanders from Chicago, and I have been listening with 'great interest for twodays. I see many of our Chicago problems appearingin other cities. May I get back to this business of teacher selection and how itrelates to what we have been talking about here.

Illinois law, at least as it relates to teacherselection in Chicago, is a little like Federal CivilService as reported here. The State Legislature has said, in veryfew words, that a board of three examiners shall examine all teacher candidatesin character, scholarship, and general fitness, and prepare eligiblelists from which appointments must be made in the orderof merit.

The statute ends there, so all the rest of the jobis ours. This is good in many respects because it gives us(the Board of Ex- aminers) a great deal of flexibility but it also meanswhat it says. We must provide merit lists and appointments mustbe made in the or- der of merit. We could not do as Al Schiff saysthey are, doing in Detroit, because the law would not permit us to do it that way.Our personnel department must appoint in the order ofmerit.

Perhaps a few figures would give an idea of ourproblems. We have a total educational staff of about 23,000. About 75 to 80 per cent of these are regularlyappointed teachers who have been ap- pointed after taking examinations and appearing on meritlists. This leaves us with about 6,000 positions this year that had tobe filled with teachers holding temporary certificates.To do this and to pro- vide qualified substitute teachers, we have issued sinceJuly first about 11,000 temporary certificates.We have an over-supply in many areas, and a shortage in a few areas. If it were not for the fact that we could use our surplus High School English and Historyteachers in our elementary schools, we would be in trouble. In order to

205 elementary schoolsand for a available teachersfor our provide more several hundredPro- of shortage, wehave issued few other areas have Bachelor's Certificates tocandidates who visional Temporary assignments whereneeded and These teachersmust accept Degrees. preparation. agree to pursuea programof professional is certification ofqualified teachers I do notthink that shortage areas except perhapsin some the real problemin Chicago, positions The real problemsareshortages of which existnationwide. shortages of willing to acceptassignments, and where teachers are deployment of ourstaff. incentives for anintelligent have talkedabout is verycomplicated and we This problem to gripswith think that manyof us have come it here, butI do not tried some, system basis.We have all this problem on atotal school We professionals intothe inner-city. little incentivesto lure that we of people, butit seems to me have triedbetter training of staffing frontal attack onthis business have nevermade a real schools in theinner-city. think in the ways little toopractical and Perhaps I am a mill, today that if youwork in a steel of industry.I said once You make to how close youare tothe heat. your salaryis related furnace or an open the superintendentof a blast the most money as the heat andthe least moneyfar away from hearth. You make the hazards. and may- saying that Ibelieve in**combat par Now, I am not pay forthe same time impossible in ourbusiness to give more be it is cities we couldgive more in Chicagoand perhapsin other We worked, but willing to workin theinner-city. annual pay topeople who are thirteen school school monthsof work and could guaranteethem twelve There are month (fourweeks) vacation. months of paywith one school some ofwhich do, as Mr.Cogen suggested, other thingsthat we could such as working climateand conditions would involveimproving the reducing classsize. believe ourmajor problem emphasize thatI do not I would certification, noris other largecities is in Chicagoand perhaps is in ourability teachers. The shortage it a criticalshortage of willingness to pay the hazards,and in our to reducethe heat and another, to of money necessary,in one way or the peoplethe amount that we needdone. get them todo the job floor, I findI want tomake another As long asI have the discussions. get muddied uphere in our distinction whichappears to legal separation cities, ingeneral, have a It seems to methat large meeting we tryto put the and placement,and at this of certification in the Cityof Chicago. Legally they arenot together two together.

206 the Board of Examinersmust examinecharacter, The law says that candidates to teachand scholarship, and generalfitness of all lists in the orderof merit. place successfulcandidates on eligible there. Appointment in theorder The Examiner'sfunction ends right of staff is apersonnel function. of merit andintelligent deployment city problems areunique, Sam Brownell wassaying that big with teacher selec- and I am surethey are, at leastin connection with him that in amarket of a tion and staffing. I would agree job is theelimination of poor great shortage ofteachers, our main have a surplus of On the other hand,in areas where we risks. of the best ac- teachers, our jobwill continue tobe the selecting cording to our. needs. selecting poten- It should be madeclear that Examiners are professionals.It is tial teachers andnot presentingfull fledged to make them realprofessionals the function ofsc.hool administr,.'ors probationary period orto decide thatthey have failed. during the is concerned with Much of what wehave been sayingat this meeting involving in-servicetraining this second levelof teacher selection and in-serviceevaluation. ought to have Paul Dena: I think thenation's capitol something to sayhere.

I think we have aproblem that isdif- Rufus Browning: Congress, have been describing. We work under ferent from those you and I Congress because lastweek it raised my pay and God bless our everybody's of euphoria. The teachers'salary plan and am a state single deputy, went upap- salary, except theSuperintendent and his Preedent Johnsonsigned proximately 9 percent and we neededit. that when he gotback week ago. The week before the salary bill a that is he signed ourbudget. You may think from Southeast Asia, that's sup- next year; it isnot! It is the budget the budget for we've been this past Julyfirst. Since July first, posed to start Now in the middle under the samebudget as the onefrom last year. the job of finding515 new teachers, of October andNovember we have Now that to usis a sizeable including counselorsand librarians. task. Negroes now con- Problem number two:Washington, D. C., This and the factthat stitute 91 per centof the studentbody. Rock Creek Parkmake it most of the whites arecongregated west of integration. As impossible situationto have classroom an almost lot of it, but integration isconcerned, we do have a far as staff dissatisfied with the integration is veryimperfect. I personally am personnel man, I havelittle con- the integrationof staff, but as a of our teachers areNegro as trol over assignments. Two out of three administrators andsupervisors.We're are twoout of three of our locating highlyqualified white having a great dealof difficulty in

207 through the individuals tobecome teachers. Our white people, either fact that theyare disappearingin our system or they dislikethe situation, arenow applyingfor supervisory positions,principalships, etc.

One of the problemswhich has been partiallysolved con- cerns ourprocedures for employing teachers. The Board of Education recently changed our teacheremployment rules, which makesit pos- sible for us to bettercompete with our blue-ribbon,silk stocking suburbs.We're surrounded by suchsuburbs and they have approxi- mately twice as many people as wehave in the city. As you know, the boundaries of our city neverexpanded. We have now changed ourrules so that we cantake advantage of what isperhaps our greatest at- traction, that is being inthe shadow of the Capitol dome.We prob- ably have more goodwalk-in candidates for teachingpositions than Washington, any city ofcomparable size.When the bureaucrats come to very often theirwives want to teach.The wives of officers, sci- entists, etc., also provide uswith a fine pool of talent. Under our old employment rules, which were verywonderful for the depression years, we weretoo slow to process teacherapplicants and the people went to Arlington County,Virginia, Montgomery County,Maryland, or somewhere else. We now have rules whichpermit us to compete with the suburbs and our salaryschedule is the best in the area.Of course, our salaryadvantage will only last forabout one year, be- cause thesuburbs are already acting tomeet our current teacher salary schedule.

Paul Denn: Well, thank you very much. You've been an aw-% fully good group; you'veparticipated, and I've been a poorchairman because you participatedall along and I let you runmuch too much overtime. Session No. 7:Needed Research in the Area of Teacher Selection

Chairman: Murray Rockowitz, Board of Examiners

John C. Flanagan, American Institutes for Researchin the Behavioral Sciences Harry B. Gilbert, Pennsylvania State University

Paul Denn: Again, now I begin to feel that our conference is a success because when I see various members caucusingbefore they come in to the plenary session, it looks as ifwe've had some effect on some of the thinking on the part of membersof the group. This gives me an opportunity to express the thanksof our Board and of the group as a whole to the powers that made itpossible for us to meet, the U.S. Office of Education, representedby Dr. Steiner. We'd like him to stand up. (Applause) Thank you very much. Dr. Steiner is the Associate Research Coordinator,in charge of Ele- mentary and Secondary Research in theU.S. Office of Education which has made this conference possible and which I trustwill make other conferences of a similar nature and on similar problemspossible in the future. I'd like also to express may deep regret that Dr. Bogen, the Chairman of our Board, was not able to be presentat these meet- ings. I think I told you when the conference firstbegan he suf- fered a very severe knee injury, a torn cartilage, Ibelieve, and had to undergo an operation. We understand that the operation was successful, and he's resting comfortably. I think he would have been much happier, much happier than I, I think, hereat the lectern, but I'm sorry that he could not attend.

I also want to state my regrets that,not realizing that I was going to be thrust inthis position, I made some other commit- ments and I plan to leave this morning.So I'd like to take this opportunity to thank all of you for cooperating sonipely and so ef- fectively and making this job that much easier for me.And while I'm in the mood to thank, I'd like to thank themembers of our Board and the research associates who did the planning,did all the prepa- ration, again who made this an easier task all around:Dr. Bogen, Dr. Gilbert, Dr. Perry Kalick and Dr. Gerhard Lang,and of course, Mrs. Louise Klein.

Finally, I'd like to thank all my colleagues on theBoard of Examiners, each of whom had a specific task to carryout in the course of the conference, chairingvarious panels. Much as I dis- like to say it, there are a considerable number ofbenefits that arise from a war crisis. Research, for example, is carried out under pressures of war much moreeffectively, much more rapidly than in or- dinary peace times, and the finances for such researchmiraculously

209 become available.Most of this research weknow is in the biological of World War and physical sciences. But I recall that in the course II there was a considerableamount of research in thedevelopment of personnel instruments. The assessment of personnel,for example, was an importantfactor, the work of the OSS, andof various war agencies in England and elsewhere. Perhaps with the war on poverty some more worthwhile needed research,particularly in the area of education where we know it is badly needed,will result. And this morning's panel is devoted to theproblem of where to place that research. I'd like to turn the meeting over atthis time to my colleague, Dr. Murray Rockowitz.

Murray Rockowitz: Thank you, Paul. For me this conference has been an intense personalexperience. As the junior memberof the Board of Examiners, I've seen alot of names become people, andin the research field alone, forexample, I used to read thestudies of David Ryans. Until now, he was a name and lastnight he became a living being for me, and thelegendary figures like Brownell and others have become part of anenvironment of which I am a partand, therefore, I can say that this hasbeen for me a very exciting ex- perience--that of sharing the thinkingof these very considerable individuals.

If we were to look at thisentire conference as a develop- mental lesson, we are in theposition of giving out homeworkassign- ments this morning.We're going to tell you and I say we,the panel- ists principally, are going totell you whatyou have yet to doin terms of further study in orderto effect the objectives that weall came here todiscuss.

Dr. Schiff was much surprisedby the budget of the New York City Board of Examiners-1-1/2million dollars per annum. And yet, we have herewith us the two research associatesthat the Board has had in the last few years, and we nevercould get them together at one time becauseof budgetary limitations. The maximum research staff that the Board had availableto it to evaluate this1-1/2 million dollar yearly operation was oneresearch associate. And even as good as each wasindividually, each year I thinkwe'd agree that this allowance was inadequate. A few brief words withregard to the functions of research.

Last night I just listened to anumber of statements and I heard all kinds of assumptionsbeing made. For example, an assump- tion was made that a school couldnot be staffed because ofthe pe- culiar nature of the principal,alone. Another assumption that was made was that empathybetween teacher and student is moreeasily at- tained if there's a closematch--social class of teacher withsocial class of student, race ofteacher with race of student - -we can go on with religion,national origin, etc. These are the types of as- sumptions that have to be tested inthe crucible of research.

210 I VAS told that it's nice toinject some levity into the proceedings. I can advert to Oscar Wilde's classificationof lies. He says there are ordinary lies, white lies,and statistics.Re- search can be perverted, and it has been. Hitler had his research staff, and Garrett is considered a research person too. Research has been perverted to national interests so thatthe Soviets dis- covered baseball, but what we're talking about now isthe type of research which seeks the answers to persistentquestions. I think that what came out of yesterday's discussions was ageneral feeling that there's a great deal that remains to be done. Dr. Bray indi- cated that we hadn't even gotten to the problemof establishing criteria, and Dr. Ryans stressed that the criterionproblem was still primary in his thoughts after afew decades of work in the field.

This morning, we are very privileged to havewith us two gentlemen who have extensive backgrounds inresearch. Dr. John C. Flanagan fills in the gap which was left by Dr.Bray yesterday morn- ing. He's a Harvard man and he received hisdoctorate from Harvard in mental measurement and went on toextensive experience with the cooperative testing service of the American Councilof Education. He did memorable work in the Air Forcein selection of personnel and for the past 20-odd years he has been connectedwith the American Institutes for Research which hefounded and of which he is the director. His most extensive current project, the onewhich is be- ing watched with considerable interest, is,of course, Project Talent. I feel very privileged to present to you now Dr.John Flanagan.

John Flanagan: I have been very interested in getting back in touch with the field of teacherselection after many years. In 1939, at the suggestion of Ben Wood, I wrote aproposal to get the National Teacher Examinations funded by theCarnegie Corporation. Dr. David G. Ryans was selected to undertakethe direction of this program. Since then some progress has been made but asDave and the other .people who are working most intensively onthis problem indi- cated yesterday, they have hardly scratchedthe surface in the past 25 years and it is certainly time for a real majoreffort. I'm very encouraged about the likelihood that a majoreffort will be made on the basis of what I heard yesterday, and it seemsto me there are problems. We have difficulties, but they donot represent insur- mountable problems.

I can recall a group of researcherssitting at lunch in the Pentagon and saying, "What we oughtto do now, right in the middle of World War II, is to bring in an applicant groupwithin which we don't do any selection and see what happens. We'd find out whether our pro- cedures are valid. Nobody will believe now that we're onlyselecting the top-scoring applicants for training,that the others would do as poorly as we say. They'll say we don't really know what the other

211 we're losing alot of pilotsbecause people wouldhave done, and start training."We all agreed you're eliminatingthem before they let us do thisbut we really that headquartersprobably wouldn't suggested that oneof the peoplewould write should try. So it was This indi- and see if wecould get approvalfor it. such a proposal two later hehadn't such low prioritythat a week or vidual gave it of time to prepare gotten to it. He said hefelt it was a waste So I sat downand started towrite the proposal, such a proposal. By the time I I wrote the morereasonable it seemed. and the more sensible they wouldn'tturn it got through, Ithought it sounded so down, and theydidn't. All the applicant They let us bringin 1,000 applicants. Aviation Cadet,"and he became an had to say was,"I want to be an matter what hescored on thescreening tests. Aviation Cadet, no regardless of to be pilots, sothey became pilots They all wanted This gave us areal pic- their scores onthe classificationbattery. procedures were. ture of justhow effective our analagous to the one youhave at A situationwhich is more the war. It became quitedifficult the present timedeveloped after af- cadets and thepersonnel peoplesaid, "We can't to find aviation did before; we and classifythese people the way we ford to screen with the sugges- take what we canget."So we came up just have to of the otherwise to give the testsbut accept all tion to continue working in to determinehow the procedures were qualified applicants that tests may pro- The statement wasalready being made peace time. it's a differ- screening device,but in peace time vide an effective can't generalize. The training isdifferent, and you ent situation. experimental group. and again wehad a very good This was accepted taken in as experimental becauseeverybody was being The group was of some of theschool systems wasdescribed in therecent experience fine opportunityto again findthat yesterday. Thus, we had a very used were veryeffective. the proceduresthat were being developed usingresearch methods. These procedures were that a good to the discussionyesterday, it appears After listening I'd like to run over, is needed inthis field and deal of research think ought to be the kinds ofresearch that I under four headings, that I have are: few years. The four headings done in the next behaviors which de- of the patternsof teacher first, determination the effectiveness teaching; second,the evaluation of fine effective of teachingeffective- performance; third,the prediction of teaching teachers to leaveteaching. fourth, factorswhich cause good ness; and although it'snegative This seems to meto be a partof selection, selection. behaviors these is research ondetermining the The first of this be given a effective teaching,and I propose which constitute characteristics of this neworientation neworientation. The principal

212 are that insteadof a single set of specifications for the require- ments of an effective teacher, there shouldbe a very large number of seta of requirements defining the behaviorswhich make for effec- tive performance of each of the various aspects ofthe teaching task. Although such an analytical approach has advantages, itbecomes of great value in a situation in which the activitiesof the teacher are likely tochange markedly with the advent of new teaching meth- ods, new objectives, new technology, and the provisionof clerical assistance by computers and teaching aids.

Effective teaching in the past has involved manyactivities such as lecturing, maintaining discipline,leading discussions, checking on students' learning, correcting students' errors,stimu- lating students' thinking and arousing students'interests. Although many of these activities arelikely to be important for effective teaching in the schools of the future, it seems probablethat there will be considerable change in emphasis. More importance will be given such activities as individual tutoring,assisting students to plan their educational program, developing a senseof responsibility in the student for his own educational development,helping him to discover his interests and special abilities, helping him todiscover the satisfactions that can be obtained from creativeactivities and assisting him generally in his personal and socialdevelopment.

Since it's unlikely that most of us will be able topredict the precise nature of the teacher's role in activitiesin the school of the future, it is of considerable importance that we carryout re- search to define the requirements for effective teaching of many kinds. It is also important that in gathering this information,the characteristics of the individual student involved in theteaching activity be clearly specified. It seems likely that the require- ments for motivating one type of student arelikely to be different from those for motivating another.

In carrying out research in this field, it isproposed that task analysis procedures and the criticalincident technique be used. These can be supplemented by other observational analysisprocedures. A device which has alreadyshown its value in this type of research is a video cane ra which can capture the teachingsituation and play it back for observation and analysis as manytimes as is required. The procedures of task analysis are likelyto be of special value in developing '2equirements for new types of teacheractivities. Of course, in using this procedurethe characteristics of the student, his previous knowledge and many otherfactors can be specified and varied at will.

The critical incident technique alsomakes possible the de- tailed description of the specificstudent and the definition of the particular aim of the teacher's activities. It should be of great

213 eacL of which is clas- value to collect manythousands of incidents, teacher sified according tothe type of studentand the specific activity involved.

Before we leave thismatter of teacherrequirements and observation mentioned in the study of teacherrequirements involving perhaps the some ofyesterday's presentations,I should say that will be fairly inadequate. I study of teaching asit's going on now carrying out a study for a recall an instancewhich occurred in careful study us- corporation of third-levelmanagers. We did a very watch and record what ing task analysisprocedures; we had somebody and for a whole day. they were doing everyminute, every 5 minutes position, we obtainedseveral Since a number ofpeople were in this good pic- thousand critical incidents. We gave this company a very This was to have been ture of just whattheir people weredoing. at our report and saw the prelude for selection,but when they looked they said, "Oh not We what their people weredoing with their time, As our first job, don't want any peopleselected to do that job. we're going to changethe nature of thisjob." circumstances if we wentin We might find asimilar set of five minutes by us- and found out what ourteachers are doing every "We don't want to se- ing a time sampling.We might very well say, I'm suggesting lect teachers forthat job; let's changethe job." in the job, then we canchange the we getrequirements for each task emphasis without majorchange.

The second area ofneed in research isthe evaluation of also the the effectiveness of ateacher. Of course, in this case in each of a wide focus would be on theeffectiveness of the teacher of types of students. variety of activitieswith each of a number in terms of the process Teacher effectiveness maybe evaluated either procedures are developedwhich measure or theproduct, as assessment The computers pro- all of the desiredchanges in studentbehavior. procedures for these data;it vide efficient storageand analytical records can be collectedand seems likelythat much more detailed analyzed than has beencharacteristic of pastevaluation procedures. educational developmentprocedures It appears thatif individualized each student, the are used,we'd set specificattainable tasks for student attain the goal details of whether aparticular teacher or specified will provide amuch more preciserecord of teacher effec- tiveness than has beenpreviously available. High priority should procedures for a com- be given to researchand development work on effectiveness in many activi- prehensive evaluation ofthe teachers' systematic measurement of ties with many typesof students based on the effectivenessof the instruction. research on process It also seemsdesirable to continue the most promisingline variables in teaching.It is suggested that systematically developed of research on processvariables involves

2l1. observational record forms on which specific observations by a com- petent judge are made of effective and ineffective performanceon the part of a teacher. These, of course, should be related to the particular activity and the particular type of student involved.

A factor which could have considerable influence on the quality o: teachers is the use of merit increases and the assignment of tenure status only as a result of demonstrated effectiveness. This was mentioned yesterday. The lock step promotions characteris- tic of many school systems and the failure to eliminate the ineffec- tive teachers produce a situation attractive only to the mediocre and marginal teachers. Of course, improvement in these areas must be based on valid procedures for the evaluation of teachers.Re- search on the evaluation of teacher effectiveness should be given much attention for another reason which is its essential importance to other types of research on teacher selection.

I should perhaps underline at this point the difficulty of product evaluation of teacher effectiveness. In many studies over quite a few decades now it has been shown that within the variation in quality of teachers that we find in the United States, the dif- ference in the amount the students learn as a result of being taught by one of the better teachers rather than by one of the poor teachers is relatively slight. It is only a few per cent.

When we're talking about the problem of what effect the teacher has, we have a situation in which in the total variability in students' performance only a small proportion of it is under the control of the teacher and can be attributed to variation in teach- ers' effectiveness. From one point of view this can be regarded as a good thing, since even if we don't have very good teachers the students will learn anyway. Although this may be the case in many instances, it does represent and provide problems when we are trying to do research in this area.

The third area of research relates to the development of predictors of effective teaching. It appears that long range follow- up studies are among the most important approaches to this problem. It is anticipated that data from students tested as twelfth graders in Project Talent who have graduated from college and entered teach- ing will become available this fall. Similarly in the next three years data from other classes will be collected and analyzed. These data should be of distinct value in defining the type of individual who enters a teaching career. Of even greater value will be the ten- year follow-up plan for Project Talent which will begin in 1970. These studies will provide data on those who have, insofar as we can measure it, not only selected careers in teaching but have been found to be effective teachers over a substantial period of service.

215 of research studies men- As in the caseof the other types predictors not be focused onthe tioned, it seemsimportant that the effectiveness ofthe quality of teachers ingeneral but rather It is also be- teacher in performingspecific roles andactivities. respect to predictorswill be lieved that the trendin research with predictors rather of a very largenumber of specific in the direction Medley and Mitzel than a few moregeneral measures. Drs. Bray, Baker, supplementing pencil-and-paperaptitude all emphasizedthe need for information on theteacher. I types of tests withother types of I thought someof might say that as anold line measurement person supplementing thesepredictors. In them overemphasizedthe need for found that pencil-and-paper several recent AIRstudies it has been increased by situationalperformance tests and validities could be situ- of observation ofperformance in a realistic by different kinds ob- I'm sure that aprocedure which doesnot include such ation. situations is servation of behaviorin relativelyrealistic teaching which does have testedand vali- going to be lessefficient than one dated measures ofthis type. to the conditions The last area ofneeded research relates negative selec- special emphasis onthose which have of teaching with teaching is very It is well knownthat the turnover in tive value. why teaching loses someof high, and it seemsimportant to determine One of the problemscalling for re- the most effectiveindividuals. family duties onremaining in search in this areais the effects of question of whether It would seemworthwhile to study the teaching. full-time employment part-time employment orsomething less than might retain someof the better womenteachers. relate to the studyof Other researchtopics in this area with a view todetermining specific teacheractivities particularly administrative dutiescould whether or not someof the clerical and computers. be performed byteacher aides, clerks, or directed at the questionof Another valuablestudy might be teachers leave teachingearly in their ca- why many promising young practice of assigning It seems likelythat the administrative reers. with the more un- to difficult classes,loading them down new teachers procedures which give pleasant extracurricularassignments and similar impression of teaching areresponsible the new teacher abad initial for losing effectiveteachers. of teach- it is believedthat the new methods In conclusion, much more possible to evaluateteaching effectiveness ing will make it This will make re- precisely than hasbeen the case inrecent years. in specific search on thecharacteristics foreffective performance As proposalsfor new edu- teaching tasks bothfeasible and important. be accompanied byplans cational programs aredeveloped, they should activities in the new program. for research on theteacher's role and research in teaching isthe Perhaps one of thegreatest needs for

216

r - application of systems concepts to the teachingand learning system. This application would include comprehensive measuresof the in-put and out-put of the system as well as study the processvariables. For such studies to be effective, much better measuresof the in-put and out-put of the system in terms of assessingthe student's be- havior concepts and abilities will be required than are nowavailable. Perhaps the first step in a program of teacher selectionshould be the definition and development of measures of the products ofteach- ing. Unless we have valid measures of the effectivenessof a teach- er's efforts, other types of research must be regarded astentative, exploratory and perhaps in many cases trivial.

Murray Rockowitz: Thank you Dr. Flanagan, that was a very large order and a quite complex homework assignmentfor us, but I think we welcomed your considered remarks.

Our next panelist is a gentleman whose veryconsiderable shoes I'm trying to fill. I inherited Harry Gilbert's responsibili- ties on the Board of Examiners, and aside from a deep personaldebt to him which I publicly acknowledge, I am finding out thatthe job he did, the image he created and just abodt everything thathe as- sociated with the job has created a kind of halo effect for me. He wears two hats at the present time:he's still a member of the Board of r-liminers but he's also Assistant Dean for Resident Instructionat the College of Education of the Pennsylvania State University.In New York City, he served as a teacher of children inclasses for children with retarded mental development, he was a school psycholo- gist and a supervisor of school psychologists beforebecoming an examiner.With typical tongue in cheek, he likes to refer tohimself as a clinical psychologist insheepskin clothes. Dr. Gilbert will now lead us further along the pathof needed research in teacher selection.

Harry Gilbert: There are three basic assumptions that under- line the specific proposals I shall outline. Permit me to make them explicit. One, interest in the area of teacher selection is minimal, based upon the actual amount of research under way. However, a great deal of interest does exist among teacher personnel selectorsand uni- versities. The problem is to make patent what's latent.Two, pro- fessional teacher selection practices are rarely employed. In large school systems that are presumed to be using selectiontechniques, screening is actually done. In smaller, affluent school districts, hunch rejections and global perusals sometimes in actual observations, serve as selection techniques.A moment's reflection on the current shortage of teachers particularly in those school districtsperceived to be tough will confirm the probability that this assumption is warranted.

I want to digress from my own paper for a minute toindi- cate that I think that screening is a highly important process. It

217 VIA mentionedhere earlier, and I'm particularlyindebted to Medley for the use of the achievementtest concept vs. the aptitude test concept. There is no need to be bashful or abashedabout our using this concept of expecting a certaindegree of achievement for pro- spective teachers. This brought to mind a very recentexperience. I VW walking through aschool in a city I shall not nameand it hap- pened to be on a day when there was acity-wide testing program; one of the teachers had put a sign onthe door, "Testing, do not distrub." As I walked throughwith the principal, I laughed aboutit and made some comment to theprincipal that this is one of theoccupational hazards, when you teach childrenwith reversals, etc., and hesaid, "No, this teacher can't get aregular license; she fails thetest for written English." It isn't funny, and those of uswho use tests of written English recognize that afair number of college graduates cannot demonstrate that they canwrite acceptable English. So I'm not at all disturbed aboutthis achievement test model, Ithink it's a goad one. We can probably refer to it ascontent validity if we wish, but nobody is going to take awaythe responsibility that se- lectors will have for making a judgmentabout some expected achieve- ment a prospective teacherwill have to be able todemonstrate.

Now the third of my assumptionsis this: Since the field of teacher selection is a greatbig area, up for grabs, it ismost desirable that research be encouragedin varieties of approaches, without too much specificity. It follows, of course, thatwide dis- semination of research be encouraged,and that investigators be supported with the notion thathypotheses may be rejected as well as verified by experimental data.Regrettably, this simple dictum, readily understood by researchers, seemsto be a heretic notion in an age wheninnovations are publicized as successesbefore evaluation.

Again on this notion of encouraging avariety of approaches, John Flanagan's being here reminds methat he headed the Air Force Selection Program, and I workedin the Naval Aviation SelectionPro- gram (John Jenkinsheaded my group). Here were two very competent psychologists who started out with verysimilar problems and came up with very dissimilar answers.

The Air Force approach included a1 , e number of psycho- motor tests. I went through one of theirtesting stations at San Antonio and it was like a NewYorker's conception of a psychologist The gone wild. There were all kinds of intricatepsychomotor tests. Navy, by way of contrast, came upwith a fairly short series of paper and pencil tests, abiographical inventory, a shortintelligence test, Bennett's test of mechanicalcomprehension. And yet when you looked at the results of theirvalidity studies, I can see John'sstanines now and the relativelysmaller degree of failures intraining as you go up the stanineladder. A very impressive exampleof the use of their tests, but the Navy came upwith the very same thing, and the

218 results were almost the same as you went upthe ABCD categories that were used in the Navy.I'm commenting here that starting with the very same approach, the sameproblem, two utterly different approaches resulted, andthey gave about the same results asfar as I could see.

Now for the proposals. One, attention must be paid to the process of stimulating attention tothe methodology of teacher selec- tion and to research, which must be inevitably anassessment of the validity of presently employed procedures and those yetto be tried or designed. Surely we can agree that fundamental research onteacher behavior and on pupil-teacher interaction as an outconeof the teacher stimulus must be encouraged.What is needed is understanding of why so important afieldfor research continues to be neglected, despite promising starts by such as Withall, Mitzel and Ryans.The simple answer, too difficult, seems notenough in an age when complex prob- lems are tackled with the resources made availableby contemporary cash flow. I suggest, therefore, that social psychologists andsoci- ologists be invited to investigate broadly the field of researchin teacher selection for their aid in understanding avoidanceattitudes of potential research workers.There is no disposition to hide the obvious, that I have made a value judgment and that the purposewould be to be able to persuade research talent into actionin this sphere. I propose, too, that we evaluate the effectivenessof the design of this conference with its influential participants as a meansof stir- ring interest and action in teacher selectionpractice and research.

Two, we should take advantage of the currentsupply and de- mand in the area of teacher selection. Specifically, I refer to the shortage of teachers as the country begins to wake upto the great big need for teachers and pumps green blood into localschool systems from the great big Federal artery. Teacher shortage gives the person- nel research man a rare opportunity.He can lower his selection cut- off scores to provide a greater range for assessmentof predicted validity.It would take some degree of courage to dothis, although some reflection on the lackof predictive validity data should be en- couragement enough. Now this is a point that Mitzel made earlier. At the same time, there is always anover-supply of applicants for supervisory or administrative positions ineducation. This should serve to be a sourceof comfort for those who would argue that one cannot be selective or try to be, withteachers when we need all who apply. Furthermore, it is palatable to applicants to befed a variety of selection techniques when there are moreapplicants than positions to be filled. This presents the opportunity to employ variedand mul- tiple devices toward a socially accepting, evenwilling group. Spe- cifically, I'm suggesting that we ought tothrow a good deal of energy into the selection of supervisors where wedo always have an over- supply.

Three, I now come to some specificproposals. FirstI make reference to the sixteensuggestions for further research on

219 pages 55 to 57 of "Teacher Selection Policies and Prod- Aires in Large School Systems in the United States." That is the first study that we completed in this series. There should be enough ideas here to generate several bushels of Ph.D.'s or Ed.D.'s.I should like to take the time to express myself on the perennial dissuader, the lack of sound criteria. The fact is that sound criteria exist only in relatively simple occupations and not in complex professions.Worker output in terms of quantity or quality or time or a combination there- of can be employed as criteria for production line employees, but everybody knows how hard it is to define "success in teaching," par- ticularly since this amorphous generalization, bad to begin with, keeps shifting in different school settings at different levels and with different subjects. At least we have learned from previous study that the overall general estimate of "success in teaching" is a concept to be discarded.We can work with dimensions of teacher performance as the local hiring school systems define them. This is important; these dimensions are going to be defined differently by people who hire them, but I think Harold Mitzel made this point earlier. These dimensions may be with respect to teacher behavior such as "evocative of pupil participationlff "encouraging pupil-pupil interaction," "accepting of deviant behavior," etc.They may be with respect to the ultimate in expectancy, namely, pupil development in skills, knowledge or attitudes. Let's not beat a dead horse. We can all agree that research is necessary to refine and to define the dimensions and to determine methods objectively or reasonably objec- tively to assess them. I propose that we accept crude estimates of the extremes in dimensions and not wait for the millennium.I pro- pose that we engage task forces of teachers, teacher educators, school administrators and personnel experts to agree on working definitions of dimensions of teacher behavior on a scale from most de- sirable to least desirable. Then obtain nominations for extremes on these scales, the best and the worst. Various procedures can be em- ployed for these nominations. They can be peer nominating techniques, observer ratings, supervisor ratings, pupil nominations and measured pupil performance. Hopefully, a variety of techniques would be em- ployed as Doug Bray suggested earlier. I am arguing in short for our getting started with best available techniques to obtain criteria while others among us continue more basic research.

In another setting at the A.P.A. annual meeting in Septem- ber, I also urged that we utilize the best informed personnel minds to review present selection procedures. While I would hardly clas- sify such activity as research, I do maintain that some improvement in present procedures can ensue without waiting for results of years of study. In short, I am urging a redirection of energy to the problem of teacher selection, research and procedure, in a reasonable and practical way without sacrificing the on-going need for basic research.

220 Finally, I would like to proposethe need for establishing I a clearinghouse for research and practicein teacher selection. don't know who would establishthis, I believe the fieldis important enough that some universityinterested in developing thisbranch of educational administration mightbe encouraged to assume suchleader- ship. Certainly a repository of findingswith dissemination facili- ties would be a major resourcein the development of interestin the field.

Discussion

Murray Rockowitz:As is frequently the case,Dr. Gilbert is heard as the honest brokerto bring together thechallenge hurled by Drs. Bray, Ryans,Mitzel, and Medley. In any case, I think tLat we're at a point now wherequestions can be fired at the panelfrom you to clarify anything that they have said thismorning.

David Darland: Aren't we really talkingabout applied re- search? Would it be feasiblefor some university, orcombination of universities, to define thisproblem in all of its ramifications so that the research would bedirected toward a team kindof an effort to get away from ourtendency to be so piecemeal?

Paul Denn: As I get it, Dr.Darland's proposal is not to have the clearing houseof results of research,but also a clearing house prior to research.

David Darland:I'm suggesting an approachfor getting at a problem of massive dimensionswith appropriate means.

Murray Rockowitz: Any comment on that?

John Flanagan: My- experience with havingsort of a committee approach has not been verygood. Researchers typically just don't I think want to do somebodyelse's research that theyhave defined. field ten years by that in certain instancespeople have set back the outlining all the research andspelling out the research designsfor all the projects that neededto be done in that fieldand, of course, this immediately then tookall of those out of the realmof possi- bility for any self-respectingindependent researcher, becausehe wouldn't do a study that hadalready been designed and printedin a book by another psychologist orresearcher. So that I don't know, I think that theoretically itsounds fine, actually I'm not so sure that that's the way to getthings done. I guess that the particular formula which I have found veryeffective is that instead ofpiece- meal projects, you have a programand you could have a programand perhaps that's what you meant. It's not sort of a committeewhere we all get together, but youhave to have a director and youhave to have somebody planning the program, ora groupplanning the program, and then make assignmentsof things to do.

221

1* David Darland: For the record, by God,I didn't use the word committee.

John Flanagan: All right. Well, I think there's no ques- tion but what one needslarge scale, long range,concerted attacks on things and I'm certainly onthe record for it. A. T. and T. would Standard fail if they didn't dothat. That is correct, and so would good example where in arecent Oil. Yes, I think that that's a very study we did for A. T. andT., we probably pushed thefield ahead about twenty years abovowhere it would have normallybeen by just their giving us enough moneyto do a good study.

Murray Rockowitz: Are there any otherquestions? Yes, Dr. Mitzel.

Harold Mitzel: I would like to echowhat John said about publishing lists of neededresearch. You know about ten years ago, Education put out a the American Associationof College Teachers of little green book which wascalled "Needed Researchin Teacher Educa- tion," and as far as I know,not one of the hundredsof ideas or sug- gestions in that little book was everpicked up.

David Darland: I want to again insistthat I was not sug- gesting publishing lists;I was suggesting quiteanother technique. I just want to be clearhere about what I wassuggesting.

David Ryans: I just want to respondtoDave here, just for university or the heck of it you know,that he did say couldn't some some group dothis, and I think that'sthe thing that Dr. Flanagan much oppose, any one and Harold Mitzel andthat I, too, would very No one of us, group, becausethere are lots of differentapproaches. I hope, is God andknows all the answers ofhow this should be done and there's lots of roomfor different programs but,if you mean pro- gramatic, Dave, fine; if you mean someone group say nowhere's the scheme of all that shouldbe done in this area,then I don't believe it. advantage of the op- Samuel Brownell: I would like to take portunity to suggest somethings that weren't saidthis morning in As reference to looking aheadin this field ofteacher selection. that I've listened and recognized someof the traumatic experiences to do a we wentthrough when I was inDetroit and we were trying change that is tak- better job of teacherselection, I recognize. the ahead of us. ing place very rapidlyand the changes that are

In this vein I'd liketo comment on a coupleof things. One is to look atthis problem from thestandpoint of the person who goes through it.We talked pretty muchabout the students and the people who were doingthe examining business. I think we have to recognize that people who come upto the point where theyreach the

222 Board of Examiners or theteacher selection process haveput in a very sizeableinvestment over a period of years.Most of them have been looking forward toreaching that point.We may well consider what is their attitude andwhat's the effect on them ofgetting the job or not getting the job. That's a pretty important decisionthat's made from the standpoint ofthe human beings that areaffected, and I think we can't overlook that. I suspect that theirexperience in go- ing through the selection processhas a lot to do with theattitude toward the school system based uponwhether they are selected or whether they are rejected. It has a lot to do, too, withthe atti- tude of a great many otherpeople who are their friendsand relatives as to theiropinion in regard to the schoolsystem, and particularly in regard to the examining process.I'm afraid there's been a lotof systems on the negative attitude developedtowards schools and school part of those who have hadunfortunate. experiences as a resultof what has happened to themeand they may have come torespect teaching that less because of theirexperiences.I think we ought to realize this is an area which has tobe taken into account.

A second thing that Iwanted to comment on is citizen ex- pectation of teachers as I seeit in our large cityschools. I think we've centered in ourconsideration on certain of theaspects of last teaching. I happened to be unableto be here the night before because I was with a groupof citizens who were talkingabout prob- lems of the city and what canbe done in the function ofschools and other agencies toward makingcities better places for peopleto live. The concern of the citizenswith whom I was meeting wasthat in our approach to urban redevelopment wesometimes forget that cities are to for people! The question was raisedby this group of citizens as whether or not, in view ofthe complexity of cities, we aregoing to be able to make cities work asplaces for people to live andto bring up families.

Well, obviously cities haveto work, and what's the roleof the teacher going to be inthis? Is there any way that we canin our teacher selection procedures aidin selecting those people whobelieve in people and their worth,their dignity, and the role ofthe school in making cities betterplaces for people to live?I'm not at all sure but what we mayfind in the next decade thatwe'll have more citizen pressure on us tohave the schools contribute moreto the human aspect of education andliving, than perhaps on someof the concerns we haveabout reading and writing.I think we ought to be looking to see whether or notthere is anything we can doin our teacher selection to encourage moreof those people who haveatti- tudes of responsibility forpupils as important, developingindividu- als, and discourage peoplegetting into teaching who maybe tech- nicians, but who themselves arefleeing from their responsibilities of making the city a goodplace to live.

223 comment on two other thingsthat I wanted to Now, one or is the have to work onin teacherselection. What that I think we organizations and ofthe organizedteach- appropriate roleof teacher procedure?I have a verystrong ing professionin this selection teacher decade we haveto re-examine our feeling thatwithin the next of as towhat the moreactive participation selection procedures school opera- school policymaking and in teacherorganizations in profession and what the people who arein the teaching tions means to school systemthat isn't ex- their role shallbe. I think that any find it- of this questionis going to amining itsprocedures in light should be doing point. I think we self in a crisissituation on this somethinking onthat. we must that in thisteacher selection area I think, too, selecting attention to theappropriateprocedures for give much more every aides in ourschools. Unless I'm wrong, the paraprofessional spending about asmuch time and energy school systemis going to be schools as paraprofessionals orthe aides in our on theselecting of surprised to seethat we I wouldn'tbe at all they are onteachers. professional teachers people in theschool who are not have as many We'd better gear ourteacher se- have peoplewho are teachers. as we of these otherpeople so that lection proceduresfor the selection and so that we candiscriminate be- they can beutilized effectively valid those who shouldnot be employed on tween thosewho should and bases where wehave to makediscrimination. has to do withthe which hasn'tbeen mentioned Another area unsuccessful in a the teacherwho is evaluated as rehabilitation of select people andthen we put particular school orassignment. We well in They don'talways make out them into a largeschool system. throw them second job. We can't just their first job ormaybe their and by too muchinvested in them into the discard. There has been them. wondering if this groupisn't theappropriate Finally, I'm teacher selection. consider thematter ofreciprocity in group to the carefulselection procedure Why should a personwho goes through have to husband move outto Los Angeles, in Detroit andthen has her who has not of selection processas a person go throughthe same kind screening for acity schoolteaching position? gonethrough a careful through a care- good or ifthe person goes If the processin Detroit is other city repeat in Los Angeles,why should the ful selection process in terms oftrying to findout The comingtogether 'of this group find that it? it seems to methat you might who weresuccessful teachers, be able to considering. One, you might there are twothings worth once, having gonethrough that process find a lot moreteachers who, than would rather stayout of teaching when they movesomeplace else And secondly, Ithink you might to go throughthe process again. all of us now work just aswell because find out thatthe procedures

2214 accept that the children whocome from one school system andgo to another adjust pretty well. There are some mistakes, butI believe this area of reciprocity is one that this group mightvery well consider.

Now, what I tried to do thenis simply to mentiona few of the things that I haven'theard as things thisgroup might very well do that I thought mightbe put in the basket.

Murray Rockowitz:And we welcome them. Yes, there are going to be many comments.

John Flanagan: I'd like to commenton just one or two of these. I think that it'svery obvious that the decision should be made early, and we've foundin our Project Talentstudy that the guid- ance procedures provided students intoday's schoolsare just not ade- quate to prepare Students tomake good decisions in theelementary and secondary schools. Of course, the later thedecision is made bya person to prepare for a career in teaching, the more unlikely itis that this person willget the best training for it. In our work at the American Institutes forResearch at the present timewe are try- ing to developa set of guidance procedures basedon the Project Talent findings which willidentify the prospectiveteacher at an early age, will help him identifyhimself as this kind ofperson, and will enable him to plan hiseducational development. If our guidance proceduresare correct, these people will haveno difficulty in obtaining positions in schoolsystems. We need to help people make the right decisions early. If you bring collegegraduates into your system and then find outone, two or five years later thatthey don't belong, this is muchworse for them and theircareers than having kept them out in the firstplace.

Harry Gilbert:I'd like to makesome comments. I'm very glad that Dr. Brownellbrought in the wholearea of the social climate in which we are existing. Frankly, it was our failure to getsocial psychologists in here that ledus tothe conclusion thatwe had better limit our discussions. I think probably the mostimportant thing we would do would be toagree that we have to examine the socialclimate which says that the teacher isthe most importantperson in society, we entrust our children to him. But the same people whosay that are not willing to supporta teacher in a style to which he shouldbe ac- customed. The fact is that in almostany community there is resistance to raising teachers' salaries,providing the professionalmeans of be- ing a teacher in the realsense of the word. It may well be,as you've pointed out, that in thelong run, it's the strength ofthe teacher organizations that will providethe professionalization and setre- quirements and demands on society that will be the mostsignificant thing in this whole field.

225 I wouldlike to takeexception On a lessimportant level, that because I feelrather strongly with the notionof reciprocity define the taskfor community thatwill have to it will bethe local selection If that isthe case, its the teacher inthat community. At the same geared to itsexpectancies. procedures willhave to be in any oneof through theselection procedures time, anybodywho goes to get through a to be able,with confidence, the largecities ought is, if wemake it sotime- elsewhere. The problem selection procedure ordeal. I don't knowof complex, thatit is in fact an consuming or will acceptthe selection organizationanywhere that anyemploying agencys orwhat have you,before procedures fromanother company or for its with respectto hiringthe person it makes itsfinal decision own agency. But that'sminor. problem that Dr.Brownell hasunearthed I thinkthe real hasn't beea There is oneother lastthing that disturbs allof us. to do withthe bring up too,and that has said here thatI'd like to selection proce- to examine yourown tendency generallynot to want readily and accept theinterview so dures. Why isit that people back around1955 what they aredoing?Lowell Kelly, rarely question conventions, took astab at thepsycho- or so at oneof the ETS that this is a He suggested,and I think dynamics ofinterviewing. and makesde that when maninterviews man veryworthwhile notion, he shouldget a job or this fellow man asto whether cisions about to be askingof himself. He, get a job,this is a lot should not notion thathe's not capable therefore, rejects asunacceptable the important factorin thepsychodynamics and this is avery unnoticed, to do this has been pushedaside or interviewer. It too often training of the selection ofpeople and in our but it issignificant in our area for Again, that'sanother untouched of them forthese jobs. research. to com- Harry. I just wanted Murray Rockowitz: Thank you, point that Dr.Brownell made. one point,the second ment briefly on representativeshere will Dr. Gold andthe university preparation from I think that movement awayin teacher admit thatthere has been a and principles in history,and philosophy the standardcanned courses sciences greater emphasison thebehavioral of educationto more and Would thatbe sociological impactoneducation. and thestudy of the Pretty valid, teacher preparationcourses? true, Dr.Gold, in the yes. subjects forresearch is a that one ofthe possible I think SuperintendentBrown indi- the teachers areunwilling, as study of why to do theirteaching. to move tothe inner-city inner-city ex- cated yesterday, keep them awayfrom this What are themotivations that that the fact this is just ahunch, of course, perience?I suspect, to it is abasic worked at it orbeen introduced that they'venever teaching that the programsin student reasonwhy, andthat some of

226 are centered onthe inner-city and the experience inthe inner-city might dissipate some of the fear thatexists because of an inadequate contact with the area.

Jay Greene:One additional area of research Ithink has been suggested, I don't know whetherit's been indicated specifically, might be in the first selection step onthe college level. Would-be teachers are being selected for teachereducation programs in the col- leges on various bases. If you compare the inadequacy of someof the selection by comparison with the selectionthat goes into other pro- fessions, I think the inadequacy becomes morereadily apparent. If you want to become apsychologist in a graduate program, there's a rigorous selection procedure.If you want to become a physician or go into other professions,ditto. If you want to become a teacher, there is not that degree of selection. Now some of the research cer- tainly should go into helping in thatselection and to challenging even some of the basicassumptions. It may be that in normal times, when supply and demand equalize,that the preparation for teacher education should be in the graduateschool, after an individual has had a chance to develop as anundergraduate. I know the establish- ment will oppose that tremendously,but it's a consideration merely for investigation and research as apossibility. If this is what's happening in all professions, why isn'tit happening in teaching? Why isn't it good for teaching?I'm not saying this is the answer, but it's an example of challengingassumptions for research.

The second point is this. Harry Gilbert has indicated that to wait for the millennium until weget complete criteria is to post- pone action unnecessarilylong. He suggests that we try to do the best we can with criteria and get alongwith that job. I would go one step further thanthat. I'm mindful of the fact that researchin education and in other fields of humanityhas very little and long de- layed impact on practice. You consider the thousands of researchef- forts done in the area of human activityduring the year and the infinitesimal effect, unfortunately so, onpractices. If you wait for research to really lead intomaking changes, it's a long term ef- fort. I think probably that what happensis that individuals doing research develop insights into what'shappening and gradually partici- pate in meetings of this sort, andby discussion bring about changes. Therefore, my proposal, Harry, is to go onestep beyond and to say to prepare to, make changesbased upon meetings of this sort wherepeople who have developed these insightsthrough research, through experi- ences, through thecross-fire of exchange, can get togetherand examine practices and suggest areas wheredifferent cities, towns and groups, can try things out. This is the way changes have beenmade in educa- tion by discussion and the determinationoften of one person in a com- munity to try certain things. I think we ought not shy awayfrom that and validation of researchfindings, but we ought to recognizethat.

227 I'd like to statethat this discussion Murray Rockowitz: back to Mr. but now I want toturn the proceedings will be continued, I'd like to grant asubjec- Before I do, I'dlike to say that Denn. the homework to the panelthis morning. I think that tive grade of A it's accepted they've given us isclear. I think that assignment that motivated, and it has cer- by the class,that it hascertainly been that we have beenholding. I tainly emergedfrom the discussions into back to our ownbailiwicks withincreased insight think we can go little we know about and a fullknowledge of how what has to be done discussing in the lastfew the actual proceduresthat we have been back to Mr. Denn. days. I'll turn this now five-minute break andthen re- Paul Denn: Let us take a blueprints for action. assemble toconsider specific

228 Session No. 8: Summation of Conference

Chairman: Harry B. Gilbert,Pennsylvania State University

Harry Gilbert: Ladies and gentlemen,may I have your atten- tion, please. I have a few charges thathave been expressed tome. I'm going to suggest a way of effecting a compromise.Many people have suggested they wouldlike to be out of hereearlier than 12:30. I think it's quite important;you need time to pack, get ready, etc., and I'm going to suggesta procedure for gettingus all out of here at 11:45, one hour fromnow.

It is this. It has been impossible toget everybody here to participate, much as we would like to have done this. I've talked this over witha few and it seems acceptable, ifyou won't object. This is the procedure Iam now proposing: that we break up into four groups of approximately 8 to 10to meet for 30 minutes,to elect a spokesman and to come up with three, four, five specificideas that represent summary, outcomes, and suggestions for keepingmomentum for this group. There are places rightoutside in the lobby whereyou can be separated from one another. But you have to limityourselves to exactly 30 minutes, andbe back here at 11:15with an elected spokesman who will give a 2- to 3-minute summary of whatyour group has recommended. The spokesmen whocome second, third, and fourth must also be disciplined enough to say weagree with Items 1 and 2 that have been suggestedand add their additionalideas.

Now, does thisseem like moving us too fast?I would like to give everybody a chance to bring in some suggestion,and I think this is one way where we can come up with some specificproposals. Does anybody object to this procedure?No? Then I suggest thatwe do the following: Dr. Brown of Los Angeles,will you take the first comfortable settee and willHarold Mitzel report tothat group. I'm going to try to splitup some of the research people. Mr. Perz, will you take a second group, a little bitaway from group one, and Don Medley, will you report tothat group.

Al Schiff, will ymi takea third group, somewhere further up in the corner, and John Flanagan, would you stay with thatgroup. And Dr. Sanders, will you stay here with the remaininggroup. Now, what research person have I leftout?Dave, will you stay here, please, Dave Ryans. It's a nice time to duckunder a table, but I caught you. Will the industrialistsamong us break up, be schizo- phrenic, be in two places,do anything but, Doug andPaul, don't you stay in the same group. And, I think if you just don'tsit here, and get started but, rememberyou've got to be back hereat 11:15 with an elected spokesmanand a 2- to 3-minutesummary of sugges- tions, specific outcomes,some kind of closure for what this Con- ference has meant and whatit can mean.

229 but I'm sure This will be abit rambling, Albert Schiff: will work it consolidate allthese suggestions those of youwho will out somehow. composed of PaulBaker, RufusBrowning, Our Committee, Irving Gold,Harry Miller,Ralph Paul Fitzgerald,John Flanagan, Bernard Berger,believe: Walter, FredWilliams and to better screeningof admission (1) there must be a and per- real testingof attitudes the collegesof education, a sonality; education at the training andexperience and (2) that the modified and ex- must beintensified, college ofeducation level lead to more criteria that webelieve will panded in termsof the criteria haveyet to bedeveloped; effective teaching,but these be taken andcloser co- that theinitiative must (3) and theteacher train- exist betweenschool systems operation must there's adifference between Somehow or other, large city ing institutions. level and theexpectation of the trainingat the college be workedout mutually; school systems,and this can of any teacherhas to be that theinitial assignment (Li) develop andmaximize the oppor- running start.We have to of the pro- a good to be builtin as part tunities for successand this has of thesupervisors andadministrators fessionalresponsibilities research; must work, andthis needsadditional under whomthis teacher that we have aninteraction process (5) it is important perhaps simulatedteaching, but going on,situationalperformances, selection procedure; this must bepart of the research on"real" teacher there is really adearth of (6) dearth inresearch. We must make upfor this selectiontechniques. are inorder, but studies by Medley,Mitzel and Ryans Continuation of in theclassroom; needed aboutwhat happens actually moreresearch is of measurementfrom the (7) there has tobe a continuum employment. And thisdoesn't teacher traininginstitution through But let's go onand that it stopsat theprobationary period. mean experiencedteacher; expand this tothe moreseasoned research onmatching we haveto do alittle more (8) talk aboutthe broad specific jobs, You know, we teachers for teachers willbe able to flexibility that wewant inteachers, that to trained, butperhaps we have they're wellqualified and kinds fit in if other words, arethere certain do a little morethan this; in specifically cando? of jobs thatteachers

230 of (9) improvement of procedurcF. This whole universe "brand the young teacher;here we have thousandsof young teachers, coming into our classrooms, new spanking"teachers, inexperienced, Let's see what we we have alarge sample to test andto measure. can do withthis vast untapped source. though all our And last, but mostimportant, remember, even problems today of city school systems havesaid we have immediate we're preparing teach- staffing the classrooms,let's not forget that and ers to teachkids in the 70's andBD's, that we have to research plan, and the role ofthe teacher for thefuture, both from the institution's point of view school system's andthe teacher training is most important.

Harry Gilbert:Thank you, Al. That was exactly three Dr. Ted minutes, and I appreciateit. Speaking for the next group, Lang, of New YorkCity.

Theodore Lang: I would like tomention the names of our Mr. Entenmann, Dr. group membersfirst: Mr. Benson, Mr. Cogen, Gold, Dr. Greene, Dr.Medley, Mr. Perz.

Going directly to theresearch that we believeto be es- the sential, ny committeewould like to cite,first, the research in one of selection of supervisors.We have regarded this area as prominence because of ourfeeling that it has been even moreneglected of teachers and, more than the subject ofresearch in the selection importantly, because ourobservation is that a goodprincipal makes possible good staffing.Good leadership inthe schools goes far In the area of toward meeting thestaffing problems of theschool. immediately of research for the selectionof principals, we think effectiveness of a prin- the need for establishmentof criteria for of principals. cipal, and second,techniques for the selection

Turning now to teachers, mycommittee feels that thefirst What are essential is to determinewhat makes an effectiveteacher. teacher and what are the criteria forselection of an effective teaching? We be- teacher behavior patternswhich lead to effective Because we feel lieve the time to startthis investigation is now. of the selection that such research isessential to the improvement city develop in the near process, we urgethat at least one large We future a research programof the kind I have justdescribed. think that this shouldbe a cooperative endeavorof the teacher the examining group, education institutions,the Board of Examiners or systems and, the personnel people andadministrators of the school of what perhaps, of the teacherorganizations. In this question distinguish the varied makes a good teacher: wedeem it important to city or in dif- patterns of good teachingin distinct parts of the been said by Dr. ferent subject areas orlevels. Picking up what has patterns for Schiff, we think that suchidentification of behavior

231 ,...,..-marNrwinwimpt7e.ww,pacwiwproptlyv.impftwmwmaF

specific teaching areas mightreflect itself in the se]ectiontech- niques, which would become morediagnostic. The results of the test- ing of the prospective teachershould then be usable for placement purposes. Consequently, we would seek to placein a middle-class neighborhood the teacher who couldteach well in such a neighborhood, and the teacher who has thepotential for handling thedisadvantaged child in the disadvantaged areasof the city, thus leading tobetter utilization of the teacher andenhancing the opportunitiesfor his retention.

This leads me to the next major areaof research, the area first, sup- of retention. My committee breaks thisinto two parts: port for the beginning teacher;and second, research intothe reasons why new teachers leave theservice. Parenthetically, research in the support of the beginningteacher has been started in NewYork City through the Centerfor Urban Education, which,in cooperation with the Office of Personnel,is trying out six patternsof assist- ance for theentering teacher.Research into the reasons why teachers leave will be undertakenin New York City in the nextfew the sys- weeks.We have already collecteddata on teachers who left tem during the first sixweeks of the term.We shall call them in for interviews and find outwhy they resigned, and alsogive them suggested, an opportunityto return to teaching. As Dr. Brownell we will seek toplace them back in the Cityschool system.

Our committee recognizes thegreat value of this kind of meeting. We think this type of sessionshould proliferate region- ally throughout the UnitedStates and should be repeated,perhaps annually. Further, we urge a discipline inthe conduct of future meetings, in the sense that eachmeeting should bring forth a pro- gram for the ensuing year,developed by the researchers andthe practitioners, and that each followinggathering should bring a re- port of what has transpired inthe preceding year, so that we can evaluate our own efforts. Finally, we suggest that leadershipin this entire area might well comethrough the regional researchlab- oratories being established under TitleIV of the ESEA. Thank you.

Harry Gilbert: Thanks very much, Ted. And now, speaking for group three, Murray Rockowitz.

Murray Rockowitz: The one unanimous thing aboutthe group's action was that I wasto be its spokesman, forbetter or for worse, and thepeople who made the decision wereDrs. Brown, Mitzel, Kalick, Lang, Lucey andBray. I don't think that Dr. Luceyhas been introduced.Would he stand and take a bow? He's here for the Council of Supervisory Associationsof New York City.

Our concerns weresimilar to those expressedearlier by Dr. Schiff and by Dr. Langand if I repeat some of them,I think it's important to repeat them becauseof the fact that the emphaseswill emerge.

232 There was concern for the necessity for resources. Dr. Bray, the industry representative, started with hard cash as the basic to any research that could follow, and, armed with that carh, he would look for a place to employ it.We all agreed that the climate where this research should take place would have to be a favorable climate, one not susceptible to pressures from various groups. Just where to find that, we leave to later deliberation.

There was further agreement with respect to the kinds of research that should take place, and there we identified three dif- ferent kinds of research. Everyone agreed that the research that Dr. Flanagaa outlined, basic longitudinal research, was essential, and that should go on regardless of any other consideration, and Dr. Kalick felt that the colleges could be counted upon in the actual effectuation of a good deal of the research program that would be developed. Dr. Mitzel stressed the two- to three-year intermediate type of research that would be based on process variables, perhaps, in observation of teacher behavior in the classroom as a starting point. He would stress that the New York City Board of Examiners, for example, move off in directions that would result from new con- ceptualizations of the examining process, all the while continuing with current procedures because of the necessities and exigencies of contemporary needs.

There is a third type of research, and therets a difference in the committee on this. The researchers feel that this third type of research is non-productive. The people more close to the firing line feel that itts necessary. I refer to the ad hoc type of re- search which seeks to dissipate generalizations which are generated by emotions and pressures of temporary circumstances with a cold douche of fact. I think, too, that we have to get a climate where post-employment, the teaching staff is willing, with certain guaran- tees, to submit itself to research techniques, being reassured that the data that are being asked of them will in no way prejudice their status in the schools.

And the final point that was made by Dr. Brown is, I think this was unanimous too, that there are problems which are common to the largest districts in this country and this entire session will have been wasted if we just let it drop here. It is essential that we have a dialogue, a continuing dialogue, and that there be follow- up meetings of these groups to implement these suggestions, and I think Dr. Lang made the point earlier, step-by-step implementation over a period of time with specific goals to be set from meeting to meeting.

Harry Gilbert: Thank you, Murray. And now, the friend of the no committee, Dave Darland.

233 As a group, we werevery action- David Darland: Thank you. We felt thatthere were specificdecisions centered in ouroutlook. of se- especially at crucialpoints in the process that must be made, itself, assign- These pointsincluded selection lection of teachers. great deal more of tenure, etc.We felt we knew a ment, the earning than we tend to develop programsrelated to these areas about how to had to be flexi- There was completeagreement that there practice. needs of anydefined community bility so that thecharacteristics and various interest groups,teachers would be met. This would include associations, etc. that the problemsof selec- Our group wasalso in agreement immense and so largethat tion, and all itsrelated problems, were so solve them. Indeed, there probably no city, nodistrict or unit can educational manpowerstudy. wasconsiderable supportfor a national utilization of personneland give some Such a studymight include particularly the typethat would provide some priority to research, to include Such a study would,of course, have predictive analyses. of the things current society,recognizing that many the dynamics of educational manpower we havetried in an attemptto solve the shortage just haven'tworked. of two kinds of Our gr( ipunderscored againthe importance There wasconsiderable discus- research, namely,applied and basic. problems in anappropriate con- sion on theimportance of defining enough to really getat themeffectively rather text, i.e., broad In this regard,it is than continuingto work on apiecemeal basis. the AssociateCommissioner interesting toknow that Louis Bright, Office of Education,has made the for Research ofthe United States after the basicresearch observation that, inthe case of industry, much to disseminateand innovate, is done, itcosts seven times as to do the basicresearch in thefirst and apply theidea as it does falling down ineducation since we are place. He feels that we are attention to theapplication aspectsof research. not paying enough thinking of our group. Possibly this somewhatcorresponds to the Thank you. I thank eachof the Chair- Harry Gilbert:Thank you, Dave. Bill Brown, andDick Sandersfor really men, AtSchiff, Bob Perz, getting the groupsto move. have a few closingremarks We have aboutfive minutes and I and a sad privilegeto close out this to make. It's both a happy opportunity to meet Happy and sadbecause it gives me an Conference. fruition of ideasthat have beenunder all of you andit represents a consideration for along time at theBoard of Examiners. par- We would hopethat each and every Some specifics. time when he Invitational Conferencewould take the ticipant to the Mrs. Klein. We returns home towrite to theConference Secretary,

234 suggest that you consider such things in the letter as, has this been a stimulus to any local action; secondly, has this been a stimulus to any action in organizations with which you are affili- ated; third, what specific suggestions do you have for follow-up; and fourth, do you have any specific suggestions for the composi- tion for the final report?

This is by way of announcing that there will be a report. We have each of the papers, the formal papers that were presented or summarized. These will be published in the proceedings.We have the comments that everybody has made, and may I suggest that we will assume that we have youi. permission to include the comments that have been recorded, unless you specifically direct us other- wise. You may wish not to be reported or recorded.Yes, we will edit them rather carefully, and if there is any doubt, we will com- municate with you, but you'll have to leave some discretion with the editorial task force.

I want at this very last minute to repeat the thanks that Paul Penn and others have voiced.We thank Perry Kalick and Ger- hard Lang. I want to thank Paul Fitzgerald particularly, for the many hours that he gave to this work. I want to thank our gracious secretary, Mrs. Klein. I want to thank the Board of Examiners for their participation, and Jerry Eisenberg, here, who has come up here, given a couple of days, and has been working with back to the pro- ceedings, eager to get into the discussion, but he has stuck to his guns, or I should say, his tapes. Thank you very much, Jerry.

And my final thanks, of course, go to the participants. You are all busy executives. That you have taken the time to pre- pare for this Conference and to participate in it is a tribute to you and to your recognition of the significance of the topic. On behalf of the Board of Examiners I thank you very much and hope you have a very pleasant trip home.

235 SECTION IV:REVIEW OFCONFERENCE

on thevariety and at timeslengthy, comments The diverse, participants makeit difficult which occupiedconference consists of topics synthesis ofideas. This review their views as a 2) a blueprint to present of thediscussions, of three parts: 1) a synthesis evaluation ofthe conference. for action,and 3) an Synthesis ofthe Discussions teacher selection participants allagreed that Conference of teachertraining, with in theoverall context must be dealt and on-the-jobtraining dur- recruitment,teacher placement, teacher selec- teacher The problemsinherent in ing theprobationaryperiod. political, of philosophical,psychological, tion involvebasic issues One cannotgrapple well aseconomicdimensions. sociological, as unless anduntil onedeals with meaningfully withteacher selection selection," re- "The stateof the artof teacher the totalgestalt. emerged fromthe dark ages." participant,"has hardly this con- marked one surveys,l which served asbases for The resultsof the two pessimisticobservation. ference,reinforce this of people are,wittingly or of agenciesand grorps A number selection process: 7) candi- partners inthe teacher 3) unwittingly, 2) teachertraininginstitutions, dates forteaching positions, 4) boards of (e.g., boardsof examiners), teacherselection agencies who appointteachers to (including directorsof personnel education administrators andsupervisors, assignment), 5)school their first organizations,7) accrediting 6) teacher unionsand professional community. of education,and 8) the agencies andstate departments centered onthe conferencedeliberations The thrustof the in furthering agencies and groupsof people roles of someof these teacher selectionprocess. orhindering the

Teacher Lang, G.; andKalick, P. M. 1Gilbert, H. B.;Bogen, I.; S stems Procedures in Lax. ePublic School Policies and Researc Selection Unpu lis ed repor Cooperative States. Health, in the United Education, U.S.Department of Project No. S-331, Office of 63 p. Education andWelfare,1966. Procedures Teacher SelectionPolicies and Lang, G. andKalick, P. M. School-- Affiliated withthe Metropolitan in Sixty-twoSchool Systems Public SchoolS stems in ared with thoseof Lar:e StudCouncil Com xaminers, oar New Yor e Bo: o e U t States. 28 p. Rducation of theCity of NewYork, 1966.

236 Candidates for Teaching Positions

The image of teaching as a career is still quite tarnished. Despite some adjustments in recent years, teacher salaries arestill too low and are not attracting bright young people who areveering toward more lucrative fields. The real or imaginary image of the blackboard jungle is a greater deterrent to selecting teachers for the large urban schools than are possible inadequacies of the selec- tion instruments. Moreover, the selection process itself tends to be often viewed as a barrier. These and other factors resulted in the observation made by all representatives of the larger urban schools: we have a shortage of teachers,let alone "effective" teachers, and a sizeable number of ,successful candidates (having passed the screening process) are lost to the more attractive suburban communities. The participants did not deal with the dynamics of vocational choice, i.e., why people select teaching. Teacher recruitment begins already in the elementary school. Teaching is the only profession which fur- nishes positive or negative models to all people.The "good" and the "bad" teachers who fill our classrooms are the unofficialrecruiters and initial selectors.

Appointments to teaching positions in large cities must be made earlier than they are now made. May is the modal month for Sep- tember appointments. These should be made in March.

Teacher Training Institutions

Teacher selection is a developmental process with at least three stages: 1) selection of students for teacher preparatory pro- grams, 2) selection for the initial job,and 3) selection during the probationary period culminating with the award of tenure. There must be better screening at the entry level into the profession. Teacher training curricula must be modified better to prepare beginning teachers for the realities of the classroom. Tin recently announced programs by Fordham University and The City Collegeof the City Uni- versity of New York as well as others throughout the nation, are steps in the right direction. These programs are emphasizing more experiences in the schools for students. Representatives from the colleges and the school systems issued a call for much closer co- operation and coordination of efforts, a continuous dialogueof these two partners.

Teacher Selection Agencies

The following questions were posed:1) What is the role of the teacher selection agency?2) What are the candidates, perceptions of the selection agency?3) Who should be selected?4) What methods are and should be used?5) How should we validate selection methods?

237 1) Role of Teacher Selection Agency

The teacher selection agency must consider that its selec- tions have to be made within a framework of reality, which was spelled out in great detail by Dr. Redfern. Several administrators from the larger school systems joined him in this overriding consid- eration: current, and most likely future, teacher selection methods represent a compromise between what we are able to do, can do, and should be doing.

The representatives from civil service voiced the unanimous opinion that the merit system must always be an important component of the selection process. Even in times of teacher shortages, there exist oversupplies of candidates for certain teaching areas and, usually, for supervisory positions.Lacking definitive selection criteria, at the very least, teacher selection agencies must devise a selection process which screens out the "poor risks,u'e.g., candi- dates who lack basic communication skills, who are severely physically or psychologically handicapped, and who might be prejudiced against the kinds of children whom they will be teaching.

2) Candidates and the Teacher Selection Agency

Intensive efforts must be exerted to "sell" the selection process to the candidates and to improve and modify(streamline) pro- cedures wherever possible. It is essential to discourage a seemingly widespread impression among candidates that they have to battle an impersonal system which seems to place too many roadblocks in the way of getting a teaching appointment. We must be cautious lest these at- titudes toward the selection agency and its process are carried over into teaching. For the candidate's contact with the selection agency' is his first contact with the "system" and it should, hopefully, be a mutually satisfactory one.

3) Who is to be Selected

The participants enumerated the usual array of characteris- tics presumed desirable for "effective" teachers. Three qualities were mentioned most frequently: commitment to teaching, ability to grow professionally, and flexibility. We should aim our efforts at the potential career teacher, not the temporary "drop -ins." We are selecting teachers for the 70's and 80's. Teacher roles are most likely to change in direction and emphasis. To educate children who can cope with the demands and uncertainties of the future, we must select teachers who themselves have the requisite attitudes and skills to deal with the demands and opportunities which the future will bring. Selection of paraprofessional personnel will develop into a major task, it is believed.

238 4) Methods Which Are andShould Be Used

Current methods, describedin the two surveyreports,2 seem to screen out the "poorrisks." There was a general consensus,how- ever, that ourselection methods do not assessthe primary qualities in- we are lookingfor in our candidates. The representatives from dustry cited techniqueswhich school systems couldexperiment with. simulation technique as One of the mostpromising of these is the job exemplified by A. T. and T.'sassessment centers. Teacher assessment centers could be used asresearch criteria againstwhich one could Use validate techniques andindexes used in the employmentsituation. of the video recorder as a meansof training interviewers wasanother highly relevant suggestionoffered.

5) Validation of Teacher SelectionMethods

Agreement on validcriteria of effectiveselection methods has stymied and the development ofadequate validating instruments researchers over a periodof at least 50 years. This conference has not produced any quick orsimple solutions. However, it did wrestle with this issue atpractically every session. Three researchers out- lined specific paradigmsfor validating teacherselection methods. test model be abandoned It was stronglysuggested that the aptitude not predictive for the achievement model,that we focus on content, validity, and that we use measuresof classroom performancederived The problem from systematic classroomobservation as data sources. tackled at various of validating selectiondevices can and should be levels of complexity. In many instances wemust settle for practical, immediate criteria and usecrude instruments. The real breakthrough piecemeal, at- in this area, however, can comeonly by a broad, not Medley, Mitzel, tack as outlined in theposition papers of Drs. Ryans, cooperative venture and Flanagan. Such a frontal attack must be a and led by joined by school systems,teacher training institutions, that "overall the hard-core researchers; All of the experts agreed teacher effectiveness" was auseless concept. Instead, specific ex- should be used as amples of teacher behaviorin specific settings in these criteria and selectioninstruments devised to predict success limited and circumscribed areas.

Boards of Education

Two of the many tasksof boards of education wereexplored: climate which is 1) to make teaching attractive,and 2) to provide a well as the conducive to research onteacher selection problems as means to conductresearch.

2Ibid.

239

,...4..40,1e4.4"/"4144,5VON,re,.., selection methods and Efforts to evolveeffective teacher vitiated selection standards are,to a large extent, to maintain high classrooms of our shortage of candidatesto man the by the persistent several schooladminis- The unionrepresentatives and large cities. school sys- the weak holding powerof our larger trators emphasized making teachingmore Boards of educationmust concentrate on tems. significantly Several avenues are open,such as offering attractive. teachers to schoolswhere there higher salaries,appointing neophyte or,conversely, by exists a maximizationof probability of success schools which bafflethe resources not appointingthem to "difficult" enabling teachersto teach, Orhighly experiencedteachers, and by physical safety,creating smaller i.e., providingfor the teacher's that he canconcentrate on classes, employingteacher aides so instructional duties,etc.

paradigms for research onteacher selection A number of staffing and re- The boards ofeducation must provide were proposed. protect the re- undertake the neededresearch and to sources to dictate attacks by pressure groupswhich may wish to searchers from data to further methodology used, andthe reporting of the thrust, the profession and of their own endsrather than thoseof the teaching society at large.

School Administratorsand Supervisors play crucial.roles School administratorsand supervisors The participants re- in the developmentalteacher selection process. probationary years as anintensive on-the-jobtraining garded the initial screen- golden opportunityto validate the period as well as a It was felt the college andat the job entrylevels. ing efforts at improved by focusing selection methodscould be greatly that teacher and supervisors. methods of selectingschool administrators on better of recruit- incompetent supervisorsvitiate the best Uncommitted and who feel that ment and selectionefforts. Inexperienced teachers will not remainlong in teaching. they are flounderingand are failing called for anintensive and compre- One directorof teacher training administrators to takethe place of the hensive programfor school which prospectiveadministrators all too commonhodgepodge of courses take to fulfillcertification requirements. and the Educational Teachers Unions,Professional Organizations Establishment teachers unions,profes- The participantsrecognized that the "educationalestablishment" sional organizations,and membersof agencies and statedepartments ofeducation) play (e.g., accrediting in It was suggestedthat their role key roles inteacher selection. be thoroughlystudied. However, the ,.ariousdecision-making processes this area. The efforts oftwo this conferencedid not delve into represented at thisconference, militant organizationsof teachers,

2140 are well known with respect to upgrading teachers' salaries, reducing, class size, greater per-pupil expenditure, and greater teacher involve- ment in educational policy-making, just to cite a few illustrations.

Community

The conference touched upon but did not come to grips with knotty issues such as the role of laymen in teacher selection, citi- zens' expectations of teachers, and the use and the selection of paraprofessional teacher aides.These and many other facets were relegated to deliberations which should be held in the future.

Blueprint for Action

A number of concrete suggestions emerged from the con- ference which are recorded in Session 8: Summation of Conference. Additional recommendations follow:

1) The problem of teacher selection must be attacked ina systematic manner, as spelled out in detail by Drs. Ryans, Mitzel, and Flanagan.

2) Teacher selection methods should be validated. Appro- priate paradigms were offered by Dr. Ryans, Medley, Mitzel, Flanagan, and Gilbert.

3) Research efforts should be launched on the selection of school administrators and supervisors.

4) A number of recommendations were offered designed to cope with immediate problems (e.g., use of walk-in-tests; better train- ing of interviewers by use of the video recorder, job simulation, etc.).

5) A working partnership should be formed among teacher training institutions, school systems and research workers drawn from diverse disciplines for the purpose of carrying out the major studies outlined in various position papers.

6) Regional seminars should be held for leaders in educa- tional research, in allied disciplines, and school personnel adminis- trators to bridge the gap in their respective conceptualizations.

7) A conference should be convened which deals with effec- tive utilization of educational manpower. Since there appear to exist a good deal of uncoordinated and wasted effort in the field of teacher selection and considerable personnel pirating, such a confer- ence seems very much needed.

8) Annual meetings, like this conference, should be or- ganized, each meeting to have a specific program and course of action for the ensuing year to be assessed at the next meeting. Evaluation of theConference

at this time, bepurely An evaluationmust, of necessity which seeds will cometo fruition. subjective. It is too early to see of satisfaction All participantsexpressed a great deal much-needed opportunity with the conference. They felt it afforded a negative ideas. But, as must beexpected, a number of to exchange felt that the college comments were alsoexpressed. One participant too defensive and representatives and theschool administrators were Another person deploredthe ap- the researchers tootheoretical. agonize over existingproblems parent greater tendencyto analyze and lead to solutions. than to seek action programswhich he felt might In the meanwhile, Time, of course, willprovide the answer. many ideas -often conflicting -which were it may be that the very who did expressed at theconference will serveto stimulate readers not attend as well asthose who did.

242 MR1111-

SECTION V: LIST OF PARTICIPANTS

Baker, Paul - Manager, Social ScienceResearch Division, Standard Oil Company of New Jersey Baldwin, H. W. - Supervisor, CertificatedSelection Department, Los Angeles City School Districts Benson, Arthur L. - Senior Director,National Teacher Examinations, Educational Testing Service Berger, Bernard - Director of Trainingand Career Development Divi- sion, Department of Personnel, The Cityof New York Bransford, Thomas L. - Director of Examinations,New York State Civil Service Department Bray, Douglas W. - Director of PersonnelResearch, American Telephone and Telegraph Company Brown, William B. - Associate Superintendentof Schools, Los Angeles City Schools Brownell, Samuel r. - Professor of UrbanEducational Administration, Graduate School, Yale University Browning, Rufus C. - AssistantSuperintendent, Public Schools of the District of Columbia Cogen, Charles - President, AmericanFederation of Teachers Darland, David D. - Editor, Journal ofTeacher Education, National Education Association of the U. S. Denn, Paul - Vice Chairman, Boardof Examiners, Board of Education of The City of New York Eisenberg, Jerrold - Acting InstructionalRecordist, Bureau of Audio- Visual Instruction, Board of Educationof The City of New York Entenmann, ThomasE. - Acting Director, Division ofExaminations, The School District of Philadelphia Fitzgerald, Paul-Assistant Professor, Department of Education, St. John's University Flanagan, John C.-President, American Institutes for Researchin the Behavioral Sciences Gilbert, Harry B. - Assistant Dean. for ResidentInstruction and Pro- fessor of Education, Pennsylvania StateUniversity Gold, Irving J. - Acting Examiner, Boardof Examiners, Board of Edu- cation of The City of New York Gold, Milton - Director of Teacher Education,Hunter College of The City University of New York Greene, Jay E. - Examiner, Board of Examiners,Board of Education of The City of New York Jacobson, Raymond - Deputy Director, Bureauof Recruiting and Examin- ing, U. S. Civil Service Commission Kalick, Perry M. - Assistant Professor,Department of Education, Hunter College of The City Universityof New York Keating, Richard - Education Specialist,U. S. Office of Education

213 Klein, Arthur - Examiner,Board of Examiners, Boardof Education of The City of New York Kolodny, Jules - Secretaryand Assistant to thePresident, United Federation of Teachers Lang, Gerhard - AssociateProfessor, Department ofEducation, Mont- clair State College Lang, Theodore H. - DeputySuperintendent of Schools, Boardof Education of The City of NewYork Lierheimer, Alvin P. - Director,Division of Teacher Educationand Certification, The Universityof the State of New York of Educa- Lucey, Stuart - AssistantSuperintendent of Schools, Board tion of The City of New York Mackey, Charles - TheState Education Department,The University of the State of New York Medley, Donald - Head,Teacher Behavior ResearchGroup, Educational Testing Service Mitzel, Harold E. - AssistantDean of Research, Collegeof Education, and Professor of Psychology,Pennsylvania State University Perz, Robert E. - AssociateSuperintendent for Professionaland Classified Personnel, Divisionof Personnel, Board of Education,Philadelphia Redfern, George - AssociateSecretary, American Associationof School Administrators Resnick, Lawrence -Chairman and Professor of Philosophy,State University of New York atCortland Rivlin, Harry N. - Dean,School of Education, FordhamUniversity Rockowitz, Murray - Examiner,Board of Examiners, Boardof Education of The City of New York Ryans, David G. -Director, Education Researchand Development Center, College of Education,University of Hawaii Sanders, Richard H. - ViceChairman, Board of Examiners,Board of Education, Chicago Schiff, Albert - Director,Division of Personnel,Detroit Public Schools Steiner, Arch K. - AssociateResearch Coordinator, ResearchBranch, Division of Elementary andSecondary Research, U. S. Office of Education Streicher, Samuel - Examiner,Board of Examiners, Boardof Education of The City of New York Unser, Gertrude - Examiner,Board of Examiners, Boardof Education of The City of New York Montclair State Walter, Ralph - Chairman,Department of Education, College Williams, Fred H. - AssistantSuperintendent, Board ofEducation of The City of New York

214.it