DOCUMFNIT RE S lIMF EA 001 341 ED 021 312 24 By- Olton, Robert M.; And Others THE DEVELOPMENT OF PRODUCTIVETHINKING SKILLS IN FIFTH-GRADECHILDREN. Wisconsin Univ., Madison Research andDevelopment Center for CognitiveLearning. Spans Agency-Office of Education(DHEW), Washington, D.C. Bureau ofResearch Report No- TR-34 Bureau No- BR- 5- 0216 Pub Date Nov 67 Contract- OEC- S-10-154 Note- 47p EDRS Price MF-$0.25 HC-$1.96 EDUCATIONAL Descriptors-CLASSROOM ENVIRONMENT, COGNITIVEDEVELOPMENT, *CREATIVE THINKING, QUOTIENT, PROBLEM SOLVING, EXPERIMENTS, *GRADE 5,INSTRUCTIONAL MATERIALS, INTELLIGENCE SEX (CHARACTERISTICS), *PRODUCTIVE THINKING, *PROGRAMEDINSTRUCTION, *PROGRAMED MATERIALS, STATISTICAL ANALYSIS, STUDENTS, TEACHERPARTICIPATION This study investigated (1) the extentto which thinkingand problem-solvingof fifth-grade students could be improvedby the use ofself-instructional programed lessons (the productive thinkingprogram), (2) the relationshipbetween productive thinking abilities and 10-sexcharacteristics of the learner,and (3) the relationship between level of productivethinking performance and theextent to whichoverall classroom environment was judiged tofacilitate creative thinking.Results from 44 fifth-grade classes in Racine, Wisconsin,involving a total of 704students, showed statistically significant increments inthinking and problem-solvingperformance on a wide variety of productive thinking measures.These instructional benefitsoccurred for virtuallyalltypes of students,regardless of sex or general10 level, and were especially marked for s-tudents inclassrooms having environmentswhich were judged to provide relativelylittlesupport and encouragementfor the development of productive thinking. Apart from theeffects of the instructionalmaterials, performance related to sex on theproductive thinking measures used inthis study was significantly (girls generally scored higher thanboys) and showed a strongand positive relation to 10. EA 001 340 is a related document.(Authors/JK) (A

e

ft> e

THEpEVELOPMENTOF, ,PRODUCTIVE THINKING SKILLS ,IN FIFTH-GRADE CI-KDREN

ai -.

ta

*

o r

d

...

s

1

..

p

i , WyStONSIN RESEARCH AND, DEVELOPKENT CENTER FOR COGNETIVgLEARNING

. )

... Technical Report No. 34

THE DEVELOPMENT OF PRODUCTIVE THINKING SKILLS

IN FIFTH-GRADE CHILDREN

Robert M. Olton, James L. Wardrop, Martin V. Covington, William L. Goodwin, Richard S. Crutchfield, Herbert J. Klausmeier, and Teckla Ronda

U.S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE OFFICE OF EDUCATION

THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE

PERSON OR ORGANIZATION ORIGINATING IT.POINTS OF VIEW OR OPINIONS STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION POSITION OR POLICY.

Wisconsin Research and Development Center for Cognitive Learning The University of Wisconsin Madison, Wisconsin

"PERMISSION TO REPRODUCE THIS November 1967 COPYRIGHTED MATERIAL HAS BEEN GRANTED By Martin Covington, Richard Crutchfield, and Lillian Javies

TO ERIC AND ORGANIZATIONS OPERATING UNDER AGREEMENTS WITH THE U.S. OFFICE OF EDUCATION. FURTHER REPRODUCTION OUTSIDE THE ERIC SYSTEM REQUIRES PERMISSION OF THE COPYRIGHT OWNER."

The research reported herein was performed pursuant to a contract with the United States Office of Education, Department of Health, Education and Welfare, under the provisions of the Cooperative Research Program Cen- ter No.C-03/Contract OE 5-10-154), and with additional support of a grant from the Carnegie Corporation of New York to the Creative Thinking Project, Institute of Personality Assessment and Research, University of California. Berkeley.

GPO 1506459-1 PREFACE

High priority is now being given to the improvement of children's learning in the cognitive domain, particularly in connection with productive thinking skills.The federal government, various profit and non-profit agencies, and many individuals are attempting to extend knowledge about this aspect of human learning and to improve related educational practices through programs of research and development.Behavioral scientists and school people are joining together in an attempt to learn more about the nature of productive thinking skills and the conditions associated with more efficient learning of them.This report of a large-scale controlled experiment involving personnel from three different organizations shows how specialized knowledge and re- sources can be brought to bear upon extending knowledge and improving edu- cational practice through research and development activities.Personnel participating in the study were from the Wisconsin Research and Development Center for Cognitive Learning, the Creative Thinking Project of the Institute of Personality Assessment and Research at the University of California, Berkeley, and the Racine (Wisconsin) Unified School District, No. 1. The instructional materials used in the experimentThe Productive Thinking Program,Series One: General Problem Solvingwere developed by Professor Martin Covington, Professor Richard Crutchfield, and Mrs. Lillian Davies of the Creative Thinking Project.Dr. Robert Olton, of the same project, was chiefly responsible for the development and subsequent scoring of the large battery of productive thinking tests used in the experiment and wrote the first draft of the major portion of this report.Both the instructional and test materials are copyrighted; the instructional materials are now available for regular school use.Professor Covington and Dr. Olton, in addition to their role in the development of the new materials, worked directly with the staff of the Wisconsin R & D Center in planning, executing, and reporting the ex- periment. Dr. William Goodwin of the Wisconsin R & D Center took primary responsi- bility in working out the design of the experiment with Dr. Olton.Dr. Goodwin coordinated the various components of the experiment, making sure that the pretesting, the instruction, and the posttesting were carried outaccording to schedule during the second semester of the 1965-1966 school year.Dr. James Wardrop of the Wisconsin R & D Center was responsible for the computer analysis of the data during the 1966-1967 school year, and for writing the first draft of the results during the spring and summer of 1967.Professor Herbert J. Klausmeier of the Wisconsin R & D Center initially consulted with Professor Crutchfield regarding the experiment in the summer of 1965 and subsequently developed and directed the strategy for bringing the resources of the three organizations to bear most effectively upon execution of the experi- ment. He consulted with relevant personnel on all matters regarding thedesign and conduct of the experiment, per se.He and Professor Crutchfield defined the more precise roles of the staff and the contribution of their two respective organizations; he and Mr. Harris Russell of the Racine schools similarly defined the roles and contributions of their two respective organizations.

iii Mr. Harris Russell, Director of Instruction, Racine Unified School District No. 1, made the necessary arrangements for his central staff, building princi- pals, and teachers first to decide whether to participate in the experiment and, subsequently, to carry it out.Miss Teckla Ronda of the Racine schools esti- mated the extent to which each participating classroom represented a favorable creative environment, made sure that materialS were received by teachers and used according to the plan outlined, and secured relevant information regarding each child, including IQ score, educational achievement test scores, socio- ecoriomic status, and sex.The elementary school principals and fifth-grade teachers gave generously of their time, and also offered many useful ideas concerning the experiment. Through this union of personnel and resources from the three organizations, answers to the following questions were secured. 1. To what extent can specified productive thinking abilities be taught to elementary school children through the use of programed instructional material designed to teach thinking skills independently of the content of any specific subject field ? 2. Are some productive thinking skills more amenable to instruction than others ? 3. Can productive thinking skills be learned by children of all IQ levels ? 4. Are there consistent sex differences in productive thinking skills ? 5. How are productive thinking skills affected by a classroom environment judged to facilitate creative expression by the students ? The reader will find this report interesting from two viewpoints.First, it is a clear account of the development of productive thinking skills in school children; answers to the preceding questions were obtained and are more defini- tive than in most experiments.Second, the execution of the experiment, includ- ing the placement of classes into experimental and control groups, the type of information gathered, the execution of the treatments, and the analysis of the data, shows that useful information can be obtained through carefully conducted research in school settings. Herbert J. Klausmeier Professor of Educational Psychology Director, Wisconsin Research and Development Center for Cognitive Learning Richard S. Crutchfield Professor of Psychology Director, Creative Thinking Project of the Institute of Personality Assessment and Research, University of California, Berkeley.

iv CONTENTS

Page

List of Tables vii Abstract ix

I. Rationale of the Study 1 Background 1 Aims of the Study 1

Methods 3 Subjects and Conditions 3 Instructional Materials 3 Description of the Materials 4 Administration of the Materials 5 Measuring Instruments and ScoringProcedures 5 Performance Indicators and Scoring 7 Test of the Master Thinking Skill 8 Measures of Understanding Thinking 10 Experimental Design and StatisticalAnalyses 10

Results 11 Pretest Results 15 Internal and Posttest Results 20 Effects of IQ 22 31 IV. Discus sion Effects of Treatment 31 Effects of Environment 32 Effects of Sex 33 Effects of IQ 33

Appendix 35

References 39 LIST OF TABLES

Table Page

1 Classification of the Tests of Productive Thinking Used in This Study 6 2 Normative Quality Ratings for Iaeas in Thel4s t ei,yDf_g_le Old Black House 9 3 Means and Standard Deviations of Stratification Variables 11 4 Means and Standard Deviations of Pretest Variables 12 5 Means and Standard Deviations of Internal and Posttest Variables 13 6 Summary of Analyses of Variance for Stratification Variables 14 7 Mean Level of Performance of Students in Treatment and Control Groups and in Facilitative and Nonfacilitative Environments: Stratification Variables 15 8 Mean Level of Performance of Males and Females: Stratification Variables 15 9 Summary of Analyses of Variance of Pretest Variables 16 10 Mean Level of Performance of Treatnent and Control Groups: Pretest Variables 17 11 Comparison of Ideas Generated by Students in Facilitative and Nonfacilitative Environments 18 12 Mean Level of Achievement of Solution for Males and Females: Pretest Variables 18 13 Comparison of Ideas Generated by Males and Females: Pretest Variables 18 14 Summary of Analyses of Variance: Main Effects for Internal and Posttest Variables 19 15 Mean Level of Achievement of Solution for Treatment and Control Groups: Internal and Posttest Variables 21 16 Comparison of Ideas Generated by Students in Treatment and Control Groups: Posttest Variables 21 17 Means for Treatment and Control Groups on Remaining Internal and Posttest Measures 22

vii List of Tables (coned) Page Table by Males and Females: 18 Comparison of Ideas Generated 23 Posttest Variables Interactions 23 19 Means for SignificantTreatment X Environment Stratifying Variableg24 20 Summary of Analyses ofIQ as a Main Effect: 24 21 Means of IQ Groups onStratifying Variables Pretest.Variables 25 22 Summary of Analyses ofMain Effect of IQ: Achievement of Solution forIQ Groups: Pretest 23 Mean Level of 26 Variables Pretest Variables 26 24 Comparison of IdeasGenerated by IQ Groups: Posttest Variables 27 25 Summary of Analyses ofMain Effect of IQ: Achievement of Solution forIQ Groupe: Pretest 26 Mean Level of 28 Variables Posttest Variables 29 27 Comparison of IdeasGenerated by IQ Groups: Posttest Variables forWhich Significant IQ 28 Means for Remaining 29 Effect was Found and I'reatment 30 29 Means for SignificantInteractions Involving IQ and Environment 30 30 Means for SignificantInteractions Involving IQ

viii ABSTRACT

The major purpose of this study was to investigate the extent to which increments in the thinking and problem-solving performance of fifth-grade students could be produced by the use of self-instructional programed lessons (The Productive Thinking Program, Series One: General Problem Solving) which were designed to teach skills and strategies of creative think- ing independent of any specific subject field.The study also investigated the relationship between productive thinking abilities and certain character- istics of the learner (IQ and sex), and the relationship between level of pro- ductive thinking performance and the extent to which overall classroom "environmentwas judged to facilitate creative thinking. Results from 44 fifth-grade classes showed that The Product' Ire Thinking Program produced statistically significant increments in thinking and prob- lem-solving performance on a wide variety of productive thinking measures. These instructional benefits occurred for virtually all types of students (regardless of sex or general level of IQ), and were especially marked for students in classrooms having environments which were judged to provide relatively little support and encouragement for the development of productive thinking.These effects were obtained when the materials were used as an entirely self-contained, self-instructional program; considerably greater educational benefits could be expected under conditions where the materials are reinforced by active teacher participation. Quite apart from the effects of the instructional materials, performance on the prcductive thinking measures used in this study was significantly related to sex (girls generally scoring higher than boys) and showed a strong and positive relation to IQ,

ix RATIONALE OF THE STUDY

BACKGROUND trict endorsed the study enthusiastically since it offered the possibility of extending the pro- During the past five years, the Creative gram of creativity to a large segment ofthe Thinking Project has been developing a set of entire school population. special instructional materials designed to de- velop and facilitate creative thinking in students of the upper grades of elementary school. These AIMS OF THE STUDY materials (described in Section II) are a series of self-administering programed lessons which The major purpose of this study was to in- require a total of about 16 classroom hours for vestigate the extent to which increments in the completion.The materials have undergone ex- thinking and problem-solving performance of tensive pilot development and revision and fifth-grade students could be produced by the were used in two recent experimental studies use of progt.amed lessons(Series One of The involving several schools in the San Francisco Productive Thinking Program) which were de- Bay area.In both of these studies, the ma- signed to teach skills and strategies of creative terials produced a substantial increase in the thinking independent of any specific subject student& creative thinking performance (Cov- field. ington and Crutchfield, 1965; Crutchfield, Within this framework, several specific aims 1966) and were very favorably received by both were incorporated: students and teachers. After the second San (1) To investigate further the extent to which Francisco Bay area study, the materials under- productive thinking (i. e., creative thinking went a complete revision, and the desirability and problem solving) abilities can be taught to of a large-Lcale experimental study with the elementary school children independent of any revised materials (titled The Productive Think- specific subject field.Currently, there is ing Program, Series One; General Problem strong emphasis in the schools upon specific Solving) became apparent. subject matter areas, with little attention being Researchers at the University of Wisconsin given to teaching general skills which transcend have also studied creativity in school children, specific curricula.If such cognitive skills can particularly in gifted children (Klausmeier, be taught successfully, they may be applicable Harris, and Ethnathios, 1962; Klausmeies and in a variety of subject areas and should enhance Wiersma, 1964), and have organized programs performance in these areas. for the gifted which included education for (2) To discover whether some of these skills creativity (Klausmeier, 1963; Klausmeier and or behaviors are more readily taughtthan others. Teel, 1964).In addition, the focus of the For example, one aspect of problem solving is Wisconsin Research and Development Center the asking of appropriate questions. Can stu- for Cognitive Learning is on improving educa- dents be taught this behavior more or less suc- tion through a better understanding of cognitive cessfully than they can some other skill, such learning, including both creativity and problem- as generating a set of possiblesolutions to a solving processes. problem ? Thus, the directors of the Creative Thinking (3) To investigate the relationship between Project and the Wisconsin R 8c D Center agreed certain learner characteristics (intelligenceand that an outstanding research opportunity could sex) and performance on a broad diversity of be realized by pooling some of the specialized productive thinking measures.It is generally resources and capabilities of each group in a reported that girls perform slightly better than large-scale experimental study.The officials boys on most measures of academic achieve- of the Racine (Wisconsin) Unified School Dis- ment.Is this also true of measures of productive thinking ?It has also been reported that intel- gram by using these materials as an entirely ligence and "creativity" are, relatively independ- self-contained program of instruction, with all ent.Does this independence persist throughout forms of teacher participation purposely held several different aspects of productive thinking to a minimumin much the same way that a and on several different types of problems ? In teacher might assign a program of independent addition, is there a relation between intelligence reading which a particular student or group of or sex and the ability to learn productive think- students would be expected to do on their own, ing skills ? with a minimum of supervision.Moreover, the (4) To investigate extent to which overall entire program was used in a compressed period classroom "environment" facilitates productive of time at the rate of virtually one lesson per thinking in students, and the extent to which day.Thus, this study investigated the effec- students in different types of classroom envi- tiveness of these materials when they were ronments profit from instruction in productive used under the most demanding conditions thinking.To what extent does merely providing likely to be encountered in a school setting. an encouraging environment promote creative (In addition to being valuable in a practical thinking ? Are instructional materials more (or sense, the results would also become an im- less) effective in such environments ? portant component of a systematic research ef- (5) To probe some of the instructional limits fort to explore the effectiveness of materials of Series One of The Productive Thinking Pro- such as these under a variety of instructional conditions. )

2 II

METHODS

these ratings wererepresentative of thenormal SUBJECTS ANDCONDITIONS state of affairs thatexisted in the various artifact of the par- Subjects for this experiment werethe stu- classrooms and were not an fifth-grade classrooms in ticular discussion situationin which they were dents in 44 of the 47 The final, single,Environment rat- the Racine, Wisconsin,Unified School District obtained. classes par- ing assigned to eachclass was a composite No. 1.All students in these 44 the several objectiveratings ticipated in the experiment,but analysis is score derived from the supervisor made. based upon a sample ofeight males and eight to Selection Classrooms wererank-ordered according females selected fromeach class. The twenty- by rank-ordering allmale the compositeenvironment rating. was accomplished with the highestratings were students from a givenclassroom on thq basis two classrooms selecting one student designated as Facilitativeenvironment class- of IQ, and then randomly those 22 classroomswhose from each eighth ofthe distribution.Eight fe- rooms; likewise, selected from the sameclass composite rating wasbelow the median were males were then Nonfacilitative environmentgroups. in the same way.This procedure wascontinued classified as involved Within each of theseEnvironment blocks, for all 44 classes.The entire analysis randomly assigned to a total of704 students.Selection was made half the classrooms were The Pro- the Treatment group(whioh would use after the experimenthad been completed but Program) and the remaininghalf before any scoring oftests was begun. ductive Thinking the class-- reflected, in proper pro- constituted the Controlgroup. Thus, This large sample was subdivided asfollows: portion, the wide varietyof abilities and back- room population grounds characteristicof students in an urban Number school system (a cityof about 100, 000 popu- of Classes lation) and henceprovided a thoroughly repre- sentative base foranalyzing both overalland Treatment 11 internal effects.The size of the samplealso Facilitative Control 11. permitted a very stableestimate of error and of classroom means Treatment made possible the use Nonfacilitative Control (rather than individualstudent scores) as the unit of analysis, thusreducing sporadic varia- tion and also meetingthe statistical require- ments of randomassignment. Several weeks beforethe experiment began, INSTRUCTIONAL MATERIALS the teaching supervisorof the district observed pogram (Covington, teacher and students were The Productive Thinking each class while the Crutchfield, and Davies,1966) is a set of in- having an extendeddiscussion about an inter- designed to developand esting event in socialstudies, an event which structional materials inter- strengthen the productivethinking skills of provided an opportunityfor free, creative grades of elementary and students.All students in the upper change between teacher school.The Program undertakesto develop classes discussed the sameevent, and on the skills the supervisor re- the student's abilityto use important basis of her observations, and strategies ofthinking and problemsolving, corded the extent towhich the overall class- understanding of the pro- (including the teacher)seemed seeks to enhance his room atmosphere cesses of his ownthinking, and aims to pro- to provide anenvironment that facilitatedand toward the part of the mote favorableattitudes and motivations fostered creative thinking on activities which involvethe use of the mind. students.The supervisorreported that she felt 3 Series One of this Program, which deals with Each lesson is centered about a complex General Problem Solving, served as the instruc- and engaging problem, presented in story form, tional materials in this study. which the student attempts to solve.The The skills taught by Series One of The Pro- problem features a mysterious occurrence, and ductive Thinking Program are those which are as the student works on this mystery, he is essential for the kind of complex and original called upon to generate ideas, to look for and thinking in which an individual strives to pro- evaluate the meaning of puzzling facts or dis- duce new ideas, fresh insights and under- crepancies in information, to ask questions, standings, and his own solutions to problems. and to practice other productive thinking skills. Hence the Program teaches the student how to On certain pages, the student writes out generate many ideas, especially clever and his questions and ideas.Feedback to his re- unusual ones, how to look at a problem in a sponses is then provided in following pages, different and more fruitful way if he gets where he finds a range of illustrative ideas Il stuck, " and how touse the implications of appropriate to the problem at this point.In crucial facts or events to gain new perspective this way, he is led to understand what consti- on a problem which seems to resist solution. tute relevant, fruitful, and original ideas, and Moreover, although productive thinking is not his readiness to generate such ideas is en- entirely a matter of logical reasoning, it is hanced.Each lesson is so designed that the obvious that reasoning and planfulness play student works gradually toward a solution to an important role in this kind of thinking. the problem and eventually is brought to dis- Consequently, the Program also shows the cover it by himself.Thus, he experiences student how to proceed in an organized and the pleasurable excitement which accompanies systematic way when attacking a problem, successful problem solution and acquires a how to clarify the essentials of a problem, and growing sense of competence in coping with how to pay attention to relevant facts and con- difficult and complex problems. ditions of the problem in evaluating one's In addition to giving the student guided ideas. practice in using productive thinking skills, In addition to teaching certain essential the lessons also give some direct, didactic thinking skills, the Program seeks to promote instruction about thinkingand about each of those motivations and attitudes that are re- the skills which the student is being taught. quired for effective performance in productive This direct instruction is incorporated into thinking tasks.It attempts to increase the each lesson as the central problem develops student's interest in and liking for activities and as the student works toward solution. In which involve the use of the mind, to build up addition, the major points taught in each les- his readiness to work on thinking tasks in a son are summarized for the student at the end persistent and concentrated way, and to of that lesson. strengthen his self-confidence in his ability The human element in this series of lessons to think. is provided by a continuous story-line built around the Cannon family of Elmtown, USA. Description of the Materials Lila and Jim Cannon are in the fifth grade and sixth grade, respectively, and their Uncle Series One of The Productive Thinking Pro- John is a high school science teacher who gram consists of 16 programed lessons, each occasionally dops some high-level detective of which is an individual booklet about 40 work for the Elmtown police.Uncle John pages long.The lessons have a cartoon-text teaches Jim and Lila how to become better format which serves to heighten interest and thinkers as they work on various "cases" to- at the same time permits clarity and emphasis gether. in the presentation of the material.(Four Jim and Lila are intended as counterparts sample pages from one of the lessons are pre- of the students who work on these lessons. sented in the Appendix. )The lessons are They are typical children with whom the stu- written primarily for fifth- and sixth-grade dent can identifylikeable and human.Their students and can be used by students with a thinking provides examples of how to attack wide range of intelligence within these grades. problems, andin some caseshow not to Each lesson is self-administering, permitting attack them.Jim and Lila are intended to the student to work at his own pace; virtually serve as stimulating "companions" for the all students complete a lesson within 50 reader as he pursues these intellectual adven- minutes. tures, and in many cases the student works on

4 the problems in concert with Jim and Lilafirst ceptualized as an intricate and highly complex the student generating his own ideas and ques- problem-solving process; one which requires tions, then Jim and Lila responding with theirs. the individual to use and to coordinate a number of different cognitive skills as he seeks to reach Administration of the Materials his goal, whether this goal is to create an in- sightful explanation, to design an effective The 16-lesson series requires a total of 16 course of action, or to produce a tangible prod- classroom hours of instruction time.In the uct.This complex problem-solving process in- present study, the lessons were administered volves:(1) the use of a number and variety of one per day for 4 days of each week; the re- cognitive skills, such as generating ideas, maining school day in each week was used for aYoiding premature judgment, and breaking makeups.Generally, each student worked in- mental "sets" in order to look at a problem in dividually at his own pace and had an assign- a new and different way; (2) the effective ment to which he turned when he had completed management and deployment of the various cog- the day's lesson. nitive skills by what might be called a "master" It was the intention of this study that the thinking skill.For example, this Master Think- teacher's role in the use of these materials be ing Skill would be used to determine, at a par- held to a minimum so that any posttest superi- ticular point in the problem-solving sequence, ority of students who had used the materials whether it would be more fruitful to suspend (as opposed to the controls, who had not) would critical judgment and give free reign to specu- result from the effectiveness of the materials lation and fantasy in a search for entirely new themselves, rather than from whatever effects ideas, or whether, in contrast, it would be might occur when they served as raw material more fruitful to examine systematically and fthr imaginative treatment by capable teachers. critically the ideas one already has; and (3) a Thus, the teachers were instructed only to high degree of involvement or dedication on the distribute the lessons and answer procedural part of the thinkerthe individual is highly en- questions.They were not to conduct class grossed with the creative task and finds it discussions based on the materials or otherwise meaningful and engaging. participate in the way they were used.One ex- Hence the measuring instruments used in ception to this policy was permitted in classes this study attempt to present the student with with a number of poor readers.In these classes, thinking tasks which he will find meaningful teachers often read portions of the lessons and engaging, and which require him to use aloud with small groups, orotherwise assisted and to coordinate a number of different cogni- students with substantial reading problems. tive skills.These tasks do not involve the use of specialized knowledge or technical skills, nor do they require the mastery of any particu- MEASURING INSTRUMENTS AND SCORING lar curriculum material. PROCEDURES Because of the complex thinking which these tasks involve, they are not "pure" in any fac- Most.of the measuring instruments used in torial sense and cannot be neatly classified this study were developed by the Creative as measuring thinking of any particular "type" Thinking Project at Berkeley.1Since they differ (e.g., "convergent" or "divergent").Instead) from traditional educational tests in both con- each test typically requires the interrelated tent and structure, it is helpful to present a use of several types of thinking.Thus the brief description of their underlying rationale classification of tests which is given below before describing the instruments themselves. merely indicates the relative emphasis given A more complete discussion of this rationale to a certain kind of thinking by the tests in may be found in Covington (1968). each category; it does not mean that the tests Briefly, the instruments are based on the in a certain category involve only one kind of proposition that creative thinking can be con- thinking. A complete list and classification of the tests of productive thinking used in this study 1Four tests in the battery (Circles, Squares, is presented in Table 1.In this classification, Truck Improvement, and Dog Improvement) were the terms "convergent" and "divergent" think- developed by Dr. E. Paul Torrance while at the ing are used in the traditional way to denote University of Minnesota. His permission to thinking which, on the one hand, involves pro- use those tests as part of this study is grate- cessing information so as to "converge" upon fully acknowledged. a single solution to a problem or, on the other hand, involves the use of information as a 5 Table 1 Classification of the Tests of Productive Thinking Used inthis Study

Name of test Use Performance indicators 1. Complex Extended Problems with anEmphasis on Convergent Thinking(all verbal)

Pit problem pretest number of ideas; quality of ideas(two indices); achievement of solution Jewel problem (short form) pretest number of ideas, quality of ideas(two indices); achievement of solution Rare Coins problem internal test achievement of solution Jewel problem (long form) posttest number of ideas; quality of ideas (two indices); achievement of solution Black House problem posttest number of ideas; quality of ideas (two indices); achievement of solution; number of odd facts or discrepancies in informationthat are noted X-ray problem posttest number of ideas; quality of ideas (two indices); achievement of solution; number of relevant questions asked or preparatory steps taken before work on problem was begun

1. Complex Extended Problems with anEmphasis on Divergent Thinking (all verbal)

Elevator problem pretest number of ideas The Deep Sea Dive internal test number of odd or puzzling facts within the problem that are noted; achievement of solution The Ancient City posttest number of ideas; quality of ideas (two indices); number of relevant questions asked or prepara- tory steps taken before work on the problem was begun A Visit to Karam posttest number of puzzling facts which are noted; intellectual persistence 3. Brief Problems with an Emphasis on ConvergentThinking (all verbal)

Ingenuity Items pretest achievement of solution (number of solutions (Form A) achieved: 3 possible) Ingenuity Items posttest achievement of solution (number of solutions (Form B) achieved: 10 possible) 4. Brief Problems with an Emphasis on DivergentThinking

Squares pretest number of ideas; quality of ideas (two indices) nonverbal Truck Improvement pretest number of ideas; quality of ideas (two indices) verbal

6 Table 1 (cont. ) 11"ame of Test Use Performance indicators Suppose People Could pretest number of ideas; number of inventions Read Minds verbal suggested A Summer Carnival pretest quality of ideas (two indices); number of (Form A) verbal available elements used when creating a product Circles posttest number of ideas; quality of ideas nonverbal (two indices) Dog Improvement posttest number of ideas; quality of ideas verbal (two indices) Suppose There was an posttest number of ideas; number of inventions Intelligence Pill verbal suggested A Summer Carnival posttest quality of ideas (two indices); number of (Form B) verbal available elements used when creating a product 5. Test of the Master Thinking Skill (verbal) The Missing Statue posttest total number of "best" or "second best" alternatives to which the student gives a rank of one; number of occasions on which the student chooses the "give up" alternative point of departure from which one's thinking ticular measures employed depended on the "diverges" into a variety of new mental prod- nature of the problem. The following example ucts."Complex Extended Problems" in the both illustrates the general nature of a Complex, classification are those which feature a gradual Extended Problem and presents a detailed dis- development of the problem and require the cussion of the rationale and procedures by which student to use a number of different cognitive performance was scored. skills as he proceeds.The student responds The problem, titled The Mystery of the Old at several points and is presented with feed- Black House, is presented to the student in the back in the form of more information about the form of a brief written story.The story begins problem after each response opportunity. (A as a detective drives out to the country to in- problem which uses these response-feedback vestigate an old black house in which gold is sequences is illustrated and discussed below reported to be hidden.The detective finds the in the section on Performance Indicators and house in the late afternoon and begins his Scoring. )Those tests labelled "Brief Problems" search, but he stops work just before sunset require the use of relatively few cognitive and goes to a nearby white house where he has strategies and do not employ response-feedback dinner and spends the night. When he awakens sequences; instead, they present the entire the next morning, the black house has com- problem to the student at once, and he then pletely disappeared, without leaving any trace makes his response(s). whatsoever.The problem is to explain how the All the problems are further classified as black house could have disappeared. Embedded verbal or non-verbal, and whether they were in the story are several puzzling discrepancies used as pre- or posttests or as internal tests (e.g., the detective saw the sunset through his in this study.The name of each test and the bedroom window but saw the sunrise through indices of thinking proficiency on which it was the same window when he awoke the next morn- scored are also listed. ing) which remain unaccounted for if one as- sumes that the black house must somehow have Performance Indicators and Scoring been torn down or moved away. However, the problem can be solved in a way that neatly ac- On each of the various tests, several mea- counts for these discrepancies by concluding sures of performance were obtained; the par- that it was actually the detective who was

7 rated with re- moved, in his sleep, to asimilar white house of working on the problem wes This, then, is the prin- spect to this scale. As inthe case of all some distance away. study, the ratings cipal solutionone which meetsall the require- quality ratings done in this and accounts for all the were made bytrained scorers, each ofwhom ments of the problem first "practice rated" 50previously obtained puzzling discrepancies aswell. protocols The story, containing allessentials of the protocolsand then exchanged these student on the with another scorer whorated them independ- problem, is presented to the ratings first page of a shortbooklet.On the next page, ently.The different sets of quality all the ideas he has were thencompared for reliability, andwhen he is asked to write down of reliability between scorers for explaining howthe black house could have a high degree He is encouraged to write as was evident theregular ratings were begun. disappeared. of protocols in many ideas aspossible, especially unusual Because of the large number the this study, the task ofrating the protocols ones.After writing his ideas, he turns divided among five page and then isasked to write down anyodd from any given test was scorers.In general, eachprotocol was rated or puzzling factshe has noticed in the story reliabilities could (besides the disappearance ofthe black house only once, so no interjudge provides feedback, be computed. However, asystematic program itself).The following page randomly selected protocols focusing the student'sattention on the several of rescoring sets of the story.Then he resulted in high interjudgeagreement.More- odd or discrepant facts in protocols were divided is given another opportunityto write down any over, because the black among the scorers on asemirandom basis, the new ideas hehas for explaining how the particular scorer house could have disappeared.Next, a suc- effect of a scoring bias by any should not affect the outcomesin a systematic cession ofquestion-response-feedback units scoring procedure gradually provides the studentwith more and way.In addition, a "blind" was used, suchthat the scorers had no wayof more informationabout the problem, giving him experimental condition a additional opportunities towrite down any new knowing from which how the black given protocol had beendrawn. All quality ideas he has for explaining made by the same house could have disappearedand leading him ratings in the study were five scorers. step-by-step toward the principalsolution. quality rating Finally, he is given alast opportunity to write When a scorer had assigned a the disappearance of to each of a student'sideas in a problem, two down ideas for explaining indices of quality were generallyrecorded: the the black house. number of "high quality"ideas (in the case of A number of indices ofthinking proficiency those receiving a can be applied toperformance on such a prob- the Black House problem, massive amount of rating of three or higher),and the quality rating lem, but because of the student produced. test data which this studyproduced, it IN7's of the best idea which the indices which were This general procedure forscoring and re- decided to use only those cording the quality of ideas wasused for all felt to be essential for areasonably compro- quality was scored as a per- hensive yet manageableanalysis of perform- measures in which ance.In this particularproblem, the following formance indicator. performance indicators wereused: whether or not the studentachieved the principalsolution Test of the Master ThinkingSkill number of odd facts or dis- to the problem; the battery requires crepancies in the problemwhich he noticed; One problem in the test pecial mention because ofits unique construc- the number of ideas he wrotefor explaining This problem, a test of the how the black housemight have disappeared tion and scoring. adequacy of these Master Thinking Skill, wasdesigned specific- (regardless of the quality or ability to manage ideas); and, finally, twoindices of the guality ally to measure the student's and deploy the variouscognitive operations of the ideas he produced. extended prob- The quality of an idea wasjudged on the involved in solving a complex imaginativeness ex- lem.The student is first presentedwith a basis of (1) the degree of problem. He then con- hibited and (2) the extent towhich it accounted statement of the basic fronts a "choice point" atwhich he is presented for the various factswithout violating the con- of action, straints of the problem.Based on previous re- with a list of five alternative courses normative scale of any of which mightbe taken at that point inthe search with this problem, a His task is to indicatewhich he quality which incorporatedthese two criteria problem. Each thinks is the "best" course ofaction, then the (presented in Table 2) wasprepared. by ranking these idea which the studentproduced in the course second best, and so on,

8 Table 2 Normative Quality Ratings for Ideas in The Mystery of the Old Black House

Rating Type of ideas Examples 0 Ideas which are irrelevant, impossible, The black house was never there at all; or contrary to fact a magician destroyed it

1 Ideas which explain the apparent The black house blew up; it was torn down disappearance of the black house, but which account neither for the fact that no trace of the house was found nor for the discrepancies in the story

2 Ideas which account for both the It was removed by a helicopter; it was apparent disappearance of the house moved by a truck and then the tracks were and for the fact that no trace of the covered; it was carefully camouflaged house was found, but still do not during the night explain the discrepancies in the story

3 Ideas which explain ihe apparent The detective was drugged during supper, disappearance of the black house, so he was confused when he woke up and and which account for all the facts only thought the black house had disap- and discrepancies in the story, but peared; the drugs made him see things can only do so by denying the reality strangely; it was all a dream of the problem

4 Elegant, feasible, ideas which account The principal solution: The detective was for all the facts and events moved to another highly similar white house during the night

alternatives from one to five.Next, he turns information).For the last three choice points, the page, and the development of the problem this "Jumping to conclusions" alternative is continues. After sufficient new information replaced by a "give up" alternative (in which has been presented, the student encounters a the student indicates that he wants to stop second "choice point" with its accompanying working on the problem and simply wants to list of alternative courses of action. Again, find out what the solution is). he indicates which of these courses he thinks Two different scores were used as perform- would be best, second best, and so on. There ance indicators: the number of times(summed is a total of six choice points, each with its over the six choice points) that the student own list of possible courses of action. gave a rank of one to either the "best" or the Each list of choices always contains the nsecond best"course of action,and the number following alternatives (in a random order, of of times he gave a rank of one to the "give up" course):a "best" course of action (the action alternative presented in the last three choice that would be most effective at this point), a points.The first score provides a measure of nsecond best"course of action (an action that the Master Thinking Skill per se, while the is reasonable, though not as effective as the second score constitutes a reciprocal measure "best" course), a "contrary to fact" course of intellectual persistence (i.e., the extent to of action (one that ignores or violates an al- which the student keeps on working on the ready established fact), an "appealing but problem when he is given several opportunities irrelevant" action (one that is attractive but to give up). Another measure of intellectual would not aid in solving the problem), and persistence was also obtained by the same finally, in the first three choice points, a general means in one other test A Visit to "jumping to conclusions" course of action Karam). (one which "forces" a conclusion to the prob- lem, but does so on the basis of insufficient

9 Measures of Understanding Thinking design, and standard analysis of variance pro- cedures were used for the analyses. In addition to measuring the student's per- Because the investigators also wished to formance on problem-solving tasks, an attempt look at the relationships between IQ and the was made to probe his understanding of the na- Treatment and Environment factors, a second- ture of problem solving by presenting him with ary analysis was performed, using a 3 X 2 X 2 three open-ended sentences which he was factorial design.Three levels of IQ were asked to complete.The sentences were: represented in this designhigh (above 115), "Making mistakes in working on a problem middle (101 through 115), and low (100 and is "; "Wild or silly ideas are below).The other two factors were Treatment- 11; and"In order to solve problems you Control and Facilitative-Nonfacilitative Environ- must .11 ment.After all subjects had been blocked into The student's answers were rated by scorers three groups on the basis of IQ, a random on a three point scale, indicating the degree sample of 20 was selected at each IQ level of insight or understanding which was shown within each Treatment X Eavironment combina- by the answer.For example, in the first sen- tion in the design.Thus, a total of 240 sub- tence above, a rating of zero was given to an jects was used for this secondary analysis. answer such as "Making mistakes in working (In order to obtain such a sample, it was nec- on a problem is bad." A rating of one was essary to ignore classifications according to given to answers such as "Making mistakes in sex and classroom units involved in the experi- working on a problem is something that happens ment. ) even to good thinkers." A rating of two was For boll the main analysis (the 2 X 2 X 2 given to an answer such as "Making mistakes factorial design) and the secondary analysis in working on a problem can show you where (the 3 X 2 X 2 factorial design involving IQ), you went wrong and lead you to new ideas." separate analyses of variance were performed Each of the three sentences was scored as on each of five stratification variables a separate variable and is referred to as socioeconomic status, overall mean score on Sentence I,II, or III, respectively, in the the Stanford Achievement Battery, IQ, perform- analysis. ance on the Sequential Test of Educational Progress (Social Studies, Form 4A) when given as a pretest, and scores on this test when EXPERIMENTAL DESIGN AND STATISTICAL ANALYSES given as a posttest (using Form 4B).Similar analyses were performed on all pretest, in- The basic analyses for this experiment ternal test, and posttest measures. were done with a 2 X 2 X 2 factorial design. In the second design, the analysis was The factors represented were: Instruction performed using scores of individual students (Treatment vs. Control); EnVironment (Facilita- as the basic unit, in contrast to the main de- tive vs. Nonfacilitative); and Sex.There were sign where the classroom mean score for each 11 classrooms within each of the cells of this variable was the unit of analysis.

10 111 RESULTS

Tables 3, 4 and 5 present the means and with the exception of socioeconomic status, standard deviations of all the variables ana- the mean differences between Facilitative and lyzed in this experiment.Values for the strati- Nonfacilitativ.e classes on these stratification fication variables are in Table 3, for the pre- variables were fairly small, although statis- test measures in Table 4, and for the internal tically significant.Means for the Treatment- and posttest measures in Table 5. Control and Facilitative-Nonfacilitative The results of the analyses of variance for Environment groups on the stratification vari- the stratification variables are given in Table ables are preserted in Table 7. 6.Note that there were no significant Treat- The data in Table 6 also indicate significant ment-Control differences on any of these sex differences in achievementand IQ. These variables. The Environment factor, on the differences arise from the fact that girls are other hand, represented a significant source superior to boys on these two variables(Table of variance.on all five of these variables. 8).(Although differences on the two forms of Hence the supervisor's ratings of what con- the STEP test were not significant, they were stituted a "facilitative" classroom environ- in the same direction.) These differences are ment reflected, to some extent, thecalibre consistent with other findings concerning sex of students in the class being rated. However, differences in aptitude and achievementat this level of development.

Table 3 Means and Standard Deviations of StratificationVariables

Standard Variable Mean Deviation

Socioeconomic status 2.98 1.49 Stanford achievement 5.09 .65 Kuhlmann-Anderson IQ 107.39 7.68

STEP Test %Pretest: Form 4A) 26.44 3.22 STEP Test (Posttest: Form 4B) 25.48 3.15

11 Table 4 Means and Standard Deviations ofPretest Variables

Mean Standard Variable Deviation Extended problems in convergentthinkinga

Pit problem: 20 Achievement of solution .42 3. 63 .74 Number of ideas .37 'Number high quality ideas 1. 08 Quality rating, best idea 3. 02 .43

Jewel problem: Achievement of solution 06 09 3.53 1. 06 Number of ideas .21 Number high quality ideas .37 Quality rating, best idea 1.10 .35 Brief problem in convergentthinkinga

Ingenuity test: .26 Achievement of solution .48 Extended problem in divergentthinkinga

Elevator problem: 1. 08 Number of ideas 2. 92 Brief problems in divergent thinking

Squares problem (nonverbal): 2. 49 Number of ideas 8. 45 ideas 1. 92 .83 Number high quality .26 Quality raling, best idea 1. 65

Truck improvem en t problem (verbal): 2. 30 Number of ideas 8. 04 1.42 .62 Number high quality ideas .22 Quality rating, best idea 1.64 Mind-reading problem (verbal): 3. 94 1. 43 Number of ideas .20 .25 Inventiveness

Summer carnival problem (verbal): .40 .29 Number high quality ideas 6.10 .30 Quality rating, best idea .59 Ability to synthesize elements 2.10

aProblems in this group are allverbal.

12 Table 5

Means and Standard Deviationsof Internal and Posttest Variables Standard Mean Variable Deviation

Internal testmeasuresa

Rare coins problem(extended, convergent): .19 Achievement of solution .21 (extended, divergent): Deep Sea Dive problem .27 .19 Achievement of solution .57 Sensitivity to incongruousinformation 1.46 Extended problems in convergentthinkinga

Jewel problem: .51 .25 Achievement of solution 6.18 1.57 Number of ideas 1.29 .35 Number high quality ideas .45 Quality rating, best idea 2.23

House problem: .65 .20 Achievement of solution 4.18 .79 Number of ideas 1.17 .38 Number high quality ideas 4.09 .58 Quality rating, best idea .36 Sensitivity to incongruousinformation .53

X-ray problem: .15 .11 Achievement of solution .83 Number of ideas 2.79 ideas .82 .41 Number high quality 2.58 .45 Quality rating, best idea .44 Questioning and clarifyingability .63

Brief problem in convergentthinkinga

Ingenuity: .99 Achievement of solution 3.27 Extended problems in divergentthinkinga

Ancient city problem: 4.79 1.54 Number of ideas ideas .35 .20 Number high quality 1.00 .33 Quality rating, best idea .58 Questioning and clarifyingability .87

A Visit to Karam: .74 .78 Sensitivity to puzzling facts .84 Intellectual persistence 2.46 aProblems in this group are allverbal. (Continued on next page) 13 Table 5 (coned) Standard Variable Mean Deviation

Brief problems in divergent thinking Circles (nonverbal): Number of ideas 14.08 2.52 Number high quality ideas 2.08 .75 Quality rating, best idea 1.76 .18 Dog improvement (verbal): Number of ideas 7.92 2.18 Number high quality ideas 1.19 .60 Quality rating, best idea 1.54 .24 Intelligence pill (verbal): Number of ideas 2.81 .87 Inventiveness .06 . 09 Summer carnival (verbal): Number high quality ideas .47 .26 Quality rating, best idea 1.16 .32 Ability to synthesize ehements 1.80 .61 Special tests Statue problem: Effectiveness of Master Thinking Skill 3.53 .66 Intellectual persistence 1.59 .80 Understanding thinking: Sentence I .66 .37 Sentence II 1.00 .28 Sentence III 1.24 .23 aProblems in this group are all verbal. Table 6

Summary of Analyses of Variance forStratification Variables (df = 1,80)

Treatment Environment Sex Variable MS MS MS Socioeconomic status .05 <1 24.04 11.58** .00 < 1 7.81** Stanford achievement .01 <1 2.09 5.64* 3.01 Kuhlmann-Anderson IQ .75 <1 484.02 9.37** 454.70 8.81** < 1 STEP Pretest .17 <1 47.39 455* 6.60 STEP Posttest 6.36 <1 42.62 4.30* 3.95 < 1 *p < . 05. **p < . 01.

14 Table 7 Mean Level of Performance of Students in Treatment and Control Groups and in Facilitative and Nonfacilitative Environments: Stratification Variables

Variable Group Environment Treatment Control Facilitative Nonfacilitative

Socioeconomic status** 2.95 3.00 2.45 3.50 Stanford achievement* 5.11 5.09 5.25 4.93 Kuhlmann-Anderson IQ** 107.49 107.31 109.74 105.05 STEP Pretest* 26.40 26.49 27.18 25.71 STEP Posttest* 25.22 25.77 26.18 24.79 *"Environment" difference significant at p < .05. **"Environment" difference significant at p < .01.

Table 8 Mean Level of Performance of Males and Females: Stratification Variables

Variable Males Females Socioeconomic status 2.98 2.98 Stanford achievement* 4.91 5.28 K-A IQ* 105.12 109.67 STEP Pretest 26.17 26.72 STEP Posttest 25.27 25.70 *Difference significant at p < .01

PRETEST RESULTS all reflect the superior performance of students in the Facilitative environment. In Table 9 is a summary of the analyses of Tables 12 and 13 present the means for the 21 pretest variables.In the 21 analyses, males and females on the pretest measures. there was only one Treatment effect significant Except for the Pit problem, where males were atthe .05 level. It seems likely that this sig- superior to females on three of the four per- nificant F is simply an artifact of the number of formance indicators, females performed better analyses carried out. The means for treatment than males for all measures on which statis- and control group's are presented in Table 10. tically significant sex differences were found. The Environment factor produced significant These, then, were the differences which differences on five pretest measures, all of existed among fifth-grade children on various which concerned the two brief problems in di- measures of productive thinking before any vergent thinking-the nonverbal Squares prob- special training was administered.Let us lem and the verbal Truck problem. Inspection turn now to the results obtained after Series of the means in Table 11 reveals that these One of The Productive Thinking Program had differences are all in the same direction, and been administered,

15 Table 9 Summary of Analyses of Variance of Pretest Variables (1 and 80 degrees of freedom)

Main effects Treatment Environment Sex Variables MS F MS F MS Extended problems in convergentthinkinga Pit problem: Achievement of solution .00 < 1 .00 < 1 .41 11.59** Number of ideas 1.34 2.59 1.93 3.74 2.21 4.27* Number high quality ideas .06 < 1 .07 < 1 1.18 9.31** Quality rating, best idea .02 < 1 .00 < 1 2.16 13.07** Jewel problem: Achievement of solution .04 5.46* .00 < 1 .01 1.78 Number of ideas .54 < 1 2.42 2.11 .49 < 1 Number high quality ideas .03 < 1 .11 < 1 .07 < 1 Quality rating, best idea .38 3.17 .11 < 1 .07 < 1 Brief problem in convergentthin'inga Ingenuity test: Achievement ofsolutionb 00 < 1 .25 3.68 .22 3.34 Extended problem in divergentthinkinga Elevator problem: NuMber of ideas .57 < 1 .07 < 1 3.87 3.19 Brief problems in divergent thinking Squares problem (nonverbal): Number of ideas 22.76 3.92 23.66 4.08* 19.64 3.38 Number high quality ideas .26 < 1 5.73 8.90** .80 1.24 Quality rating, best idea .00 < 1 .32 4.70* .15 2.21 Truck improvement problem (verbal): Number of ideas . 00 < 1 .98 < 1 4.91 < 1 Number high quality ideas .00 < 1 2.04 5.37* .24 < 1 Quality rating, best idea .03 < 1 .27 5.56* .02 < 1 Mind-reading (verbal): Number of ideas 05 < 1 .07 < 1 15,54 8.38** Inventiveness .00 < 1 .01 < 1 .15 2.45 Summer carnival (verbal): Number high quality ideas .01 < 1 .03 < 1 .47 5.67* Quality rating, best idea .01 < 1 .01 < 1 .47 5.10* Ability to synthesize elements .05 < 1 .01 < 1 5.77 19.11** aProblemsin this group are all verbal. bNumber of solutions out of a possible three. *p <05 **p < . 01

16 Table 10

Mean Level of Performance of Treatment and Control Groups:Pretest Variables

Variable Treatment Control

Pit problem: Achievement of solution .43 41 Number of ideas 3. 75 3. 51 Number high quality ideas 1. 05 1.10 Quality rating, best idea 3. 00 3. 03 jewel problem: Achievement of solution . 08 .04 Number of ideas 3. 61 3. 45 Number high quality ideas .39 .36 Quality rating, best idea 1. 16 1. 04 Ingenuity test: Achievement of solution . 48 .48

Elevator problem: Number of ideas 3. 00 2. 84

Squares problem: Number of ideas 8. 96 7. 94 Number high quality ideas 1. 98 1. 87 Quality rating, best.idea 1. 66 1.64

Truck improvement problem: Number of ideas 8. 03 8. 04 Number high quality ideas 1. 43 1. 42 Quality rating, best ideas 1.65 1. 62 Mind-reading problem: Number of ideas 3. 97 3. 92 Inventiveness .20 .21

Su Mmer carnival: Number high quality ideas 42 39 Quality rating, best idea 1.11 1. 09 Ability to synthesize elements 2.12 2. 08

17 Table 11 Comparison of Ideas Generated by Students in Facilitative (F) and Nonfacilitative(N) Environments: Pretest Variables

Fluency No. high quality Quality of best Problem 3.48 3.78 1.05 1.11 3.01 3.02 Pit 1.06 jewel 3.36 3.70 .40 .35 1.13 Elevator 2.94 2.89 8.97* 7.93* 2.18** 1.67** 1.71* 1.59* Squares 1.58* Truck 8.14 7.93 1.58* 1.27* 1.69* No. of Ideas 3.97 3.91 Me. Mind:Inventiveness .21 MO. 1.01 1.11 Carnival 2.11a 2..0199a .42 .38 aNot a measure of fluency.This entry represents the number ofavailable elements used when creating a "product." *Difference significant at p < . 05. **Difference significant at p < .01.

Table 12

Mean Level of Achievement of Solutionfor Males and Females: PretestVariables

Test Males Females

Pit* .49 .35 Jewel 07 05 Ingenuity .53 .43 *Difference significant at p <01

Table 13

Comparison of Ideas Generated byMales (M) and Females (F): PretestVariables Fluency No. high quality Quality of Best

3.47* 3.79* 1.19** .96** 3,17** 2.86** Pit 1.13 1.07 jewel 3.46 3.61 .39 .36 Elevator 2.71 3.12 7.97 8.92 1.83 2.02 1.61 1.69 Squares 1.65 1.62 Truck 7.80 8.28 1.48 1.37 No. of Ideas 3.52** 4.36** SON MO. Mind: .16 .24 MO. Inventiveness .33* 1.17* Carnival 1.84a**2.36a** .48* 1.03* aNot a measure of fluency.This entry represents the numberof available elements used when creating a "product." *Difference significant at p < .05. **Difference significant at p < .01.

18 Table 14 Summary of Analyses of Variance: Main Effects for Internal and Posttest Variables (1 df for effects)

Source Treatment Environment Sex Variable MS F MS F MS Internal Measuresa Rare coins problem: Achievement of solution .02 < 1 .07 2.15 .00 < 1 Deep sea dive problem: Achievement of solution .05 1.46 .00 < 1 .00 < 1 Sensitivity to incongruous 2.25 7.41** . 00 < 1 1.25 4.11* information Extended problems in convergentthinkinga Jewel problem:c Achievement of solution .11 1.61 .01 < 1 .01 < 1 Number of ideas 13.46 6.59* .05 < 1 4.78 2.34 Number high quality ideas .25 2.37 .02 < 1 .37 3.50 Quality rating,best idea .62 2.80 .16 < 1 .04 < 1 Houseproblem:d Achievement of solution .23 6.47* . 11 2.98 04 1.15 Number of ideas .54 < 1 .14 < 1 1.61 2.52 Number high quality ideas 2.05 16.99** .03 < 1 .57 4.71* Quality rating, best idea 1.84 5.98* 1.13 3.67 .52 1.68 Sensitivity to incongruous 1.50 15.65** .06 < 1 .67 7.00** information X-rayproblem:d Achievement of solution .16 6.20* .03 1.04 .00 < 1 Number of ideas .00 < 1 .01 < 1 1.05 1.45 Number high quality ideas .45 2.73 .01 < 1 .01 < 1 Quality rating, best idea .99 5.22* .00 < 1 .02 < 1 Sensitivity to incongruous .50 2.83 .00 < 1 1.70 9.59** information Brief problem in convergentthinkinga Ingenuity:d Achievement ofsolutionb .45 < 1 4.84 4.89* .92 < 1 Extended problems in divergent thinkinga Ancient city problem:c Number of ideas 22.66 10.97** 1.59 < 1 .11 < 1 Number high quality ideas .01 < 1 00 < 1 08 1.88 Quality rating, best idea .16 1.39 . 00 < 1 .18 1.60 Questioning and clarifying ability .06 < 1 .49 1.47 1.15 3.44 aproblems in this groupare all verbal. bTotal number of solutionsout of a possible ten. c36 df for error. d80 df for error. < . 05 4c*p <.0 1

(Continued on next page) 19 Table 14 (cont'd) Source Treatment Environment Sex Variable MS MS MS

A Visit to Karam:c Sensitivity to puzzling facts 1.32 2.12 .00 < 1 .00 < 1 Intellectual persistence 2.74 4.15* 1.78 2.69 .23 < 1 Brief problems in divergent thinking d Circles (nonverbal): 9.37** 10.00 1.68 5.02 < 1 55.63 Number of ideas < 1 Number high quality ideas .08 < 1 .53 < 1 .37 <1 < 1 Quality rating, best idea .04 1.39 .01 .00 Dog improvement(verbal):d Number of ideas 10.98 2.72 6.20 1.53 57.05 14.13** Number high quality ideas .34 1.03 .57 1.71 1.52 4.55* Quality rating, best idea .00 < 1 .08 1.38 .34 1.09 Intelligence pill (verbal):d Number of ideas .54 < 1 .79 1.20 6.61 9.99** Inventiveness .38 4.33* .00 < 1 .00 < 1 Summer carnival (verbal):d Number high quality ideas .02 < 1 .02 < 1 .98 16.97** Quality rating, best idea .01 < 1 .01 < 1 1.70 20.49** Ability to synthesize elements .22 < 1 .01 < 1 4.99 15.41** Special tests Statueproblem:d Effectiveness of Master Thinking Skill 1.97 5.06* .14 < 1 1.59 4.08 Intellectual persistence .04 < 1 .09 < 1 1.50 < 1 Understanding thinking:c Sentence I .68 6.79* .33 3.27 .81 8.12** Sentence II .13 1.58 .01 < 1 .28 3.42 Sentence III .08 1.56 .06 1.17 .18 3.45 aProblems in this group are all verbal. bTotal number of solutions out of a possible ten. 36 df for error. d80 df for error. *p < .05. **p < . 01.

INTERNAL AND POSTTEST RESULTS which permits only a small set of "best" solu- tions).2 Two of the six measures show sig- Table 14 summarizes the results of the nificant differences, and on five of the six analyses of variance of all internal and post- measures performance of the Treatment group test measures.Of particular interest are the exceeds that of the Controls. Treatment-Control differences, thirteen of The sixth measure, involving the X-ray which are statistically significant (as opposed problem, requires special comment since this to only one such difference that reached sig- problem was the only one in the entire posttest nificance on the pretest measures). Table 15 presents the means for the Treatment and 2 The Deep Sea Dive problem, although it em- Control groups on incidence of achievement phasizes divergent thinking, was included in of solution for problems which have an empha- this Table because it, too, permits only a sis on convergent thinking (the type of problem small set of "best" solutions.

20 Table 15 lem) favored the Controls. The means for Treatment-Control performance on all remaining Mean Level of Achievement of Solution for internal and posttest measures are presented in Treatment and Control Groups: Table 17,and all significant differences in Internal and Posttest Variables this table were the result of superior perform- ance by the Treatment group. Problem Treatment Control In brief, then, Series One of The Productive Thinking Program produced significant Treatment- Rare coins .22 .19 Control differences on one-third (13 out of40) Deep sea dive .29 .24 of the internal and posttest measures,when only Jewel .56 .45 ono significant difference(out of 21 measures) House* .70 .59 existed before training began.Eleven of the X-ray* .11 .19 significant posttest differences indicated supe- Ingenuity 3.34 3.20 rior performance by the Treatment group,while the two differences which did notboth involved *Difference significant at p < . 05. the maverick X-ray problem. Turning now to differences resulting from the battery on which the Control group consistently two types of classroom environment,Table 14 outperformed the Treatment group by a statis- shows that only one such difference(out of the tically significant margin.This unexpected 40 posttest analyses) was statisticallysignifi- reversal of the general pattern of Treatment cant.This is a considerable contrast to the group superiority appears to be anartifact of pretest resultsin which five significant dif- this particular problem, most likely due to the ferences (out of 21 analyses) were found, all fact that the task was so difficult (only15% of of which indicated the superiority of aFacilitative all students ever achieved solution) that an environment.The notable lack of posttest ef- outstanding performance by relatively few Con- fects may partly be accounted for by the pres- trol students could produce significant differ- ence of several significantTreatment x Environ- ence in favor of that group. ment interactions (see below), allof which On measures dealing with the number and indicate that treatment was more effective with quality of ideas (Table 16), six significant students in Nonfacilitative environments, thus Treatment-Control differences were found. Five reducing the initial differences between Facili- of these differences favored the Treatment group, tative and Nonfacilitative groups. Several addi- while the sixth (again involving the X-rayprob- tional Treatment x Environment interactions,

Table 16

Comparison of Ideas Generated by Studentsin Treatment (T) and Control (C) Groups: Posttest Variables Fluency No. high quality Quality of best Problem

Jewel 6.77* 5.58* 1.37 1.21 2.35 2.11 House 4.26 4.10 1.32** 1.01** 4.23* 3.94* X-ray 2.79 2.79 .75 .89 2.48* 2.69* Ancient city 5.22** 4.06** .36 .34 1.06 .95 Circles 14.42 13.75 2.05 2.11 1.74 1.78 Dog 8.28 7.57 1.26 1.13 1.54 1.53 No. of ideas 2.73 2.88 Pill::nventiveness .08* .03* Carnival 1.75a 1.85a .45 .48 1.15 1.17 aNot a measure of fluency.This entry represents the number ofavailable elements used when creating a "product." *Difference significant at p < .05. **Difference significant at p < .01.

21 Table 17

Means for Treatment and Control Groups on Remaining Internaland Posttest Measures

Variable Treatment Control Deep sea dive problem: Sensitivity to incongruous information 1.63* 1.29** House problem: Sensitivity to incongruous information .66* .40** X-ray problem: Sensitivity to incongruous information .55 .71 Ancient city problem: Questioning and clarifying ability .90 .84 A Visit to Karam: Sensitivity to.puzzling facts .90 .58 Intellectual persistence 2.70* 2.22* Statue problem: Effectiveness of Master Thinking Skill 3.73* 3.32* Intellectual persistence 1.56 1.61 Insight and understanding of thinking: Sentence I 78* .53* Sentence II 1.05 .95 Sentence III 1.28 1.20 *Difference significant at p < .05. **Difference significant at p < .01. though they failed to reach significance, also Facilitative environments, while for the Control showed this pattern. group, students in the Facilitative environments Each of the significant sex differences re- performed better than those in the Nonfacilita- ported in Table 14 (12 in 40 analyses) reflects tive environments.Thus, it would seem that the superior performance of females. Seven one important effect of the training materials of the 12 differences occurred on variables was to enable students from less favorable dealing with the production of ideas (see Table classroom climates to overcome the detrimental 18), threodifferences reflected the greater sen- effects that such environments would otherwise sitivitytof females to discrepant or puzzling have produced on their performance.This helps information em:Jedded in the tests, and the explain the fact (noted earlier) that classroom remaining two differences occurred on a variety environment was a significant source of variance of measures. With respect to the verbal fluency on 5 out of 2'1 pretest measures, but on only 1 measures, the data are consistentwith the out of 40 posttest measures. numerous findings that girls areverbally more proficient than boys. EFFECTS OF IA In the analyses of all these measurespre- test, internal test, and posttestthere was a The second major analysis, carried out with total of seven significant interactions, all in the 3 X 2 X 2 design, involved three levels of the internal and posttests.Two of these, IQ (above 115, 101-115, and 100 or below), the three-way interactions of Treatment X Environ- two Treatment conditions, and the two types of ment X Sex, were uninterpretable.The other Environment.The purpose of this analysis was five were Treatment X Environment interactions, to determine the effect of IQ on performance, for which the means are presented in Table 19. both as a main effect and in its interactions The interactions for these five measures all with Treatment or Environment. Since the main have the same pattern: within the Treatment effects of the Treatment and Environment factors condition those students in Nonfacilitative have already been discussed, they will not be environments performed better than students in considered in connection with this analysis.

22 Table 18 Comparison of Ideas Generated by Males (M) and Females (F):Posttest Variables

Problem Fluency No. high quality Quality of best

Jewel 5.84 6.51 1.20 1.38 2.20 2.27 House 4.04 4.31 1.08* 1.25* 4.01 4.16 X-ray 2.68 2.90 .81 .83 2.60 2.57 Ancient city 4.74 4.84 .31 .39 .94 1.06 Circles 13.29** 14.88** 2.14 2.01 1.76 1.76 Dog 7.12 ** 8.73** 1.06* 1.32* 1.47 1.60 Pill 2.53 P:C 3.08** - - - - Carnivala 1.56** 2.03** .36 Pl.,* 57** 1.02 *VA 1.30** a "Fluency" means reflect the number of elements combined to generatea "product." *Difference significant at p <.05. PiC P:c1) if f erenc e significant at p <.01.

Table 19 Means for Significant Treatment X Environment Interactions (1 df for Interaction)

Variable df for Environment error Facilitative Nonfacilitative

Rare coins problem (internal test): 80 Achievement of solution Treatment .23 .22 Control .28 .12 Jewel problem (posttest): 36 Number of ideas Treatment 6.25 7.21 Control 6.03 5.21 Dog problem (posttest): 80 Number of ideas Treatment 8.10 8.46 Control 8.28 6.86 Number high quality ideas Treatment 1.20 1.31 Control 1.34 .92 House problem (posttest): 80 Sensitivity to incongruous information Treatment .54 . 79 Control .48 .33

Table 20 presents the results of this IQ and STEP tests than do middle IQ students, who analysis for the stratification variables. With in turn perform better than those with low IQ. the exception of socioeconomic status, perform- With respect to the pretest measures, Table ance on each of these variables was signifi- 22 indicates that IQ represented a significant cantly related to IQ (p <.01). Table 21 indi- source of variance for 17 of the 21 measures cates that the mean differences for these 415 of these beyond the. 01level).The mean variables are all in the predicted direction- levels for achievement of solution to the three high IQ students perform better on the Stanford convergent thinking problems used in the pretest

23 the low IQ group. For measuresof the quantity are presentedin Table 23.In each of these produced on the various problems, the high IQ groupoutperformed the and quality of ideas with one tests, this samerank-order relation between others, and the middle IQ group, found (Table 24). small reversal, wasconsistently superior to IQ and performance was

Table 20

Summary of Analyses ofIQ as a Main Effect: Stratifying Variables (df for Effect = 2)

df for IQ Variable Error MS

Socioeconomic s ta tus 228 .11 < 1 154.58* Stanford achievement 225 108.66 215 922.08 46.34* STEP Pretest 55.66* STEP Posttest 215 1049.46 *p < . 01

Table 21

Means of IQ Groups onStratifying Variables IQ Variable High Middle Low 2.84 2.88 2.91 Socioeconomic status 3.96 StanfOrd achievement* 6.32 5.13 29.73 27.79. 22.92 STEP Pretest* 21.78 STEP Posttest* 29.18 26.75 *Differences significant at p < .01.

24 GPO 60641SMIS

APO, Table 22

Summary of Analyses of Main Effect ofIQ: Pretest Variables (df for Effect = 2)

df for TO effect Variable error MS

Extended problems in convergentthinkinga 221 Pit problem: 5.67** Achievement of solution 1.32 5.68 2.40 Number of ideas 5.48** Number high quality ideas 5.15 Quality rating, best idea 9.62 9.16** Jewel problem: 225 Achievement of solution .46 6.92** 10.82** Number of ideas 41.25 Number high quality ideas 2.47 7.10** Quality rating, best idea 9.51 12.52** Brief problem in convergentthinkinga Ingenuity test: 224 Achievement of solution 6.68 17.54** Extended problem in divergentthinkinga 221 Elevator problem: 10. 80** Number of ideas 38.39 Brief problems in divergentthinking 221 Squares (nonverbal): 26.76** Number of ideas 489.51 47.83 15.92** Number high quality ideas 12.700:: Quality rating, best idea 3.62 Truck improvement (verbal) 221 226.61 13.52** Number of ideas 2.52 Number high quality ideas 5.34 Quality rating, best idea .46 1.67 Mind-reading (verbal): 221 142.75 26.39** Number of ideas 2.38 Inventiveness .68 Summer carnival (verbal): 225 Number high quality ideas 1.17 3.99* 2.24 4.31* Quality rating, best idea 5.26** Ability to synthesize elements 7.62 aProblems in this group are all verbal. bTotal number of solutions out of apoegible three. *ID <05 p < . 01.

25 Table 23 Tables 26, 27 and 28; in all cases, the high and middle IQ groups performed substantially Mean Level of Achievement of Solution for better than the low IQ group.However, per- IQ Groups:Pretest Variables formance of the high and middle IQ groups on these posttest measures was much more alike IQ (relative to the low IQ group) than it had been Problem High Middle Low in the pretest.The absence of any systematic IQ x Treatment interaction (see below) sug- gests that this change is not of sufficient Pit* .55 .41 .28 magnitude to be considered an important result Jewel* .16 .03 04 of exposure to the program. As in the pretest, Ingenuity* .86 .46 .30 there was a general rank-order relation between *Differences significant at p < . 01 IQ and performance on most of the posttest measures. A total of nine significant interactions oc- curred in this second analysis, four involving pretest measures and the other five involving Turning now to the effect of IQ on the post- the posttest.The means for these significant test measures (IQ effects on the three internal interactions are presented in Tables 29 and 30. test measures were not analyzed), Table 25 No consistent pattern appears to be present in shows significant differences on 27 of the 38 these interactions.(Two posttest interactions measures (19 of these beyond the.01 level). are omitted, one because it is anIQ X Treat- For problems emphasizing convergent thinking, ment X Environment interaction whichyields significant differences occurred on 13 of the no meaningful interpretation, theother because 16 measures; for problems emphasizing diverg- it is a Treatment X Environment interaction ent thinking, significant differences werefound which does not involve IQ and about which on 13 of the 17 measures; and on 1 ofthe re- information is best obtained from the first maining 5 measuresa significant difference analysis.) was found.These means are presented in

Table 24 Comparison of Ideas Generated by IQ Groups:Pretest Variables

Fluency No. high quality Quality of best Problem

Pit 3.90 3.88 3.41 1.35* 1.09* .83* 3.29* 3.05* 2.59* Jewel 4.31*3.49* 2.86* .58* .33* .23* 1.48* 1.08* .78* Elevator 3.64*3.24* 2.27* - - - - 10.97*8.30* 5.87* 2.62* 1.76* 1.04* 1.86* 1.69* 1.42* Squares 1.54 Truck 9.21*8.73 6.04* 1.60 1.41 1.08 1.69 1.66 Mind. 4.99*4.45 2.37* a Carnival 2.10* 1.46* .48* 34* .23* 1.21* 1.11* .88*

a"Fluency" means reflect the number of elementscombined to generate a "product." *Differences significant at p < . 05.

26 Table 25 Summary of Analyses of Main Effect of IQ: Posttest Variables (df for Effect = 2)

df for IQ effect Variable Error MS F a Extended problems in convergent thinking Jewel problem: 95 Achievement of solution .57 2.50 Number of ideas 20.37 3.50* Number high quality ideas 3.59 3.85* Quality rating, best idea 2.41 3.07 House problem: 216 Achievement of solution 3.85 22.72** Number of ideas 20.67 570** Number high quality ideas 7.14 11.93**. Quality rating, best idea 38.56 24.79** Sensitivity to incongruous information 5.91 8.45** X-ray problem: 215 Achievement of solution I .05 1.18 Achievement of solution IIb 1.24 10.41** Number of ideas 42.18 18.07** Number high quality ideas 9.89 13.45** Quality rating, best idea 12.44 15.04** Questioning and clarifying ability 4.75 9 . 1 Brief problem in convergentthinkinga Ingenuity test: 204 Achievement of solution 100.50 35.43** a Extended problems in divergent thinking Ancient city problem 9 5 Number of ideas 42.64 7.14** Number high quality ideas 1,49 4.68* Quality rating, best idea 3.50 5.26** Questioning and clarifying ability 5.21 3.44* A Visit to Karam 98 Sensitivity to puzzling facts .32 Intellectual persistence 13.00 6.09**

aProblems in this group are all verbal. bThese two variables represent a breakdowninto separate solutions of the measureof achievement of solution reported for this problem in theearlier analysis. *p < . 05 01

(Continued on next page)

27 Table 25 (cloned) df for ICI effect- Variable Error MS Brief problems in divergent thinking Circles (nonverbal): 222 Number of ideas 198.30 10.07* Number high quality ideas 16.62 5.47** Quality rating, best idea .47 2.49 Dog improvement (verbal): 222 Number of ideas 87.04 5.01** Number high quality ideas 4.96 2.78 Quality rating, best idea .75 2.49 Intelligence pill (verbal): 223 Number of ideas 20.90 7.96** Inventiveness .23 344* Summer carnival (verbal): 223 Number high quality ideas 1.80 4.23* Quality rating, best idea 2.11 3.85* Ability to synthesize 8.57 6.56** Special testsa Statue problem: 110 Effectiveness of Master Thinking Skill 6.37 2.50 Intellectual persistence 1.11 <1 Insight and understanding of thinking: Sentence I 74 .94 9.37** Sentence II 85 .14 2.11 Sentence III 81 .17 3.01

aProblems in this group are all verbal. bThese two variables represent a breakdown into separate solutions of the measure of achievement of solution reported for this problem in the earlier analysis. *p <05 **p < . 01

Table 26 Mean Level of Achievement of Solution for IQ Groups: Posttest Variables

IC Problem High Middle Low

jewel .67 .49 .34 House .86* .78* .43* X-raya . 07,.29* .03,.12* .03,.04* Ingenuity 437* 3.59* 2.03* aThese two variables represent a breakdown into separate solutions ofthe measure of achievement of solution reported for this problem in the earlier analysis. *Differences significant at p =. 01.

28 Table 27 Comparison of Ideas Generated by IQ Groups: Posttest Variables

Fluency No. high quality Quality of best Problem H M L H M L H M L jewel 7.00* 6.23* 5.49* 1.52* 1.46* .89* 2.52 2.28 1.89 House 4.23* 4.71* 3.68* 1,33* 1.38* .83* 4.68* 4,49* 3.35* X-ray 3.41* 3.13* 2.00* 1.19* .99* 48* 3.11* 2.65* 2.24* Ancient city 5.88* 5.03* 3.86* .61* .41* .17* 1.30* 1.10* .69* Circles 15.13*14.36*12.06* 2.37* 2.18* 1.49* 1.82 1.78 1.68 Dog 8.65* 7.68* 6.55* 1.29 1.24 .83 1.62 1.58 1.44 No. of ideas 3.24* 2.79* 2.22* ------Pill: Inventiveness .11* .04* .01* ------1.00* Carnivala 2,f 10* 1.92* 1.46* .62* .47* .32* 1.30* 1.24* a"Fluency" means reflect the number of elementscombined to generate a "product." *Differences significant at p < . 05.

Table 28

Means for Remaining Posttest Variables forwhich Significant IQ Effect was Found

IQ Variable H NI L

House problem: Sensitivity to incongruous information .92 .55 .38 X-ray problem: Questioning and clarifying ability .87 .51 .39 Ancient city problem: Questioning and clarifying ability 1.30 1.10 .57 A Visit to Karam Intellectual persistence 3.16 2.84 1.77 Insight and understanding Sentence I .94 '.76 .29

29 Table 29 Means for Significant Interactions Involving IQand Treatment

IQ Variable

jewel problem: No. high quality ideas Treatment .78 .32 .21 Control .38 .32 .26 Squares problem: Number of ideas Treatment 11.40 7.20 6.57 Control 10.51 9.40 5.20 Pill problem: Inventiveness Treatment .23 .05 .03 Control .00 .03 .00 Summer carnival problem: No. high quality ideas Treatment .70 .38 .23 Control .25 .30 .24

Table 30

Means for Significant Interactions InvolvingIQ and Environment

IQ Variable

Truck problem: Number of ideas Facil. Env. 8.05 8.60 6. 47 Nonfac. Env. 10.39 8.85 5.60 Summer carnival problem: Quality rating, best idea Facil. Env. 1.22 1.02 1.10 Nonfac. Env. 1.38 1.47 0.89 X-ray problem: Number of ideas Facil. Env, 3.03 2.82 2.18 Nonfac. Env. 3.79 3.44 1.81

30 IV DISCUSSION

EFFECTS OF TREATMENT Klausmeier, Johnson, Fredrick, and Ramsay, 1967) indicates that even a moderate degree of Mean performance of the group which re- teacher participation (e.g., writinginformative ceived Series One of The Productive Thinking comments on student papers, providing encour- Program surpas sed that of the Control group on agement, etc. ) can increase theeffectiveness 30 of the 40 internal and posttest measures. of programed material by as much as50%. This Of the 13 significant differences, 11 favored facilitative effect would probably beheightened the Treatment group.Clearly, then, use of even more by the greater extentof teacher par- the program led to an increase in the level of ticipation which occurred in the earlier experi- productive thinking in this large and representa- ments. tive group of fifth-grade students.Moreover, In contrast, the present study was de- this superiority was found at all three levels signed to probe the instructional limits of the of IQ.It was evident on a broad variety of materials by using them as an entirely self- the measures, including tasks which were quite contained program, with all forms of teacher different from the problems used in the lessons participation purposely held to a minimum. themselves.The practical and theoretical im- Moreover, the materials were administered at plications of these findings merit further dis- the rate of nearly one lesson per daythe fast- cussion here. est pace feasible for regular school use.This First, while the performance of the Treat- fast-paced teacherless condition represents a ment group generally exóeeded that of the very severe test of any instructional program; Controls, it did so only by a modest margin indeed, it is difficult to imagine a program rarely did mean scores for the Treatment group normally being used under such restrictive con- exceed those of the Control group by a factor ditions as these. By way of comparison, the of more than 20%. Differences of this magni- most recent experimental study with Series One tude, although they may be both statistically featured individualized supplementary activities and educationally significant, are considerably and distributed practice, in which the Program was smaller than the dramatic differences obtained used over a much longer period of time. Results of in two previous sty s (Covington and Crutch- this study are similar to those of the earlier studies, field, 1965) using earlier version of Series with performance of the Treatment group surpass- One.In those studies the Treatment group ing that of the Control group by a substantial sometimes scored more than twice as high as margin. the Control group on certain measures. The The results of the present.experiment, then, present study, however, differs from those in conjunction with those from the other studies previous ones in several important respects, just mentioned, indicate that some degree of and failure to find such large differenceshere teacher participation (even if only modest) re- is readily understandable.One important dif- sults in much greater educational benefit irom ference concerned the amount and kirrl the lessons, even though the materials alone teacher participation during training.In the do produce significant differences in thinking earlier studies, the classroom teachers were and problem-solving performance. deliberately instructed to supplement the les- A second noteworthy aspect of this treatment sons of Series One by featuring themin class effect is its generality.Superior performance discussions, by assisting and encouraging of the treatment group was evident on a broad their students to transfer the thinking princi- variety of measures and was not restricted to ples to other curriculum areas, and bymaking any one aspect of performance. A diverse set the lessons a regular component of theoverall of performance indicators, each reflecting a educational effort.Recent evidence (Blount, different aspect of the total problemrsolving process, showed consistent benefits as a

31 skills and result of training.These included achievement where the use of simple cognitive of solution to problems, number andquality of a positive set is notsufficient for successful ideas produced, intellectual persistence, sen- performan6e, but no obvious benefitfrom sitivity to discrepant or puzzling facts, useof training would be apparent onthe brief tests employed here. a Master ThinkingSkill, and an understanding research of the process of thinking itself. Clearly, there is a need for further reveal into (1) what cognitiveskills are already well However, the pattern of results did skills which some limits to thegenerality of training.These developed in children this age, kinds of would require practice ratherthan special train- limits appear to be a function of the designed to create problems, rather than the kinds ofperformance ing or instruction which was Specifically, there was and establish them in thechild's repertoire in indicators, employed. what skills do not yet a lack of treatmenteffects on nearly all per- the first place, and (2) problems, most exist in the child's repertoireand will require formance measures for the brief training before of which emphasized divergentthinking.These specific and perhaps extensive by Torrance, the student will be able tomake use of them. brief tests, such as those developed development of are structured so asto elicit a large numberof Further efforts devoted to the relatively criterion measures should incorporatesuch rather short, discrete responses on distinction, if it is uncomplicated tasks.Under these conditions, research and reflect this it seems likely that astudent's level of per- found to be valid. formance would be affectedconsiderably by Yet another important aspectof the treat- respond in a worth- ment differences is the occurrenceof a gradual his disposition or "set" to these differences while mariner.Presumably, a positive set was increase in the magnitude of students in this study as training proceeds.The means presented in present for the instructed difference in favor as a re.sult oftraining, but performance ofthe Table 15 show a very small Control group may have beenequally facilitated of the Treatment group onthe first internal test which re- (the Rare Coins problem),which was adminis- by the presence of a positive set point in the sulted from the novel andengaging quality of tered just before the half-way the tests and from the"escape" they provided training program. On thesecond internal test from normal classroom routine.Thus, the ef- (the Deep Sea Dive problem),administered fect of set per se might notlead to differential near the end oftraining, performance of the performance in the two groups. Treatment group exceededthat of the Controls Regardless of the effects 9f set,however, by a greater margin, eventhough it did not that the training of reach statistical significance).However, on one would still predict administered when all relevant cognitive skillswould result in supe- the posttest measures, One training had been completed,performance of rior performance by the Treatment group. that of the Con- possible explanation for thelack of such re- the Treatment group surpassed sults focuses on the questionof the relevance trol group by a statisticallysignificant margin If it is assumed that thebrief on two of thethree relevant measures(i.e., of such skills. to those used in the tests require less complexcognitive processes measures comparable than those processes internal tests).There was, then, a steady for adequate performance of training as taught for in the complexproblem settings of increase in the effectiveness learned in Series One the student proceededthrough the lessons. Series One, then what is findings is only marginally relevantto performance on This result differs from previous Thus the Treatment group (Covington and Crutchfield,1965), where the these brief problems. nearly as great on the might not show any markedsuperiority on such treatment effects were tasks. Another possibilitywhich must be con- first internal test asthey were on the posttest. sidered is that most childrenof this age, However, the conditions ofteacher participa- whether trained or not, candemonstrate these tion in that study(discussed above) may have skills on demand accelerated the effects oftraining.Thus, an relatively simple cognitive participation may (under proper conditions ofshort-term motiva- important effect of teacher rarely make use be to increase the rate aswell as the amount tion), even though they may developed when of them in their regularschool work.The more by which cognitive skills are complex tasks, on the otherhand, presumably materials such as these are used. require the use of advancedskills which are not yet well developed,but which are learned EFFECTS OF ENVIRONMENT and practiced in Series One.If this is true, would produce The five significant Environmenteffects on training provided by the program resulted from the fact superior performance oncomplex thinking tasks, the pretest measures all

32 ape 11004111-4

Ii ception of three measuresin the pretest Pit that classes withFacilitative environments effects resulted performed better thanNonfacilitative classes problem, all significant sex from the superiorperformance of females. on the twobrief divergent thinkingproblems occurred on all types developed by Torrance.The probable sensi- This female superiority of problems (bothverbal and nonverbal, con- tivity of these problems tothe effects of set and on all performance has already beenmentioned.If this interpre- vergent and divergent), present data, the indicators except measuresof intellectual per- tation is extended to the sistence and of theachievement of solution to significant Environmenteffects may possibly problems.This finding of generallysuperior have resulted from thepositive or negative set consistent with induced by the two typesof classroom environ- performance by females is Facilitative en- other findings (e.g.,Klausmeier and Wiersma, ment.Indeed, in this study a verbal performance of vironment was defined as onein which an 1964) concerning the students of this age. overall positive set seemedto be present while significant Treatment X the class was working on adiverg(nt thinking The absence of any support Sex interactions isparticularly noteworthy, task.Hence these pretest findings indicating as it does thatSeries One of The the validity of theEnvironment classifications. problem-solving tasks, in Productive Thinking Program wasequally ef- On the complex and girls (although the contrast, an increase inthe student's "readi- fective with both boys be sufficient to raise girls managed to maintaintheir initial advan- ness" to think should not training period). the level of performance,and in fact these tage throughout the problems did not show asignificant Environ- ment effect. EFFECTS OF IA On the posttest andinternal test measures, intellectual ability had significant Environment The general level of there was only one significant effect uponperformance comparisons.This virtual a clear and effect out of 40 for most of the measuresused in this study. In absence of posttrainingEnvironment effects relation large part because thetreat- most cases, thepredictable rank-order seems to occur in between IQ and performancewas found, with ment was particularlyeffective with students obtaining scores superiorto environments, differentially the High IQ group in Nonfacilitative those obtained by theMiddle IQ group, which raising the performanceof this group and thus by the differences favoring the were in turnsuperior to those obtained reducing any initial The effect of IQ wassignificant Facilitative environment. Low group. group having a on all types ofperformance measures,and on Nearly a dozen Treatment XEnvironment inter- except the specialtest actions (five of which aresignificant and are all types of problems of the MasterThinking Skill. presented in detail inTable 19) support this IQ effects were found It should be notedthat the spe- Large and significant conclusion. on all ofthe brief divergentthinking problems, cial benefits of trainingfor students in Non- especially marked on the to more than and this effect was facilitative classes amounted which is one ofTorrance's simply a change of"set, " since some of the Squares problem, nonverbal divergentthinking Significant significant Treatment XEnvironment interactions found on the other three complex problems inwhich IQ effects were also involved the kind of Torrance tests, andthese effects wereoften no main effect ofEnvironment was found.Thus (Tables 25 and 27) important effect of thetrain- large in an absolute sense it appears that an as well assignificant statistically.This set ing materials was toimprove the cognitive that early reports of a of students from of findings suggests skills and general attitudes "virtual lack of relationshipbetween measures the less favorableclassroom environments, interven- of creative thinkingand IQ" (Torrance,1962) even in theabsence of direct teacher something of an overstatementand Further research isneeded to determine are perhaps tion. that such findings mayhave re-iilted from precisely what teachersof varying capabilities Since all IQ analyses and attitudes can do,in conjunction withthe sampling restrictions. produce varying amounts in the present study arebased upon the care- training materials, to fully selected subsampleof 240 students of pupil gain or change. (described on page 3) whichshould be repre- sentative of the entiredistribution of ability EFFECTS OF SEX in a total of 44 classrooms,the highly sig- occurred on 21 nificant IQ effectsobtained cannot easily be Significant sex differences of the particular of the total of 61 pretest,posttest, and internal dismissed as an artifact test measures usedin this study. Withthe ex- sample used. 33 Indeed, it seems reasonable to assume that earlier Covington and Crutchfield study.It there is a moderate correlation between IQ and should be pointed out, however, that although creative thinking when the entire distribution the materials themselves may be used effectively of intellectual ability is considered. Most with students at many different levels of intel- studies which have reported the absence of lectual ability, the kind of teacher participation such a correlation (e.g. , Getzels and Jackson, required is very different for various levels of 1962; Wallach and Kogan, 1965) involved highly ability.Students who have serious reading dif- selective, nonrepresentative samples, while ficulties obviously require a great deal of as- studies which reported a positive relation be- sistance with the materials, while teachers tween IQ and creative thinking (e.g. , Ripple whose classes are above average in general and May, 1962; Guilford and Hoepfner, 1966) ability may devote most of their effort to show- have used far more representative samples. ing the students how to extend and transfer the In any case, IQ had highly significant effects thinking principles to other curriculum topics. upon performance for the vast majority of In an average class, the self-administering measures used in the present study. feature of the materials makes it possible for In contrast to the effect of IQ on test per- the teacher actively to assist a small group of formance, there was no consistent pattern of students while the rest of the students work interactions between IQ and Treatment; that is, individually. Generally, though, the lower the the increments in thinking and problem-solving level of student ability, the greater amount of performance produced by the training program teacher participation required for the effective were approximately equal for all three levels of use of these materials. IQ. This finding agrees with the results of the

34 APPENDIX

18

Uncle John notices Jim'ssilence:

...you see, youjumped to the Jim, the reason you can'tthink conclusion that someone stole You believed so of other ideas is that youfirst the water. it narrowed down to just one strongly in your idea that blinded you to other possible possibility... \eaas.

I guess I really stumbled when I jumped, this time. How can I get Gee, it's hard to learnnot started again? (IIto jump to conclusions! How can I think of other possi- bilifig, now?

B. Davies. Covington, RichardS.Crutchfield, and Lillian Copyright 1966 by MartinV. Reprinted with permission.

35 19

Here's another thinking guide that will give you a method for discovering many of the different ideas about this problem.

Pick out each of the importantthZ7s*N in the story--each object and person. Then take each of these things one at a time, and try to figure out how it might have had something to do with the disappearance of the water.

This method will make sure that you don't miss any important part of the problem that could give you ideas. ,1111111W

Now, what will happen as Jim and Lila take Uncle John's advice? Turn the page to find out.

3 6 20

Yes, themul;....'\ And there were I'm not sure I understand some otherthings, how this would work.... too. What were they? Well, first think of someof the main things in the (Istory.... r v,

Let's make a list ofthein.

37 You try making a list, too.Go back to pages 8 and 9 and read the story again. Then pick out each of the main things in the story and write it down: REFERENCES

Blount, N. S., Klausmeier, H. I., Johnson, Guilford, J.P., & Hoepfner, R.Creative S. L., Fredrick, W. C., & Ramsay, J. G. potential as related to measures of IQ and The effectiveness of programed materials in verbal comprehension.Indian Journal of English syntax and the relationship of se- Psychology, 1966, 41(1), 7-16. lected variables to the learning of concepts. Klausmeier, H. J.Effects of accelerating Technical Report from the Research and De- bright older elementary pupils: A follow up. velopment Center for Cognitive Learning, oumal of Ecl11.orialPs.y, 1963, University of Wisconsin, 1967, No. 17. 54, 165-171. Covington, M. V. New directions in the ap- Klausmeier, H. J., Harris, C. W., & praisal of creative thinking potential. Ethnathios, Z.Relationships between di- journal of Educational Measurement, 1968, vergent thinking abilities and teacher ratings in press. of high school students.Journal of Educa- Covington, M. V., & Crutchfield, R. S.Ex- tional Psychology, 1962, 53, 72-75. periments in the use of programed instruc- Klausmeier, H. J., & Teel, D. A research- tion for the facilitation of creative problem based program for gifted children.Educa- solving.Programed Instruction, 1965,_4, tion, 1964, 85131-136. 3-10. Klausmeier, H. J., & Wiersma, W. Relation- Covington, M. V., Crutchfield, R. S., & ship of sex, 'grade level, and locale to per- Davies, L. B.The Productive Thinking formance of high IQ students on divergent Program, Series One: General Problem Solv- thinking tests.Journal of Educational Psy- ing.Berkeley, California: Brazelton Print- chology, 1964, 55114-119. ing Company, 1966.Distributed by Educa- Ripple, R. E.,Sz May, F. W. Caution in com tional Innovation, Box 9248, Berkeley, paring creativity and IQ.Psychological California 94719. Reports, 1962, 10, 229-230. Crutchfield, R. S.Creative thinking in chil- Torrance, E. P.Guiding creative talent. dren:Its teaching and testing.In 0. Brim, Englewood Cliffs, N. J.:Prentice-Hall, R. S. Crutchfield, and W. Holtzman, 1962. Intelligence:Perspectives 1965. New York: Wallach, M. A., & Kogan, N. Modes of think- Harcourt, Brace, 1966.Pp. 33-64. ing in young children: A study of the creativ- Getzels, J. W., & Jackson, P. W. Creativity ity-intelligence distinction. New York: and intelligence: Explorations with gifted Holt, Rinehart & Winston, 1965. students. New York: Wiley, 1962.

GPO 806,-459-3 39