Cognitive Skill Acquisition

Kurt VanLehn

Learning Research and DevelopmentCenter

University of Pittsburgh

Pittsburgh PA

VanLehncspittedu

fax

May

Abstract

Cognitive skills acquisition is acquiring the abilitytosolve problems in intellectual

tasks where success is determined more by the sub jects knowledge than their physical

prowess This chapter reviews reseach conducted in the last ten years on cognitive skill

acquisition It covers the initial stages of acquiring a single principle or rule the initial

stages of acquiring a collection of interacting pieces of knowledge and the nal stages

of acquiring a skill wherein practice causes increases sp eed and accuracy

To app ear in Annual Review of Psychology Vol J Sp ence J Darly D J Foss

Eds Palo Alto CA Annual Reviews

Keywords Cognitive skill acquisition problem solving schema acquisition practice

eects transfer selfexplanation

Short title Cognitive skill acquisition

Contents

Intro duction

A brief history

A framework for reviewing cognitive skill acquisition

The intermediate phase Learning a single principle

Learning a single principle Intro duction

Retrieval

Mapping

Application

Generalization

Summary

The intermediate phase Learning multiple principles

Transfer

Strategy dierences in learning from examples

Learning events

Learning from a computer tutor

The nal phase practice eects

The p ower law of practice and other general eects

Replacing mental calculations by memory retrieval

Transfer of the b enets of practice

Negative transfer

e skill acquisition research The frontiers of cognitiv

Acknowledgements

Intro duction

Cognitive skills acquisition is acquiring the abilitytosolve problems in intellectual tasks

where success is determined more by the sub jects knowledge than their physical prowess

Frequently studied tasks include solving algebraic equations and word problems college

physics problem solving computer programming medical diagnosis and electronic trou

blesho oting Researchers in cognitive skill acquisition study how p eople learn to accomplish

such complex knowledgeintensive tasks and how they b ecome exp erts in their elds

A brief history

Cognitive skill acquisition has its historical ro ots in the study of problem solving Research

on problem solving b egan early in this century by studying what makes problems dicult

to solve Duncan In the s researchers turned to studying the pro cess of solving

a problem Sub jects solved multistep puzzles while explaining their reasoning aloud Tran

scriptions of their commentaries called verbal protocols provided the empirical foundations

for developing computational mo dels of problem solving Newell and Simon intro

duced most of the imp ortant theoretical concepts including problem spaces search trees

and pro duction systems

Because problem solving research emphasized the pro cess of moving from one intermedi

ate state to another until one nally arrived at a solution researchers preferred to use tasks

where most intermediate states were physical states In the Tower of Hanoi for instance

the sub jects try to moveapyramidal stack of disks from one p eg to another bymoving one

disk at a time sub ject to certain restrictions Solving the puzzle requires manyphysical

movements of disks and thus exp oses the sub jects intermediate states Problems that are

solved with a single physical action were seldom studied by problem solving researchers

During the s two related elds develop ed Decision making studied p eople mak

ingachoice under uncertainty and reasoning studied p eople drawing a conclusion from

a combination of mental inferences In a sense these are also forms of problem solving

However p erhaps b ecause most of their intermediate states are mental and not physical

their metho dological and theoretical concerns have remained distinct from those of problem

solving

In the s problem solving researchers b ecame interested in how sub jects solved prob

lems that required muchmoreknowledge than the simple puzzle problems that were used in

the s They studied problems in chess physics mathematics computer programming

medical diagnosis and many other elds Whereas one could tell a sub ject everything they

needed to know to solve a puzzle in a few minutes solution of even an easy problem in a

knowledgerich task domain required many of hours of preparatory training

The exploration of knowledgerich problem solving b egan bycontrasting the p erformance

of exp erts and novices For instance one robust nding is that exp erts can sort problems

into categories according to features of their solutions while novices can only sort problems

using features of the problem statement itself As discussed by Ericsson Lehmann this

volume this and many other the ndings can b e explained by assuming that whenever

mental planning of solutions is p ossible eg b ecause all the information required to solve

problems is present in the initial state then exp erts typically develop the ability to plan

solutions in memory This often requires the abilitytoenvision sequences of intermediate

states so exp erts develop impressive mneumonic p owers but only for intermediate states

that they typically encounter

In the s many researchers turned to studying how p eople acquired exp ertise At

tention initially fo cused on the role of practice in the development of exp ertise Phenomena

that were often found with motor skills such as the p owerlaw of practice and the identical

elements mo del of transfer were found with cognitive skills as well Most of the recent

work has fo cussed on the role of instruction during the early stages of skill acquisition and

in particular on the role of examples In this literature an example is a problem whose

solution is given to the student along with the solutions derivation Examples app ear to

e skill acquisition play a central role in the early phases of cognitiv

Because reviews of the early days of problem solving and cognitive skill acquisition

are available VanLehn Kahney aswell as reviews of exp ertise Ericsson

Lehmann this volume and instructional considerations Glaser Basso ck Voss

Wiley Carretero this review will fo cus exclusively on recentwork in cognitive skill

acquisition Because there is much material to cover and simply listing the ma jor ndings

would make it imp ossible to assimilate them all a lo ose framework has b een provided

A framework for reviewing cognitive skill acquisition

Fitts distinguished three phases of motor skill acquisition His early intermediate

and late phases also aptly describ e the course of cognitive skill acquisition

During the early phase the sub ject is trying to understand the domain knowledge

without yet trying to apply it This phase is dominated by reading discussion and other

generalpurp ose information acquisition activities that lie outside the scop e of this review

Most investigations of cognitive skill acquisition do not collect observations during the early

phase

The intermediate phase b egins when students turn their attention to solving problems

Before they b egin solving problems themselves they often study a few problems that have

already b een solved called examples henceforth Examples maybeprinted in a textb o ok

or b e presented live byateacher As they solve problems students may refer back to the

textb o ok or ask a teacher for help but their primary fo cus is on solving problems This

dierentiates the intermediate phase from the early phase where the primary fo cus is on

studying exp ository instructional material

When sub jects start the intermediate phase they have some relevant knowledge for

solving problems but certainly not all of it They also mayhave acquired some misun

derstandings Thus the rst order of business is to correct these aws in the domain

knowledge For lackofabetterword aw will b e used subsequently to stand b oth for

missing knowledge and incorrect knowledge The second order of business is to acquire

heuristic exp eriential knowledge that exp edites problem solving

Eventually students remove all the aws in their knowledge and can solve problems

without conceptual errors although they may still make unintended errors or slips Nor

termediate phase and the b eginning of man This capability signals the end of the in

the late phase During the late phase students continue to improve in sp eed and accuracy

as they practice even though their understanding of the domain and their basic approach

to solving problems do es not change Practice eects and transfer are the main research

issues here

This phase chronology is an idealization The b oundaries b etween phases are not as

sharp as the description ab ovewould lead one to b elieve Moreover instruction on a cog

nitive skill is divided into courses topics chapters and sections Students are intro duced

to a comp onent of the skill given substantial practice with it then moved on to the next

comp onent Thus at anygiven time students may b e in the late phase with resp ect to some

comp onents of their skill but in other phases with resp ect to other comp onents Nonethe

less it is useful to make the phase distinction b ecause dierent empirical phenomena

characterize each phase

This review covers the intermediate and nal phases However muchmorework has

b een done on the intermediate phase than the nal phase so the review of intermediate

phase research is split into two parts The rst part covers studies where students learned

only a single principle These studies fo cussed on the basic pro cesses of assimilating a new

principle retrieving it from memory during problem solving and applying it The second

part covers studies where students learn many principles Following this twopart discussion

of the intermediate phase is a section on the late phase

The intermediate phase Learning a single principle

Before b eginning the discussion of learning a single principle some general remarks ab out

the intermediate phase are necessary

Perhaps the most ubiquitous nding of the intermediate phase is the imp ortance of ex

amples Studies of students in a variety of instructional situations have found that students

prefer learning from examples than learning from other forms of instruction eg Lefevre

Dixon Pirolli Anderson Chi et al Students learn more from studying

examples than from solving the same problems themselves Co op er Sweller Carroll

A year program in algebra was completed in years by students who only studied

examples and solved problems without lectures or other direct instruction Zhu Simon

termediate phase Because of the imp ortance of examples most research on the in

has used instructional material where examples are prominent Sometimes the instruction

consists only of examples and students must infer the general principles themselves

Most research has studied students solving problems alone There is some research

on learning from a tutor whichiscovered at the end of the intermediate phase section

Learning by solving problems in small groups has not yet b een studied muchby cognitive

skill acquisition researchers

Learning a single principle Intro duction

As mentioned earlier muchwork on the intermediate phase has concentrated on learning

material that is ab out the size of a single principle where a principle is the sort of thing that

a textb o ok states in a colored b ox and discusses for several pages Often the instruction

includes an example illustrating the application of the principle Table shows an example

taken from Catramb one illustrating an elementary probability principle that is often

used in cognitive skill acquisition research Knowledge of the principle includes not only

the p ermutation formula but also the meaning of the variables n and r and the kinds of

problems this formula applies to The convergenceofforces idea used to solve Dunckers

famous Xray problem is another commonly used principle One can generally teacha

sub ject a rudimentary version of a principle in an hour or less in contrast to teaching a

rudimentary version of a whole cognitive skill which might takedays or months

All the exp eriments to b e discussed have a similar format Sub jects are trained typically

by studying a b o oklet The training material almost always includes examples and may

consist only of examples After the training and sometimes after a distractor task the

sub jects are given problems to solve The problems can b e easily solved using the principle

but are dicult to solve without it

When the training consists only of examples then solving the test problems is sometimes

called analogical problem solving b ecause it involves nding an analogy corresp ondence

between the example and the problem

Although learning a principle consists of earlyintermediate and late phases these ex

p eriments fo cus exclusively on the intermediate phase Students are not given enough

practice to enter the late phase The early phase in these exp eriments consists of studying

the b o oklet or other instructional material and observations are rarely collected during this

phase

The intermediate phase consists of trying to solve problems by applying the new prin

ciple example or b oth Applying a principle or example consists of retrieving it placing

Table An example solved with the p ermutation principle

Problem The supply departmentatIBMhastomake sure that scientists get computers

Today they have IBM computers and IBM scientists requesting computers The

scientists randomly cho ose their computer but do so in alphab etical order What is the

probability that the rst scientists alphab etically will get the lowest second lowest and

third lowest serial numb ers resp ectively on their computers

Solution The equation needed for this problem is

n n  n  r

This equation allows one to determine the probability of the ab ove outcome o ccurring In

this problem n andr The represents the numb er of computers that are

available to b e chosen while the represents the number of choices that are b eing fo cused

on in this problem The equation divides the number of ways the desired outcome could

o ccur by the numb er of p ossible outcomes So inserting and into the equation we nd

that the overall probabilityis

its parts into corresp ondence with parts of the problem eg in the case of the p ermu

tation principle deciding what n and r are and drawing inferences ab out the problem

and its solution on the basis of the problems corresp ondence with the principle or exam

ple After applying the principle or example sub jects may generalize it Each of these

pro cessesretrieval mapping application and generalizationwill b e discussed in turn

Retrieval

There app ear to b e two kinds of retrieval sp ontaneous and delib erate Delib erate retrieval

often o ccurs after sub jects are given a hint eg The examples you studied earlier will

help you solve this problem or when the exp eriment simulates an instructional situation

where students exp ect earlier material to b e relevant to solving problems Brown Kane

Sp ontaneous retrieval or reminding as it is more commonly known o ccurs when the

exp erimenter hides the relationship b etween the training and testing phases of the exp eri

menttypically by telling students that they are two dierent exp eriments and yet sub jects

nonetheless notice that the training is relevant to solving the problem This exp erimental

paradigm explores why education so often creates inert knowledge which students can

recall when given explicit cues but fail to apply outside the classro om Bransford et al

A strong but obvious eect is that delib erate retrieval is vastly more successful than

reminding Many more sub jects can retrieve a principle or example after a hint than b efore

ahint eg Gick Holyoak

When reminding do es o ccur it often is triggered by surface similarities b etween the

problem and a training example eg Catramb one Holyoak Holyoak Koh

Ross For instance sub jects are likely to b e reminded of the

IBM example Table more by a problem that mentions Microsoft programmers designing

computer software than by the car mechanics problem of Table even though the car

mechanics problem has same mathematical structure as the IBM example Sub jects can b e

reminded by structural similarities but only when the training emphasizes the underlying

structure of the examples and the test problems are reworded to emphasize their deep

structure Catramb one Holyoak

Table A problem with the same deep structure as the IBM problem

Southside High Scho ol has a vo cation car mechanics class in which students repair cars

One day there are students and cars requiring repairs The cars are assigned to

students in order of the severity of their damages the car in the worst shap e go es rst

but the studenttowork on the car is randomly chosen What is the probability that the

cars in the worst shap e are worked on by the students with the highest grades in order

of their grades ie the student with the highest grade working on the worst car etc

Although sup ercial reminding seems to b e the p opulation norm it is less common

among students with high mathematics SAT scores NovickHolyoak Novick

Moreover delib erate retrieval particularly in instructional situations is often guided by

structural similarityFaries Reiser

Mapping

The mapping pro cess puts parts of the principle or example into corresp ondence with parts

of the problem For instance in order to use the example of Table as a mo del for solving

a new problem sub jects must nd values for n and r which in turn requires nding ob jects

corresp onding to the IBM scientists and computers

Sub jects are easily misled by surface similarities into using the wrong corresp ondence

mapping For instance Ross found that most sub jects solved the problem of

Table by pairing mechanics with IBM scientists and cars with computers This matches

the sup ercial characteristics of the ob jects but it pro duces an incorrect solution to the

problem

Application

In some cases sub jects have almost nished solving the problem after they have put the

principle or example into corresp ondence with the problem For instance once sub jects

have mapp ed the car mechanics problem to the IBM example the only remaining tasks

are substituting for n and for r in the p ermutation formula and doing the arithmetic

However applying the principle or example can b e muchmoreinvolved NovickHolyoak

For instance when an example is complex sub jects usually refer back to it many

times when solving a problem eg Pirolli Anderson Reed Willis Guarino

VanLehn a They app ear to use a variety of strategies for deciding whether to

refer to the example and to attempt the next step without help VanLehn a

Generalization

Problems and examples contain information that is not causally or logically related to

their solution In the IBM example the nature of the particular ob jects b eing chosen IBM

computers is irrelevant Such information comprises the surface features mentioned earlier

As discussed ab ove surface features can play a strong role in retrieval and mapping Their

role in application has not yet b een established but is likely to b e strong as well

Generalization is the pro cess of mo difying ones understanding of an example or principle

in suchaway that surface information do es not play a role in retrieval mapping and

application Generalization allows one to apply the principle or example to more problems

Avariety of instructional metho ds for encouraging generalization have b een tried Gick

Holyoak and others have found that simply augmenting the example with an explanation

of the principle b ehind it provided little generalization Using two examples was also in

eective What do es work is using two examples and some sort of highlighting device

that encourages sub jects to compare the examples and nd their common structure Gick

Holyoak Catramb one Holyoak Ross Kennedy Cummins

Generalization causes sub jects to b e reminded of examples by structural features instead of

just surface features and to b e fo oled less during mapping by surface features

These results strongly suggest that generalization is not an automatic pro cess as as

sumed by earlier theories of cognitive skill acquisition eg Anderson Although

proto coltaking exp eriments would b e necessary to conrm this suggestion it is likely that

sub jects must actively decide which prop ositions in an example are structural and which

h asp ects of an example are sup ercial Alternatively sub jects can probably b e told whic

are general cf Ahn Brewer Mo oney

As one would predict from basic prop erties of memory deciding that some prop ositions

in an example are sup ercial do es not erase them from memory nor do es building a gen

eralization from comparison of two examples erase the examples from memory Bernardo

NovickHolyoak

Summary

This concludes the discussion of the learning of individual principles Overall sup ercial

reasoning is the norm but sub jects can b e induced to use deep er reasoning under certain

circumstances Left to their own devices sub jects typically just enco de examples and

principles in memory where they languish until retrieved delib erately or via sp ontaneous

sup ercial asso ciation Once retrieved an example or principle is applied by just plugging

in sup ercially similar ob jects and copying the resulting solutions Sub jects can use more

structural features in sp ontaneous retrieval and mapping but they must rst b e induced to

generalize the examples which seems to require using multiple examples and directing the

sub jects to nd their commonalities

On the other hand when sub jects susp ect or are told that examples they have seen

might help them solve their problem their retrieval is often based on structural features

That is they search memory or the textb o ok for examples whose solution might help them

The intermediate phase Learning multiple principles

Mastering a cognitive skill often requires learning more than one principle as well as many

other pieces of knowledge that one would hesitate to call principles For instance a physics

studentmust learn facts such as the units for measuring force Newtons and mass grams

There are even b orderline cases Is the knowledge that mass is not the same thing as weight

a principle Learning a cognitive skill also requires learning heuristics or exp eriences that

will help one select the right combination of principles for solving a problem

Much of what has b een observed in the study of singleprinci pl e learning probably

applies to the learning of all these typ es of knowledge The pro cesses of retrieval b oth

sp ontaneous and delib erate mapping application and generalization probably characterize

the acquisition of minor principles eg mass and weight are dierent heuristics and

other generalizations Factual knowledge may b e to o simple for mapping and generalization

Table Examples and problems of solving equations for a

ab f g  ab g  f  a g  f b

ba f g  a f gb  a gb  f

mac b k

f a bw g

to apply

However as the quantity and complexity of the material to b e learned increases one

can exp ect to see eects that cannot b e observed when studying the acquisition of a single

principles This section fo cuses on reviewing such phenomena

Transfer

When an example uses multiple principles in its solution then it is p ossible to study an

interesting typ e of transfer First students are trained with examples and problems that

use two or more principles in a certain combination then they are tested on problems

that require using the principles in a dierentcombination For instance consider the two

algebra examples shown on the top two line of Table Their solutions require using two

principles removing a term by subtracting it from b oth sides of an equation and removing

a co ecientby dividing b oth sides of the equation by it These same principles can b e used

to solve the problems shown on the last two lines of Table but they must b e used in a

dierent combination than they were used in the examples If a student trained on the two

examples can solve the two problems then a certain rather constrained typ e of transfer has

b een obtained

However such transfer is rarely obtained Co op er and Sweller found that many

eightgrade students trained with multiple versions of the two examples in Table could not

solve the problems shown on the last two lines of the table On the other hand the students

had no diculty solving problems suchas ac g h that could b e solved bycopying

k of transfer has b een found many times the solution of the training examples Similar lac

Reed Dempster Ettinger Sweller Co op er Catramb one

Note that this kind of transfer is dierent from the generalization discussed in the

preceding section where copying the examples solution was all that was exp ected of the

sub jects Generalizing the examples shown in the top two lines of Table will solve problems

suchasac g h but not the ones shown in the b ottom of Table

Catramb one a b showed that mo difying the examples solutions in order to

highlight the application of each principle signicantly increased transfer Training that

required students to drawcontrasts among pairs of examples in order to see individual

principle applications was less successful Catramb one Holyoak

Even when principle applications are not highlighted principlebased transfer can o ccur

but it requires a large numb er of examples and ab oveaverage students Cooper Sweller

This suggests that some students study examples dierently than others and that

is the topic of the next section

Strategy dierences in learning from examples

Examples are commonly used in twoways Students study examples b efore solving prob

lems or they refer back to examples when they are in the midst of solving a problem For

each of these two activities it app ears that learning is strongly aected by the students

strategies for studying examples and for referring back to examples Let us rst consider

the activity of studying examples b efore solving problems

Chi et al found that students who explained examples thoroughly to themselves

learned much more than students who merely read the example through Chi et al had

college students rst study four intro ductory chapters from a college physics textb o ok until

they could pass exams on eachchapter This insured that all students had the necessary

prerequisite knowledge for understanding the target chapter The target chapter taught

ts read the target chapter studied students ab out forces and Newtons laws The studen

examples and solved problems Proto cols were taken during the example studying and

problem solving phase Students were classied via a median split on their problem solving

scores into Go o d learners and Po or learners When proto cols of the example studying phase

were analyzed it was found that the Go o d learners uttered more selfexplanations than Poor

learners where a selfexplanation is any inference ab out the example that go es b eyond the

information presented in the example For instance given the line F F cos

ax a

a Go o d student mightsaySoF must b e the leg of the right triangle and F is the

ax a

hyp otenuse yep Whats that minus sign doing there Po or students often just read

the line or paraphrased it then went on to the next line

Chi and Vanlehn analyzed the content of the students selfexplanations The

results suggested that there were two general sources for selfexplanations One was deduc

tion from knowledge acquired earlier while reading the text part of the chapter usually by

simply applying a general principle to information in the current example statement The

second source was generalization and extension of the example statement These inferences

help ed to ll gaps in the students knowledge most often by providing necessary technical

details that were not discussed in the text

VanLehn Jones and Chi constructed a computer mo del of the example studying

and problem solving pro cess In order to solve the problems correctly the mo del required

approximately rules The rules represented ma jor principles minor principles facts

and technical details eg how to determine whether the sign of a vectors comp onentis

p ositive or negative Less than half of the rules were mentioned in the textb o ok The

other rules were learned by the mo del as it studied the examples but only if it selfexplained

them The mo del accounted for the aggregate results from the Chi et al study as

well as the p erformance of individual sub jects Vanlehn Jones Error patterns

and other analyses are also consistent with the hyp othesis that most of the b enet from

selfexplanation comes from lling in gaps in the sub jects knowledge V anLehn Jones

The selfexplanation eect is quite general Selfexplaining examples in a variety of task

domains improves learning Pirolli Bielaczyc Pirolli Recker Fergusson

Hessler de Jong Lovett Brown Kane Pressley et al Self

explaining exp ository text Chi et al and hyp ertext Recker Pirolli instead

of examples also enhances learning Most imp ortantly students can b e trained to self

explain and this improves their learning dramatically Chi et al Bielaczyc Pirolli

Other investigators who have formalized textb o ok knowledge have also found that the textb o ok leaves

out many crucial details eg Psotka Massey Mutter section

Brown in press Bielaczyc Recker However selfexplanation only aects the

initial acquisition of knowledge and not subsequent improvement with practice Pirolli

Recker

Several of these studies also found a selfmonitoring eect Chi et al Pirolli

Recker Both Go o d and Po or students tended to sp ontaneously utter assessments

of their understanding However Po or students tended to utter uniformly p ositive self

assessments eg Yep That makes sense even though their subsequent p erformance

indicates that they really did not understand the material On the other hand Go o d stu

dents tended to monitor their understanding more accurately and frequently noted failures

to understand eg Wait I dont see how they got that Accuracy in selfmonitoring

app ears to b e correlated with learning

So far wehave b een discussing dierent strategies for studying examples b efore solving

problems However students also refer to examples as they solve problems and here we

also nd correlations b etween learning and strategies Both Go o d and Po or learners refer

to examples during problem solving Chi et al although more so with the rst few

problems solved than with later problems Pirolli Reed Willis Guarino

What matters is the way students refer to examples Proto col analyses Chi et al

VanLehn ab and latency analyses Pirolli Recker suggest that Poor

learners maximize their use of problemsolving by analogy they refer to an example as

so on as they notice that it is relevant and copyasmuch of its solution as p ossible On the

other hand Go o d learners minimize their use of analogies to examples they refer to the

example only when they get stuck solving a problem and need some help That is Go o d

solvers prefer to solve problems by themselves whereas Poor solvers prefer to adapt an

examples solution

After solving a problem sub jects often reect on the solution Pirolli Recker

found that go o d and p o or learners b oth reect on half the problems they solve but what

t Po or learners usually just paraphrased the solution Go o d learners they say is dieren

tried to abstract general solution metho ds often by comparing this problems solution to

the solutions of earlier problems

In all these studies the criteria for learning included p erformance on problems that

could not b e solved by merely copying the example solutions Thus learning requires the

kind of transfer discussed earlier section Strategy eects may not show up on single

principle exp eriments covered in section b ecause learning generalization is assessed on

problems that can b e solved bycopying solutions

In short learners seem to use dierent studying strategies that trade o eort and

likliho o d of learning Some students exert considerable time and energy in selfexplaining

examples andor by trying to solve problems without copying the examples these students

often learn more Other students exert less eort by merely reading the examples through

andor by copying example solutions they learn less As the saying go es no pain no gain

Learning events

Knowledge of a complex skill is comp osed of many pieces of knowledge Some represent

principles some represent examples or generalizations thereof and others representtech

nical details heuristics and other information that could b e relevant in solving a problem

The studies reviewed earlier suggest that learning a single principle requires attention b oth

during the early phase studying a b o oklet typically and during the intermediate phase

as the principle is retrieved mapp ed applied and generalized Moreover it app ears that

generalization of an example requires attentionit is not an automatic pro cess nor a mere

bypro duct of applying the example to solve a problem This in turn suggests that during

the intermediate phase of complex skill acquisition new pieces of domain knowledge are

acquired one at a time by taking time o from problem solving or example studying to

attend to the discovery of new pieces of knowledge and p erhaps to generalization as well

This hyp othesis is consistent with several studies of transfer For instance Bovair

Kieras and Polson taught sub jects sequences of simple textediting commands Each

command had several steps such as pressing a function key copy delete move etc

checking a prompt selecting some text and pressing the enter key They represented each

commands pro cedure with a set of rules Across commands some rules were identical

some were analogous and some app eared in only one command They trained sub jects

on all commands varying the order in which the commands were taught For each order

they calculated the numb er of new rules to b e learned for each command the number of

rules that were identical to rules learned earlier and the numb er of rules that were only

analogous to rules learned earlier They found that the training time was a linear function

of the numb er of new rules and the total numb er rules There was a second cost for each

rule b ecause the sub ject had to execute every rule of the command regardless of whether

that rule had already b een learned or not However each new rule to b e learned required

a substantial additional amount of time ab out seconds This suggests that learning a

new rule requires taking time o from executing the familiar parts of a pro cedure There

was no cost in training time for learning rules that were analogous to rules learned earlier

which suggests generalization of these rules was quite easy Singley and Anderson

also found using text editing programming and mathematical task domains that training

time was a function of the numb er of new rules to b e learned

If learning a new principle or other piece of knowledge do es taketimeaway from execut

ing familiar knowledge then p erhaps one will see learning events during problem solving

wherein a sub jects briey switches attention from solving the problem to reasoning ab out

the domain knowledge itself Learning events should also b e found during other instruc

tional activities such as selfexplanation As it turns out learning events can b e observed

via detailed proto col analysis

Learning events were rst observed in discovery learning situations wherein students

were given very little instruction and had to discover the principles of a domain during the

course of solving problems in it KarmiloSmith and Inhelder observed discrete

changes in childrens pro cedures for balancing blo cks on a b eam and explained them as

miniature discoverieslearning events Kuhn and Phelps also noted discrete changes

in college students exp eriment design strategies and assumed that they were due to learning

events Siegler and Jenkins observed children shift their strategies for adding by

ting on their ngers and coined the term learning event for the brief episo des where coun

the changes o ccurred VanLehn showed that all the changes in strategy of a sub ject

solving the Tower of Hanoi puzzle seemed to app ear during learning events

Discovery learning is rarely used to teach a cognitive skill Instead students typically

access examples textb o oks and sometimes teachers or p eers However the instructional

material is almost always incomplete As mentioned earlier roughly half the pieces of

knowledge required to solve problems were not mentioned in the college textb o ok used in

the Chi et al study When faced with incomplete instructional material students

need to discover the missing information for themselves VanLehn c analyzed the Chi

et al proto cols and found learning events in most of the places where information

that was missing from the instruction was required for solving problems and explaining

examples

In all these studies learning events were characterized by long pauses or verbal signs of

confusion All the studies were based on verbal proto cols Sub jects rarely announced their

discoveries in a clear fashion Even when prob ed their explanations were seldom coherent

Siegler Jenkins Nonetheless their problem solving b ehaviors changed

As an illustration of a learning event consider a physics studentquotedinVanLehn

c The student did not knowthatweight is a kind of force so he could not gure

out how to nd the weightofanobjecteven though he had calculated that the force of

gravity on the ob ject was p ounds After pausing and complaining a bit he recalled the

textb o ok equation F Wg a and p erformed the totally unjustied substitution of g for

athus deriving the equation F W whichheinterpreted as meaning that the ob jects

weight is equal to the force of gravityonit Amazingly he had discovered the missing

correct principle via a sp ecious derivation He said Right Is that right Okay Um

It kind of makes sense cause so Im going to get p ounds Thats the force Yeah

they they weigh you in p ounds dont they Thats force The learning event consisted of

reaching an impasse using purely syntactic algebraic symb ol manipulation to hyp othesize

a new principle that weight is the force of gravity on an ob ject then supp orting the

principle with the fact that p ounds is a unit of force and that Americans express weightin

p ounds However this is not the end of the story On the next o ccasion where the sub ject

could use the principle he failed to do so This resulted in a units error which he detected

while checking his answer That reminded him of his new principle he said Oh wait a

minuteOh hold the bus Hold the busbut after a brief pause he decided the principle

didnt applyHowever after he had failed to nd another explanation for the units error he

changed his mind and decided the principle did apply In short learning a principle in the

context of problem solving is much like learning in one of the singleprinci pl e exp eriments

reviewed earlier One may not b e reminded of it immediately but when one delib erately

searches memory the new principle can b e retrieved Yet retrieval of a new principle is not

enough One often has to delib erate ab out its generality b efore applying it Presumably

this delib eration causes generalization

Learning events can b e analyzed in terms of three characteristics

 What provoked the studenttoswitchattention from the main task to learning In

the illustration reaching an impasse provoked the physics student to try to derivea

new principle In some situations most learning events were triggered by impasses

VanLehn VanLehn c However when the student already has a correct

op erational solution pro cedure improvements to the pro cedure are triggered bysome

kind of noticing pro cess Siegler Jenkins VanLehn which can b e

mo deled Jones VanLehn but is still not well understo o d After solving

a problem some students delib erately reect on its solution in order to uncover its

overall plan or basic idea VanLehn Recker Pirolli

 What kind of reasoning went on during the learning event In the illustration the stu

dent used b oth algebraic symb ol manipulation and common sense reasoning to derive

his new principle Such educated guesses seem often to b e based on causal attri

bution heuristics Lewis or overgeneralizations VanLehn c However

students often use less constrained forms of induction VanLehn c in which

case one can observe the prototypicality eects usually found in concept formation

exp eriments Lewis Anderson

w general is it Advo cates of discov  How easily retrieved is the new principle and ho

ery learning often hop e that b ecause students discover principles by themselves the

principles will b e easily retrieved and adequately general As illustrated ab ove this

do es not seem to b e the case Principles acquired during learning events are often

dicult to retrieve and seem to require delib erate attention b efore they are general

enough Siegler Jenkins Kuhn Phelps VanLehn c

Progress on understanding learning events has b een slow b ecause it is dicult to dis

tinguish them from ordinary problem solving even in the proto cols of the most talkative

sub jects It has b een necessary to have a strong theory of the kind of knowledge that can

b e learned and a long enough p erio d of observation that one can see slowchanges in the

usage of individual pieces of knowledge Nonetheless the observations to date supp ort the

general hyp othesis that learning a cognitive skill consists of learning a large numb er of small

pieces of knowledge including principles technical details heuristics etc and that for

each one learning consists of a series of learning events that construct and generalize it

Learning from a computer tutor

The discussion of the intermediate stage up to this p oint has assumed that the learner has

no one to talk to and only written materials to refer to While this constraint simplies

the observation of learning it makes the instructional situation somewhat atypical of real

world situations where learners can obtain help by raising their hands picking up the

phone or walking down the hall to a colleagues oce As a rst step toward understanding

cognitive skill acquisition in the wild researchers have studied students learning from a

very taciturn tutor namely a computer

Although currenttechnology do es not allow a computer to converse with students the

wayhuman tutors do computer tutors do have three advantages over instruction based on

written materials

 When working with written materials students are usually assigned to solve a x set

of problems for eachchapter A computer tutor can select a problem for the student

to solve based on the students p erformance on preceding problems Thus dierent

students will get dierent problems to solve

 The computer tutor can p oint out errors to the student so on after they are committed

Errors are overt incorrect actions taken by the student Although tutors can some

wledge that caused the error many tutors times infer the aw in the students kno

leave it to the student to nd and remedy the sources of their errors

 The computer tutor can answer certain questions and requests for help although the

dialog is highly constrained

The impact of each of these three features on learning is discussed in turn

When the tutor selects problems for the student to solve a simple p olicy is to keep

giving the student problems until the student has reached mastery Mastery can b e dened

as answering the most recent N problems with a score higher than M Some computer tutors

eg Anderson et al can monitor the usage of individual principles or steps so they

implement a p olicy that keeps assigning problems until each has b een mastered Varying

the thresholds of mastery for individual pieces of knowledge can accurately predict errors

Corb ett Anderson OBrien Masterybased tutoring generally causes higher

scores on p osttests than tutoring based on xed problem sets eg Anderson Conrad

Corb ett which suggests that dierent students learn at dierent rates which in turn

is consistent with their using dierent studying strategies section dierent kinds of

reasoning during a learning events section and p erhaps other factors as well

Perhaps the most hotly debated issue concerns the control of feedback on errors The

current technology enables students to use the computer as scratch pap er Instead of simply

entering their answers to problems as early computer tutors required students to day can

show all their work to the computer This allows the computer to detect errors so on after

they are committed Many tutors inform the student as so on as an error is detected and

prevent them from going on until the error is corrected Although such feedbackmakes the

tutor easier to build one wonders what eect it has on learning Lewis Anderson

found that immediate feedback that forced students to correct the error b efore moving on

was sup erior to delayed feedback but the eect was quite small Anderson et al

contrasted four feedback p olicies no feedback feedback when the studentasked for

it immediate feedback that notied the student assoonasanerrorwas made but did

not force the student to correct the error and immediate feedback that forced immediate

ere forced to answer the problem correction of the error In conditions and students w

correctly b efore b eing allowed to continue to the next problem In condition the feedback

also insured that they answered the problem correctly Only the nofeedback condition

did not force students to correct their mistakes On p osttest measures of learning students

in the nofeedback condition did worse than those in the feedback conditions and there

were no signicant dierences among the feedback conditions Thus it app ears that

neither the timing of feedback nor its control student vs machine makemuch dierence

on whether a studenteventually acquires the target knowledge However if feedbackis

completely removed then students are often unable to correct aws in their knowledge on

their own

This result is consistent with the studies of learning events which suggest that construct

ing or mo difying a principle is a delib erate pro cess that is often triggered when students

nd out they have made a mistake It should not matter when they learn that they have

made a mistake as long as they are able to lo cate the missing or incorrect knowledge and x

it However if they are not forced to correct their errors then the aws in their knowledge

may remain undiscovered and uncorrected

Although control of feedback do es not seem to aect the basic learning pro cess there

are other asp ects of immediate feedbackthatwarrant consideration by the instructional

designer Lewis Anderson found that their delayed feedback sub jects were much

b etter than the immediate feedback sub jects at detecting their own errors This makes

sense b ecause the immediate feedback sub jects had no opp ortunity to detect their own

errors during training Anderson et al found that feedback that forced students

to correct their errors immediately caused them to complete their training much faster

than feedback that lets students cho ose when to correct their errors Apparently students

sometimes waste time going down garden paths when they are given the freedom to do so

If instruction was limited to a xed amount of time this could hurt their learning

In the feedback studies just discussed tutors only notied students of their errors

They did not p oint out why the students actions were incorrect nor what should have done

minimal feedbackSeveral studies havecontrasted minimal instead This p edagogy is called

feedback with feedback designed to b e more helpful Help that is designed to get students

to infer the correct action increases their learning rate substantially compared to minimal

feedback Anderson Conrad Corb ett McKendree Mark Greer

On the other hand adding more help that fo cuses on what the student did wrong do es

not app ear to oer any advantages Sleeman et al That is if the feedback is The

right thing to do here is X then it do es little go o d to add You did Y instead Y is wrong

b ecause This nding is consistent with studies showing that more errors are caused

by missing knowledge rather than incorrect knowledge VanLehn c VanLehn

Missing knowledge often causes students to reach an impasse Rather than seeking help

they often invent an exp edient repair which allows them to continue but risks getting the

problem wrong VanLehn Moreover they often repair the same impasse in dierent

ways on dierent o ccasions VanLehn Thus it comes as no surprise that they do not

need muchconvincing in order to get them to abandon the particular repair that led them

to receivenegative feedback

Although tutoring do es provide a dierentcontext in which learning can o ccur it ap

p ears that the same basic learning pro cesses o ccur within it as o ccurred in passive example

studying instructional situations However b ecause learning is based on noticing aws b oth

incomplete and incorrect knowledge and repairing them tutoring can make learning sub

stantially more ecientby helping students to detect their aws via feedback to rectify

the aws via help and to improve the generality and accessibility of the new knowledge

by practicing it until mastery is reached

The nal phase practice eects

The intermediate phase is ocially over when students can pro duce errorfree p erformances

However this is not the end of their learning Continued practice causes increases in sp eed

and accuracy This section reviews these phenomena

The p ower law of practice and other general eects

Perhaps the most ubiquitous nding is the famous p ower law of practice The time to do a

task go es down in prop ortion to the numb er of trials raised to some p ower In an inuential

er law applies to simple cognitive review Newell Rosenblo om found that the p ow

skills as well as p erceptualmotor skills Several studies of complex cognitive skills eg

Anderson Conrad Corb ett Anderson Fincham found that the sp eed

of applying individual comp onents of knowledge increased according to a p ower law thus

indicating that practice b enets exactly the pieces of knowledge used and not the skill as

a whole Accuracy also increases according to a p ower law at least on some tasks Logan

Anderson Fincham

Several theories of the p ower lawhavebeenadvanced Anderson claims that the

sp eed up is due to twomechanisms knowledge is converted from a slow format declar

ative knowledge into a fast format pro cedural knowledge and the sp eed of individual

pieces of pro cedural knowledge also increases with practice Newell and Rosenblo om

Newell claim that small general pieces of knowledge are gradually comp osed together

chunked to form large sp ecic pieces of knowledge thus allowing the same task to b e ac

complished by applying fewer pieces of knowledge Logan advances an instancebased

theory and reviews several other prop osals

With substantial practice p erceptualmotor skills can b e sometimes b ecome automatic

Automatic pro cessing is fast eortless autonomous and unavailable to conscious awareness

Logan One study using a dualtask paradigm suggested that dierent parts of a

complex cognitive skill troublesho oting digital circuits b ecame automatic after diering

amounts of practice Carlson et al However the skill as a whole never b ecame

as automatic as driving ones car despite substantial practice problems with only

circuits

It is likely that other eects known to o ccur with p erceptualmotor skills also o ccur

with cognitive skills eg massed vs distributed practice warm up eects randomized vs

blo cked practice Pro ctor Dutta review researchonmanytyp es of skill and nd

several general eects

Replacing mental calculations by memory retrieval

Some cognitive skills are deterministic calculations in that the answer is completely and

uniquely determined by the inputs Mental arithmetic calculations are examples of suchde

terministic calculations If sub jects are given enough practice with a particular input then

they will eventually just retrieve the output from memory rather than mentally calculate

it This fairly uncontentious phenomenon has b een found with a number of simple mental

tasks Logan Anderson Fincham Healy et al as well as deterministic

subpro cedures of complex tasks Carlson et al Not only do the sub jects rep ort

this change in strategy but they are much faster in resp onding to practiced inputs than

unpracticed ones

The change in strategy from calculation to retrieval could b e taken as an explanation for

powerlaw increases in sp eed an accuracyHowever Rickards rep orted in Healy et al

had sub jects rep ort after each trial whether they had calculated the answer or retrieved it

The calculation trials latencies t one p ower curvewell and the retrieval trails latencies

t a second curvewell but the t of a p owerlaw curve to all trials latencies was p o or

Logan also found evidence for twopower curves Thus it app ears that p ower law

learning mechanisms op erate separatedly on b oth the mental calculation and the retrieval

strategies This hyp othesis is also consistent with a study by Carlson Lundy

who found using relatively complicated mental calculations that sub jects sp ed up rapidly

when given the same inputs rep eatedlythus promoting use of retrieval When the inputs

varied thus blo cking the retrieval strategy they still sp ed up alb eit more slowly The two

aects were additive Thus it app ears that changing from a calculation strategy to a direct

retrieval strategy is not in itself a sucient explanation for the p ower law of practice

Transfer of the b enets of practice

Although transfer has many meanings it is used in this section to mean the savings

in learning on one task the transfer task due to earlier training on a dierent task the

training task Transfer is expressed as a ratio the time saved in learning the transfer task

divided by the time sp ent learning the training task Thus if practicing the training task

for hours yields equivalent improvement on the transfer task as hours of practice on

the transfer task then there is or transfer If practicing the training task for

hours only saves hours of practice on the transfer task then the transfer is or

If prior practice with the training task makes no dierence in howlongittakes to learn the

transfer task then there is no transfer If practice on the training task increases the time to

learn the transfer task then there is negative transfer Singley Anderson chapter

discuss several ways to measure this typ e of transfer

One ma jor alb eit uncontroversial nding is that the degree of transfer can b e predicted

by the numb er of pieces of knowledge shared b etween the training and transfer tasks eg

Bovair Kieras Polson Singley Anderson As mentioned earlier practice

b enets individual pieces of knowledge and not the skill as a whole When practice on the

training task sp eeds up certain subskills they continue to b e fast when used in the transfer

task

However there are sometimes limits to the amount of transfer that one can obtain

even when substantial knowledge is shared b etween tasks This o ccurs b ecause practice

can change sub jects strategies for solving problems Supp ose that practice causes sub jects

to change from strategy A to B on the training task and from strategy A to C on the

transfer task A little training on the training tasks will aect only strategy A whichis

shared with the transfer task Thus T hours of practice on the training task will save T

hours of practice on the transfer task However this only o ccurs when T is less than the

amount of practice that will cause sub jects to shift from strategy A to B Let us supp ose

this shift o ccurs with around hours of practice When T exceeds then subsequent

practice only aects strategy B which is not shared with the transfer task Thus vast

amounts of training only save hours of practice on the transfer task so the amountof

transfer is T Thus when T is less than then transfer is TT or but when T

is greater than transfer is only T which approaches zero as practice on the training

task increases In short the more practice on the training task the less transfer

One strategy change discussed earlier is that practice can cause sub jects to use direct

memory retrieval instead of mental calculations As Logan and Carlson et al

have demonstrated this is one way that practice can cause a decrease in transfer Supp ose

the training task is to master a mental algorithm with one set of inputs and the transfer

task is to master the same algorithm with a dierent set of inputs As long as the sub ject

continues to use the algorithm during training the eects of that training should transfer

However as the sub ject starts to use direct memory retrieval instead of executing the

algorithm increasing the time sp ent in training will not reduce the time to master the

transfer task b eyond a certain p oint Thus practice decreases the amount of transfer

The practicedriven decrease in transfer can also b e caused byweaning the sub jects from

the instructional material which is another kind of strategy change As discussed earlier

sub jects in the intermediate phase of their training often refer to the instructional materials

usually the examples as they solve problems However the frequency of these references

declines with practice and eventually sub jects no longer refer to the instructional materials

Supp ose that the training and transfer tasks share some of their instructional materials

eg the examples Even a little practice on the training task familiarizes sub jects with

the shared instructional materials and thus saves them time in learning the transfer task

However with increasing practice on the training task sub jects gain no further b enet on

the transfer task Thus more practice causes less transfer This eect has b een found

in a variety of cognitive skills Singley Anderson Anderson Fincham

Anderson and his colleagues call this phenomena the use specicity of transfer and consider

it prime evidence for the distinction b etween declarative and pro cedural knowledge

Supp ose the training task is actually a part of the transfer task for instance as mental

addition is a part of mental multiplication Intuitively it seems that transfer should b e high

regardless of whether the training task is practiced for minutes or days However this do es

not seem to b e the case For relatively small amounts of practice the amountofoverlap

between tasks accurately predicts the amount of transfer Bovair Kieras Polson

Singley Anderson However when the training task is practiced for hundreds of

trials only a few trials of practice on the transfer task are saved even though the training

task is arguably a part of the transfer task Frensch Geary Needless to say more

research is required on this imp ortantpoint b ecause whole educational systems are founded

on the premise that training on basic skills facilitates learning practical skills that employ

the basic skills as subpro cedures Frensch Gearys result suggests that training basic

skills past a certain p ointiswasteful

In short a great deal of practice on one task the training task will help sub jects

p erform that task but it usually only saves them mo derate amounts of time in learning

a second task the transfer task In fact the head start that they get on learning the

second task after a great deal of practice on the rst task is ab out the same as the head

start they would get if they only practiced the rst task a mo derate amount The degree

of transfer observed and howitvaries with practice on the rst task dep ends on exactly

een the tasks and on when practice causes changes in problem solving what is shared b etw

strategies

Negative transfer

Negative transfer o ccurs when the training task interferes with learning the transfer task

and slows the learning down For instance mastering one text editor often seems to interfere

with learning a similar text editor that has dierent commands

The amount of transfer seems to b e inversely prop ortional to the amount of practice

on the transfer task Using text editors designed to interfere with each others learning

Singley Anderson showed that negative transfer o ccurs mostly during the early

stage of learning the transfer task If the learner learns the transfer task by receiving

immediate feedback whenever they do something wrong including habits carried over from

the training task then they rapidly acquire the correct resp onses On the other hand

uncorrected resp onses eg ones that are merely inecient and not incorrect p ersist and

can cause negative transfer even in the later stages of learning the transfer task

As discussed in the preceding section it is often the case that one gets the same time

savings in learning the transfer task regardless of the amount of practice on the training

task b eyond a certain minimum It would b e interesting to see if this were true of negative

transfer as well That is would mo derate amounts of training delay the learning of the

transfer task as muchasvast amounts of training To put it collo quially is negative

transfer due to bad ideas bad habits or b oth

The frontiers of cognitive skill acquisition research

Historically cognitive skill acquisition research started with simple knowledgelean puzzle

tasks moved on to problems suchastheoneofTable that can b e solved with a single

principle from a knowledgerich task domain and most recently has fo cused on chaptersized

slices of a knowledgerich task domain The next step in this progression is to study the

acquisition of even larger pieces of knowledge such as a whole semesters worth of physics or

programming Some early work along these lines has b een done where students learned from

ve examined acquisition a tutoring system eg Anderson et al but these studies ha

of only the pro cedural asp ects of the skill The next step would b e to examine how concepts

mental mo dels and factual knowledge evolve together with the more pro cedural asp ects

As done in this review research on cognitive pro cessing during the intermediate phase

of skill acquisition can b e conveniently divided into research on the acquisition of a single

principle and research on the acquisition of collections of principles and other knowledge

The singleprincipl e research seems to have answered many of the initial questions con

cerning retrieval mapping and generalization The remaining areas of uncertainty lie with

application It is likely that the students use the same diverse set of metho ds for applying

principles as they use for drawing analogies to complex written examples VanLehn a

Learning from examples has b een the fo cus of researchontheintermediate phase of

acquisition for complex chaptersized pieces of knowledge However most studies have

fo cused only on passive forms of instruction where the student studies written material

with no help from a teacher p eer or tutor The next step would b e to nd out how

students learn from more interactive forms of instruction For instance when students

receive feedback it seems plausible that p o or learners merely change their answer without

thinking ab out why they got it wrong whereas go o d learners would try to nd and repair

the aw in their knowledge that caused the error

The nal phase of skill acquisition has b ecome a battleground for some of the ma jor

theoretical questions of the day The controversy b etween instancebased eg Logan

pro ceduraldeclarative Anderson and other theories of memory have b een

mentioned already The debate over implicit learning eg Berry Broadb ent also

b ears most strongly on the nal phase

The rst resp onsibility of research on cognitive skill acquisition oughttobetoaccount for

the dierences b etween exp erts and novices Surprisingly it has not done so yet Ericsson

Lehmann this volume argue that in many task domains exp erts can mentally plan

solutions to problems that novices can only solve concretelyFor instance Ko edinger

Anderson found that exp ert geometers plan solutions to geometry pro of problems

using abstract diagramatic schemas Even though a pro of would consist of dozens of lines

the plan mighthave only one or twoschema applications The exp erts are able to do such

planning in working memoryNovices on the other hand build pro ofs a line at a time

writing down conclusions as they go The ability to rapidly plan solutions seems to develop

rather quickly Carlson et al

As Ko edinger Anderson p oint out no existing mo del of skill acquisition including

Andersons ACT can account for the acquisition of domainsp ecic planning skill Exist

ing mo dels have concentrated on explaining the p ower law of practice and the increasing

sp ecicity of transfer These results call for mechanisms that convert knowledge into ever

more sp ecic forms In contrast the development of planning skill calls for converting

knowledge into more abstract forms Articial Intelligence has pro duced may computation

ally sucientmechanisms for abstraction What our science needs is empirical work that

discriminates among the many p ossible ways that planning could develop For instance are

novices held backbyalack of knowledge of planning schemas or do they know the schemas

but cannot apply them mentally due to working memory limitations

Acknowledgements

The preparation of this chapter was supp orted by the Cognitive Sciences Division of the

Oce of Naval Research under grant NJ I gratefully acknowledge the com

ments of Micki Chi and Patricia Albacete

References

Anderson J The ArchitectureofCognition Harvard University Press Cambridge

MA

Anderson J R Rules of the MindLawrence Erlbaum Asso ciates Hillsdale NJ

Anderson J R Conrad F G and Corb ett A T Skill acquisition and the LISP

tutor Cognitive Science

Anderson J R Corb ett A T Ko edinger K R and Pelletier R Cognitive

tutors Lessons learned The Journal of the Learning Sciences

Anderson J R and Fincham J M Acquisition of pro cedural skills from examples

Journal of Experimental Psychology Learning Memory and Cognition

Bernardo A B I Problemsp ecic information and the development of problem

typ e schemata Journal of Experimental Psychology Learning Memory and Cognition

Berry E C and Broadb ent D E On the relationship b etween task p erformance and

asso ciated verbalizable knowledge The Quarterly Journal of Experimental Psychology

A

Bielaczyc K Pirolli P and Brown A L April Training in selfexplanation and

selfregulation strategies Investigating the eects of knowledge acquisition activities

on problemsolving Technical rep ort University of California at Berkeley Rep ort

CSM In press Cognition and Instruction

Bielaczyc K and Recker M Learning to learn The implications of strategy

instruction in computer programming In Birnbaum L editor The International

ConferenceontheLearning Sciences pages Asso ciation for the Advancement

for Computing in Education Charlottesville VA

Bovair S Kieras D E and Polson P G The acquisition and p erformance of

textediting skill A cogitive complexity analysis HumanComputer Interaction

Bransford J D Franks J J Vye N J and Sherwo o d R D New approaches to

instruction Because wisdom cant b e told In Vosniadou S and Ortony A editors

Similarity and analogical reasoning pages Cambridge University Press New

York

hildren can learn to transfer Learning to learn Brown A and Kane M Prescho ol c

and learning from example Cognitive Psychology

Carlson R A Kho o B H Yaure R G and Schneider W Acquisition of a

problemsolving skill Levels of organization and use of working memory Journal of

Experimental Psychology General

Carlson R A and Lundy D Consistency and restructuring in learning cognitive

pro cedural sequences Journal of Experimental Psychology Learning Memory and

Cognition

Carroll W M Using worked examples as an instructional supp ort in the algebra

classro om Journal of Educational Psychology

Catramb one R a The eects of lab els in examples on problem solving transfer In

Ram A and Eiselt K editors Proceedings of the Sixteenth Annual Conference of the

Cognitive ScienceSociety pages Lawrence Erlbaum Asso ciates Hillsdale NJ

Catramb one R b Improving examples to improve transfertonovel problems Mem

ory and Cognition

Catramb one R and Holyoak K Overcoming contextual limitations on problem

solving transfer Journal of Experimental Psychology Learning Memory and Cogni

tion

Catramb one R and Holyoak K Learning subgoals and metho ds for solving prob

ability problems Memory and Cognition

Catramb one R and Holyoak K J Transfer in problem solving as a function of the

pro cedural variety of training examples In Proceedings of the th Annual Conference

of the Cognitive ScienceSocietyLawrence Erlbaum Asso ciates Hillsdale NJ

Chi M Bassok M Lewis M Reimann P and Glaser R Selfexplanations

Ho w students study and use examples in learning to solve problems Cognitive Science

Chi M de Leeuw N Chiu MH and LaVancher C Eliciting selfexplanations

improves understanding Cognitive Science

Chi M and VanLehn K The contentofphysics selfexplanations The Journal of

the Learning Sciences

Co op er G and Sweller J Eects of schema acquisition and rule automation on

mathematical problemsolving transfer Journal of Educational Psychology

Corb ett A T Anderson J R and OBrien A T Student mo deling in the

ACT programming tutor In Nichols P D Chipman S F and Brennan R L ed

itors Cognitively Diagnostic Assessment pages Lawrence Erlbaum Asso ciates

Hillsdale NJ

Cummins D D Role of analogical reasoning in the induction of problem categories

Journal of Experimental Psychology Learning Memory and Cognition

Duncan C P Recent researchonhuman problem solving Psychological Bul letin

Ericsson K A and Lehmann A C Exp ert and exceptional p erformance Evidence

of maximal adaptation to task constraints This volume

Faries J and Reiser B Access and use of previous solutions in a problem solving

situation In Patel V and Gro en G editors Proceedings of the Tenth Annual Con

ference of the Cognitive ScienceSociety pages Lawrence Erlbaum Asso ciates

Hillsdale NJ

FergusonHessler M and de Jong T Studying physics texts Dierences in study

pro cesses b etweengoodandpoorsolvers Cognition and Instruction

Frensch P and Geary D C Eects of practice on comp onent pro cesses in com

plex mental addition Journal of Experimental Psychology Learning Memory and

Cognition

Gick M and Holyoak K Analogical problem solving Cognitive Psychology

Gick M and Holyoak K Schema induction and analogical transfer Cognitive

Psychology

Glaser R and Bassok M Learning theory and the study of instruction Annual

Review of Psychology

Healy A F and L E Bourne J Learning and Memory of Know ledge and Skil ls

Durability and Specicity Sage Publications Thousand Oaks CA

Holyoak K J and Koh K Surface and structural similarity in analogical transfer

Memory and Cognition

Jones R M and VanLehn K Acquisition of childrens addition strategies A mo del

of impassefree knowledgelevel learning Machine Learning

Kanhey H Problem Solving Current issues Second edition Op en University

Press Buckingham

KarmiloSmith A and Inhelder B If you want to get ahead get a theory

Cognition

Ko edinger K and Anderson J R Abstract planning and p erceptual chunks

Elements of exp ertise in geometry Cognitive Science

Kuhn D and Phelps E The development of problemsolving strategies Advances

in Child Development and Behavior

LeFevre J and Dixon P Do written instructions need examples Cognition and

Instruction

Lewis C Why and how to learn why Analysisbased generalization of pro cedures

Cognitive Science

Lewis M W and Anderson J R Discrimination of op erator schemata in problem

solving Learning from examples Cognitive Psychology

oward an instance theory of automatization Psychological Review Logan G D T

Lovett M C Learning by problem solving versus by examples The b enets of gen

erating and receiving information In Proceedings of the Fourteenth Annual Conference

of the Cognitive ScienceSocietyLawrence Erlbaum Asso ciates Hillsdale NJ

Mark M A and Greer J E The vcr tutor Eective instruction for device

op eration The Journal of the Learning Sciences

McKendree J Eective feedbackcontent for tutoring complex skills Human

Computer Interaction

Newell A Unied Theories of Cognition Harvard University Press Cambridge

MA

Newell A and Rosenblo om P Mechanisms of skill acquisition and the lawof

practice In Anderson J editor Cognitive Skil ls and Their Acquisition pages

Lawrence Erlbaum Asso ciates Hillsdale NJ

Newell A and Simon H A editors Human Problem SolvingPrentriceHall Inc

Englewo o d Clis NJ

Norman D A Categorization of action slips Psychological Review

Novick L Analogical transfer problem similarity and exp ertise Journal of Exper

imental Psychology Learning Memory and Cognition

oak K Mathematical problem solving by analogy Journal of Novick L and Holy

Experimental Psychology Learning Memory and Cognition

Pirolli P Eects of examples and their explanations in a lesson on recursion A

pro duction system analysis Cognition and Instruction

Pirolli P and Anderson J The role of learning from examples in the acquisition

of recursive programming skills Canadian Journal of Psychology

Pirolli P and Bielaczyc K Empirical analyses of selfexplanation and transfer in

learning to program In Proceedings of the Eleventh Annual Conference of the Cognitive

ScienceSociety pages Hillsdale NJ Lawrence Erlbaum Asso ciates

Pirolli P and Recker M Learning strategies and transfer in the domain of pro

gramming Cognition and Instruction

PressleyMWo o d E Woloshyn V Martin V King A and Menke D Encour

aging mindful use of prior knowledge Attempting to construct explanatory answers

facilitates learning Educational Psychologist

Psotka J Massey L D and Mutter S A Intel ligent tutoring systems Lessons

learnedLawrence Erlbaum Asso ciates Hillsdale NJ

Recker M M and Pirolli P Mo deling individual dierences in students learning

strategies The Journal of the Learning Sciences

Reed S K Dempster A and Ettinger M Usefulness of analogous solutions

for solving algebra word problems Journal of Experimental Psychology Learning

Memory and Cognition

Reed S K Willis D and Guarino J Selecting examples for solving word prob

lems Journal of Educational Psychology

Ross B Remindings and their eects in learning a cognitive skill Cognitive Psy

chology

Ross B This is like that The use of earlier problems and the separation of similarity

eects Journal of Experimental Psychology Learning Memory and Cognition

Ross B Distinguishing typ es of sup ercial similarities Dierent eects on the

access and use of earlier problems Journal of Experimental Psychology Learning

Memory and Cognition

Ross B and KennedyP Generalizing from the use of earlier examples in prob

erimental Psychology Learning Memory and Cognition lem solving Journal of Exp

Siegler R S and Jenkins E How Children Discover New StrategiesLawrence

Erlbaum Asso ciates Hillsdale NJ

Singley M and Anderson J The transfer of cognitive skil lHarvard University

Press Cambridge MA

Sleeman D Kelley A E Martinak R Ward R D and Mo ore J L Studies

of diagnosis and remediation with high scho ol algebra students Cognitive Science

Sweller J and Co op er G The use of worked examples as a substitute for problem

solving in learning algebra Cognition and Instruction

VanLehn K Problem solving and cognitive skill acquisition In Posner M editor

Foundations of Cognitive Science pages MIT Press Cambridge MA

VanLehn K Rule acquisition events in the discovery of problem solving strategies

Cognitive Science

VanLehn K a Analogy events How examples are used during problem solving

Submitted for publication

VanLehn K b Lo oking in the b o ok The eects of exampleexercise analogy on

learning Submitted for publication

VanLehn K c Rule learning events in the acquisition of a complex skill Submitted

for publication

VanLehn K and Jones R Learning by explaining examples to oneself A com

putational mo del In Chipman S and Meyrowitz A editors Cognitive Models of

Complex Learning pages Kluwer Academic Publishers Boston MA

VanLehn K and Jones R M Is the selfexplanation eect caused by learning

rules schemas or examples Submitted for publication

VanLehn K Jones R M and Chi M T H A mo del of the selfexplanation

es eect The Journal of the Learning Scienc

Voss J Wiley J and Carretero M Acquiring intellectual skills Annual Review

of Psychology

WK Ahn W F B and Mo oney R J Schema acquisition from a single example

Journal of Experimental Psychology Learning Memory and Cognition

Zhu X and Simon H A Learning mathematics from examples and by doing

Cognition and Instruction