J Comp Physiol A (2002) 188: 851–866 DOI 10.1007/s00359-002-0351-5

PROXIMATE MECHANISMS OF SONG LEARNING

D. Margoliash Evaluating theories of bird song learning: implications for future directions

Received: 18 February 2002 / Revised: 1 July 2002 / Accepted: 5 September 2002 / Published online: 13 November 2002 Ó Springer-Verlag 2002

Abstract Studies of birdsong learning have stimulated ventrale Æ clHV lateral subdivision of the caudal region extensive hypotheses at all levels of behavioral and of hyperstriatum ventrale Æ cmHV medial subdivision physiological organization. This hypothesis building is of the caudal region of hyperstriatum ventrale Æ DLM valuable for the field and is consistent with the re- medial subdivision of the dorsal lateral nucleus of markable range of issues that can be rigorously ad- anterior thalamus Æ DM dorsomedial subdivision of dressed in this system. The traditional instructional nucleus intercollicularis Æ DMP dorsomedial nucleus of (template) theory of song learning has been challenged posterior thalamus Æ HV ventral hyperstriatum Æ HVc on multiple fronts, especially at a behavioral level by acronym used as the proper name Æ lMAN lateral evidence consistent with selectional hypotheses. In this subdivision of the magnocellular nucleus of the anterior review I highlight the caveats associated with these neostriatum Æ MLD dorsal lateral nucleus of theories to better define the limits of our knowledge and mesencephalon Æ mMAN medial subdivision of the identify important experiments for the future. The sites magnocellular nucleus of the anterior neostriatum Æ Ncm and representational forms of the various conceptual caudal medial neostriatum Æ Nd dorsal caudal neostria- entities posited by the template theory are unknown. tum Æ NIf nucleus interfacialis Æ NMDA N-methyl- The distinction between instruction and selection in D-aspartate Æ Ov nucleus ovoidalis Æ RA robust nucleus vocal learning is not well established at a mechanistic of the archistriatum Æ RAm nucleus retroambigualis Æ level. There is as yet insufficient neurophysiological data Uva nucleus uvaeformis to choose between competing mechanisms of error- driven learning and reinforcement learning. Both may obtain for vocal learning. The possible role of sleep in Introduction acoustic or procedural memory consolidation, while supported by some physiological observations, does not Perhaps the most influential theory of birdsong learning yet have support in the behavioral literature. The re- is also one of the earliest. Proposed some 40 years ago, markable expansion of knowledge in the past 20 years the template theory posits a series of conceptual objects, and the recent development of new technologies for an innate predisposition for conspecific songs, an innate physiological and behavioral experiments should permit template that guides vocal learning in the absence of direct tests of these theories in the coming decade. song models, and an acquired template formed when song models (typically, the songs of adult conspecifics) Keywords Error-driven and reinforcement learning Æ are available to the animal during the critical period Instruction and selection Æ Sleep and replay Æ Template (sensory phase) for song acquisition. The acquired theory template is conceived of as the internal sensory repre- sentation of external song models that is used to judge or Abbreviations AFP: anterior forebrain pathway Æ Am guide the young bird’s own vocal output. The form of nucleus ambiguus Æ cHV caudal region of hyperstriatum the internal representation is unspecified and is an ex- citing unresolved problem – there may be many possible forms for neural representations of sounds (e.g., see re- D. Margoliash view by Gentner and Margoliash 2002). The innate or Department of Organismal Biology and Anatomy, acquired template guides vocal learning through a pro- University of Chicago, 1027 E. 57th St., Chicago, IL 60637, USA cess of trial-and-error auditory feedback-mediated E-mail: [email protected] learning. The critical features of the sensorimotor Fax: +1-773-7020037 (feedback-mediated) learning model are the sensory 852 template and the use of error-correction. A deaf bird, geographic domains. Thus, the vocal diversity of indi- that has no access to the feedback signal, cannot correct viduals within a species may result from choosing par- errors and so his song wanders. Deafening prior to song ticular exemplars from a range of combinations of a crystallization immediately arrests vocal development relatively small set of primitives (see Marler 1997). These because it prevents access to the feedback error signals observations are consistent with a selectional model of that are compared with vocal output to guide further learning (Marler and Peters 1982a; Nelson 1992a; Nel- development (Konishi 1965, 1978, 1985; Marler 1970). son and Marler 1994). In a selectional model, the envi- The template model is an instructional model in that the ronment influences the choice of a small number of environment strongly and directly guides the song de- patterns derived from a larger set of patterns that reflect velopment, both in the sensory phase and in the senso- the natural variation in behavior that an organism rimotor phase. The template concept implies immediacy produces. of correcting vocal output during singing – feedback is Field sparrows and white-crowned sparrows may compared with a template (target) vocalization, and the develop more than one song type, then express a given errors are immediately applied to change the vocal song type based on the prevalent songs a bird encounters output. No long-term memory of the errors is posited. (Nelson 1992a, 1992b). In some species such as indigo The template theory is a succinct statement that has buntings, individuals may also maintain a private rep- stimulated a large body of behavioral research. As the ertoire of syllables (Margoliash et al. 1994a) that are leading theory it has been the natural target of multiple expressed only rarely during aggressive encounters tests. Weaknesses have been exposed in that a purely (Emlen 1972). In principle, these could be accessed to acoustic account of vocal learning does not generally achieve a rapid, ‘‘action-based’’ learning process (Marler obtain. Depending on species, birds may rely on multiple and Nelson 1993), a form of immediate copying of vo- cues as to which songs to attend to. For example, young calization. The rapid changes in song that yearling in- indigo buntings and zebra finches require social interac- digo buntings exhibit, in the extreme case completely tions to acquire songs (Payne 1981; Adret 1993). Juvenile remodeling their songs in a period of 24 h, is evidence white-crowned sparrows will learn song models prefer- for this hypothesis (Payne et al. 1988). As with the more entially from heterospecific live tutors that they can see, traditional selectional model, action-based learning ap- even if the juvenile sparrows are in acoustic contact with pears to vary from the template theory of song learning live conspecifics who sing (Baptista and Petrinovich in that the influence of the environment is more strongly 1984). Cowbirds, which are parasitic birds raised by alien constrained. This distinction may not be compelling, parents, face unusual problems as to whom to learn song however, since action-based learning is proposed for from and when. Remarkably, certain wing strokes of fe- animals that have already passed through early vocal male cowbirds in response to juvenile male songs strongly development. The role of feedback in action-based ac- shape the development of those songs (West and King quisition of new vocal material has yet to be well de- 1988). Live tutors and social interactions can perhaps scribed. There is neurobiological evidence for influence all stages of song learning. These and many re- hierarchical organization of the motor program for song lated observations do not falsify the template theory of (see below). If action-based learning is limited to song learning, but they significantly extend it with a bio- changes in patterns of expression of pre-existing vocal logical richness more consistent with the large variation in elements, then the rapidity of learning may result from behavior observed across multiple species. needing to coordinate feedback-mediated changes only A more fundamental challenge to the template theory at higher levels of the hierarchy. Such a hypothesis is of birdsong learning has questioned the extent to which neutral to the question of whether the coordination is song learning is driven by external stimuli. During vocal mediated through instructional or selectional mecha- development, an overabundance of vocal material is nisms. produced which is later whittled down when the final Selectional and instructional learning may be distinct adult song(s) crystallize. This excess material is related in at a theoretical level yet difficult to differentiate at a part to which songs birds are tutored with early in life, neural level. The potential form of a selectional process but birds can produce normal syllables to which they is highlighted by the observation that species-universals have never been exposed (e.g., Marler and Peters 1981). do not fully develop in isolate-reared or deafened birds The songs a young bird is exposed to early in life in- (Marler and Sherman 1983, 1985). Isolate song may fluence which sounds are maintained into the adult song have some species-typical structure (Konishi 1978), and and which are discarded. In song sparrows, material that females may give weak but significant responses to is faithfully copied from tutors tends to persist into conspecific isolate songs (Searcy et al. 1985). This im- adulthood in favor of material that is ‘‘improvised’’ or plies that innate representations of song cannot express ‘‘invented’’ (Marler and Peters 1982a). In several species, themselves in the absence of an appropriate auditory there is also evidence suggesting the existence of a set of stimulus (i.e., conspecific song). Under this model, the universals in vocal production (vocal features frequently distinction between selection and instruction may be- shared across individuals) (Thompson 1970; Marler and come somewhat arbitrary. If multiple, innately specified Pickert 1984). Similar vocal primitives or syllables are neuronal patterns are released by conspecific song ex- observed in many individuals dispersed across large posure, but when initially expressed the structure of the 853 neuronal patterns is not modified by the specific acoustic development can provide positive results. Negative re- features of the conspecific songs, then this can be the sults are difficult to interpret – a bird that sings an ab- substrate for subsequent modification by selection. If, on normal adult song may have failed to learn because of the other hand, upon exposure to song, novel patterns of sensory constraints, motor constraints, modified social neural connections or novel patterns of neural activity environment, or combinations thereof. Young birds are formed that depend on classes of acoustic features tend to copy the songs of their own species if presented shared across conspecific songs, then the distinction with a choice of conspecific and heterospecific songs in a between selection and instruction is not as clear. Nev- balanced experiment (e.g., Marler 1970; Konishi 1985). ertheless, collectively, these data emphasize the impor- Nevertheless, in some cases, the use of live tutors has tance of innate representations upon which a sensory induced birds to sing elements of the songs of sympatric template of song must act, and are probably the best species in preference to conspecific songs (Baptista and evidence in support of a selectional model (Marler 1997). Petrinovich 1984, 1986; Clayton 1989). This elegantly Other data cited, such as the limited number of songs addresses the issue of motor constraints and demon- required to establish a memory, and the robustness of strates the importance of social interactions in acquisi- local dialects, are difficult to interpret unambiguously as tion of the sensory template (e.g., see also Beecher 1996). resulting from either instructional or selectional pro- Tests have also been designed to assess the perceptual cesses. Comparison of similarity between neighbors’ preferences of very young birds, again demonstrating a songs in sedentary and migratory subspecies of white- preference for own-species songs (Dooling and Searcy crowned sparrows is also consistent with a selectional 1980; Nelson and Marler 1993). It would be valuable to hypothesis (Nelson et al. 2001). confirm the utility and generality of these procedures Recently, the immediacy of vocal learning – the con- across species. It will also require additional work to cept that adaptive vocal modification is limited to daytime verify the relation between the behavioral preferences periods of singing – has been challenged by physiological observed in these studies and the behavioral preferences observations demonstrating a song ‘‘replay’’ phenome- that affect which songs are committed to memory as the non in sleeping birds. This has been interpreted as part of a acquired template. mechanism for procedural memory consolidation during Ethologists have long argued that perception is in- sleep (Dave and Margoliash 2000). As yet this is only a fluenced by species-specific processes, and this extends to suggestion, and lacks support in the behavioral data. The perception of vocalizations (e.g., Zoloth et al. 1979). It null hypothesis, that the physiological phenomena ob- should therefore be possible in songbirds to demonstrate served during sleep are an epiphenomenon of some other systematic differences in the neural representation of aspect of song learning, has yet to be rejected (but see conspecific and heterospecific songs that are indepen- below). For example, during sleep, song system neurons dent of the specific acoustic features of any one song. exhibit stronger responses to song playback than in awake Such representations might involve the synchronicity in birds (Dave et al. 1998b; Nick and Konishi 2001). This activation of populations of neurons, or the accuracy could reflect an interaction of mechanisms to modulate with which songs can be reconstructed from neurons’ auditory input to the song system in singing birds with spike discharges. If such representations were identified changes in neuromodulator levels associated with sleep, in adult birds, it would be valuable to search in juvenile resulting in a physiological property during sleep that has birds for such representations in the classical auditory no behavioral consequence. It is generally not appreciat- pathway and the specialized pathway that gives rise to ed, however, that there is an equal paucity of data sup- auditory input to the song system (see below). If com- porting immediate vocal modification during song bined with cross-fostering experiments, for example be- development (e.g., see West and King 1988; Brainard and tween zebra finches and Bengalese finches, such Doupe 2000a). The first microscopic analysis of singing experiments could hope to identify sites where an innate patterns during vocal development that could detect species-specific mechanism is present to analyze songs. moment-to-moment changes in vocal performance during plastic singing has only recently been reported (Tcher- nichovski et al. 2001; see below). Acquired template

Sources of auditory input to the song system How and where are the objects of the template theory expressed in the brain? The acquired template should be closely associated with auditory inputs to the song system or auditory pathways Innate predisposition for conspecific songs within the song system. Identifying the primary sources of auditory input to the song system has proven to be Behavioral tests of theories of song learning have been challenging, but may have potentially been resolved in technically challenging because longitudinal studies are the past few years. Initial studies focused on direct difficult. There are conceptual challenges as well. projections of field L, the principal auditory thalamo- Experiments that assess the final outcome (song) after recipient area in passerines (Karten 1968), to the song manipulation of some aspect of the environment during system. Field L projects to a ‘‘shelf’’ region just ventral 854 to HVc (Kelley and Nottebohm 1979). Intracellular la- beling of single shelf neurons indicated that the axons of shelf neurons seem to avoid rather than invade HVc (L. Katz, cited in Margoliash 1987), and a robust projection of the shelf into HVc was not observed using other tracer techniques (Fortune and Margoliash 1995; Vates et al. 1996). Since the dendrites of ventral HVc neurons extend into the shelf, however, a plausible hypothesis was that axons from auditory neurons coursing through the shelf synapse onto HVc dendrites projecting into the shelf, serving to provide auditory input to HVc. This hypothesis has been popular for many years but has never achieved substantial anatomical or physiological verification. A subsequent study argued for a direct if relatively sparse projection of field L directly into HVc, but could not confirm this connection in the anterograde direction because nucleus uvaeformis (Uva) and nucleus interfacialis (NIf) fibers that project to HVc pass Fig. 1 Connections of the song system. Am nucleus ambiguus, through field L (Fortune and Margoliash 1995). Golgi DLM medial subdivision of the dorsal lateral nucleus of anterior studies described exceptional cases of axons of field L thalamus, DM dorsomedial subdivision of nucleus intercollicularis, neurons coursing in the general direction of the shelf and DMP dorsomedial nucleus of posterior thalamus, HV ventral HVc but were not designed to resolve long-distance hyperstriatum, lMAN lateral subdivision of the magnocellular nucleus of the anterior neostriatum, MLD dorsal lateral nucleus of connections (Saini and Leppelsack 1977). An intracel- mesencephalon, mMAN medial subdivision of the magnocellular lular study to address these issues has never been re- nucleus of the anterior neostriatum, Ncm caudal medial neostria- ported. These pathways remain as plausible but tum, Nd dorsal caudal neostriatum, NIf nucleus interfacialis, Ov unsubstantiated sources of auditory input to the song nucleus ovoidalis, RA robust nucleus of the archistriatum, RAm system. They do not represent the robust set of con- nucleus retroambigualis, Uva nucleus uvaeformis nections one might expect for a sensory feedback ele- ment so critical to a songbird’s reproductive success. goliash 2001). Collectively, these results implicate the HV More recently, a second pathway has been described as part of a specialized pathway in the avian forebrain for that appears to be a major source of auditory input into the perception of conspecific songs. NIf neurons are se- the song system (Vates et al. 1996). The lateral subdivision lective for the bird’s own song, a signature property of of the caudal region of the hyperstriatum ventrale (clHV) song system neurons, and were reported to be less song- is reciprocally connected to field L subdivisions, and also selective than are HVc neurons, although the sample size receives a projection from the medial subdivision of the was small (Janata and Margoliash 1999). Furthermore, caudal region of the hyperstriatum ventrale (cmHV). In recent results indicate that reversible lesions of NIf abol- turn, cmHV receives input from the caudal medial neo- ish HVc auditory activity (Boco and Margoliash 2001). striatum (Ncm), and Ncm receives input from subdivi- NIf input could represent the subthreshold song selective sions of field L and from a shelf region of the auditory auditory input to HVc observed in intracellular record- thalamic nucleus ovoidalis (see Fig. 1). The multiple in- ings, which is then further refined within HVc (Mooney puts to clHV give it access to auditory information which, 2000). Thus, there is a hierarchy with regard to processing in turn, would become available to NIf, to which clHV of conspecific songs and the bird’s own song comparing projects. NIf has a strong and direct projection to HVc HV, NIf, and cell classes within HVc. (Nottebohm et al. 1982) but has been difficult to study HVc is the source of auditory input to RA and to the because of its tortuous structure and proximity to field L anterior forebrain pathway (AFP) (Doupe and Konishi subdivisions (Fortune and Margoliash 1992). Recently, 1991; Vicario and Yohay 1993). It is unclear whether the on-line cross-correlation between HVc and NIf electrodes apparent sensory hierarchy observed comparing HV- has been used to reliably verify the placement of electrodes NIf-HVc extends to the AFP. This is difficult to assess within NIf (Janata and Margoliash 1999). without comparing song selectivity of AFP neurons with Physiological evidence supports a role for specialized a pure population of area X-projecting HVc neurons, auditory input to the song system via the ventral hyper- which has yet to be accomplished. Recent intracellular striatum (HV). In adult zebra finches, HV neurons show recordings suggest that HVc-Xn are highly selective for non-linear sensitivity to complex acoustic features of song, focusing attention on this caveat (Mooney 2000). conspecific songs (Theunissen et al. 2000; Sen et al. 2001). Furthermore, the metrics used to assess song selectivity Recent evidence in adult starlings conditioned in an op- (typically, spontaneous-corrected mean firing rates) may erant apparatus to associate different groups of conspec- not be reliable when comparing neurons with very dif- ific songs with one key or another suggests that HV ferent ongoing firing rates and phasic/tonic patterns of neurons acquire highly selective responses to songs as response (cf. Doupe 1997). Intracellular recordings from birds learn to categorize those songs (Gentner and Mar- the lateral subdivision of the magnocellular nucleus of 855 the anterior neostriatum (lMAN) coupled with local song learning. Controls received comparable lMAN in- circuit blockade demonstrate that the song selectivity of jections on non-tutoring days or in the cerebellum. The lMAN neurons is largely accounted for by the song se- experimental birds showed poorer copying of the tutor lectivity of its inputs (Rosen and Mooney 2000a). This songs than did the control birds (Basham et al. 1996). This too is inconsistent with a hierarchical model of song is an intriguing experiment that was interpreted in terms selective auditory processing in the AFP. of demonstrating that NMDA receptor activation during The recent anatomical and physiological data suggest sensory learning is required for normal song development. that pathways in adults that are apparently specialized A variety of caveats were noted (Basham et al. 1996). The for processing conspecific songs give rise to auditory cerebellar control injections and the predicted effective inputs to the forebrain song system. This suggests the size of the injection (radius <2 mm) do not rule out in- plausible hypothesis that in juvenile birds, field L2b volvement of other nearby AFP nuclei – such as area X – (which projects to Ncm), Ncm and/or HV neurons are when lMAN was targeted, nor involvement of other song modified during the sensory phase of song learning. system nuclei in the anterior forebrain such as mMAN. These neurons would then influence song system neu- The alternate-day controls only narrow the effective time rons (NIf, HVc, and beyond), which would develop their window of the injections to one full circadian cycle. In own, song-system specific properties based on auditory general, as with all lesion studies, assessing the functional inputs from HV and the multiple other influences on role of the AFP nuclei based on lesions can be difficult – song system neurons. The internal feedback within the the AFP-directed lesions could act by blocking signals forebrain auditory pathways and the song system (e.g., passing through them and by modifying processing in Wild 1993; Vates and Nottebohm 1995; Vates et al. afferent targets, which could potentially include HVc, 1996), and the extended time-course of normal sensory RA, and other regions in archistriatum (Johnson et al. acquisition, could serve to distribute the template in- 1995), as well as by acting on the AFP itself. If multiple formation over multiple structures that change over forebrain sites are recruited during sensory template time. In this hypothesis, the songbird would not have a memory consolidation, this could provide evidence for a distinct, localized engram for the tutor song. distributed representation of the template (Nottebohm Perhaps the most common countervailing hypothesis 1991). Presumably, the different sites of this distributed to the notion of a distributed representation of the template would serve different perceptual roles (e.g., song template is the hypothesis that the sensory acquired memorization) and sensorimotor roles (e.g., the many-to- template has a distinct locus, and that locus is within the many sensory to motor mapping). AFP. For example, Troyer and Doupe (2000a) favored this hypothesis in a modeling study of sensorimotor learning. The physiological evidence for the AFP-local- Error correction ization hypothesis is weak, however. At an intermediate stage of vocal development, AFP neurons respond to The template theory is focused on the sensory phase of both tutor and bird’s own songs (Solis and Doupe 1997). song development and does not explicitly address the This is observed even in birds singing abnormal songs as form of error correction during sensorimotor learning. a result of denervation of the syrinx, suggesting that the The template concept, however, makes implicit predic- responses to own song and tutor song reflects an active tions. If the acquired template is a sensory structure, process, not just similarities in acoustic features that the then the error computed during sensorimotor learning is two songs share (Solis and Doupe 1999). These data are computed in a sensory frame of reference. If the com- consistent with a role of the AFP in some form of puted error has detailed information about how to evaluating an efference copy signal, which may also re- change the song, then the template serves to directly quire information about the acquired template (Solis instruct the development of song. These predictions et al. 2000). The source of the representation of the tutor represent strong constraints on the mechanisms of song songs observed in the AFP, and the sites where physi- learning. Ultimately, such hypotheses are falsifiable, and ological properties are modified early in development probably verifiable, with presently available techniques. during sensory learning, however, are not established. This lends continued vitality to the template theory of Evidence of a representation of the tutor song in the birdsong learning as it was originally envisioned. AFP does not rule out its existence prior to the AFP, for example in HVc or in auditory structures afferent to the song system, or within parallel pathways. Instructional and selectional models: Another experiment has been cited as evidence for lo- tests and implications for brain organization cating the sensory template within the AFP (e.g., Troyer and Doupe 2000a). In that experiment, Basham et al. Direct observations during development (1996) tested the role of N-methyl-D-aspartate (NMDA) receptors in the sensory acquisition phase of song learn- Song learning normally occurs over an extended period ing. Juvenile zebra finches received injections of an of time whose termination may be well-defined by sudden NMDA receptor antagonist into lMAN just preceding emergence of crystallized song (Marler and Peters 1982b) tutoring sessions during days of the critical period for but whose onset is often poorly defined. Longitudinal 856 analyses of song learning have relied on manually scor- also be a set of rules, and a set of algorithms to decide ing of spectrographs (e.g., Marler and Peters 1982a, which rules are applied to which parts of song under 1982b; Margoliash et al. 1991). These are labor intensive which conditions. Such rules are unlikely to be imple- and do not lend themselves well to post hoc hypotheses mented in the purely sensory acquired template as and re-analysis. Also, the human observer is excellent at originally envisioned in the template theory of song scoring categorical material such as syllables, but is learning. The rules and algorithms could be represented less reliable at scoring the complex variable patterns of by constraints on song motor programs; if so, the ac- songs produced early during development. quired template could be specified not in purely sensory A recent set of studies has conquered several of these but instead in sensorimotor coordinates. Vocal devel- problems and provided new insight into the vocal de- opment could represent a trajectory towards targets in velopment process. A procedure to delay and control the an increasingly restrictive energy space (Margoliash onset of song learning in zebra finches was developed. 2001a). As with other species (e.g., Todt et al. 1979; Hultsch and To try to directly address the issue of selection and Todt 1989), few songs are required for zebra finches to instruction, Tchernichovski et al. (2001) analyzed the learn their songs, but surprisingly, it was observed that distribution of four features over 10-s time intervals of exposure to numerous songs can inhibit learning singing. At least three of the features (Wiener entropy, (Tchernichovski et al. 1999). Signal processing proce- pitch and spectral continuity) were highly positively (or dures to automatically score the songs of birds were also negatively) correlated in the immediate period sur- developed; these were applicable to variable plastic rounding the onset of vocal changes for the one bird songs as well as crystallized adult songs (Tchernichovski continuously recorded. It will be important to verify et al. 2000). In the developmental studies, birds were whether considering only one feature gives all the in- monitored from the earliest stages of sensorimotor de- formation about vocal development, as it did for that velopment until songs crystallized. Vocal ontogeny was one bird. The distributions of the sets of features were then traversed backwards to determine the antecedents used to define a measure of feature ‘‘density’’. By this of the material of the crystallized songs (Tchernichovski measure, the diversity of sounds increased after song et al. 2001). The results of these remarkable studies are model exposure and initiation of sensorimotor learning. reviewed in detail elsewhere (Tchernichovski and Mitra Although this ‘‘generative’’ process seems to support an 2002). Some technical issues remain that can be ad- instructional learning model, considerable caution is dressed in subsequent experiments. The tutoring para- warranted. First, as noted above, it is unclear how the digm could potentially affect song development in delay-learning paradigm affects normal song develop- unanticipated ways. For example, by delaying the onset ment. Compression of the early period of vocal devel- of song exposure, and by causing the onset of song ex- opment may make it particularly hard to detect posure and sensorimotor learning to coincide, the ear- production of multiple vocal patterns as envisioned in a liest stages of song development may be abnormal, and selectional model. Second, the feature density measure is abbreviated. In the learning-delayed paradigm, zebra advantageous in that it does not require partitioning or finches may experience some abnormal singing during classifying the vocal signal, as has been required with the delay period. Finally, it would also be valuable to prior automatic techniques applied to bird song analysis assess the validity of the vocal analysis procedures on (e.g., Anderson et al. 1996; Ito et al. 1996; Kogan and datasets of other species songs. Margoliash 1998). It is not well established, however, The zebra finches studied by Tchernichovski et al. how changes in signal processing feature distributions (2001) sometimes followed a direct trajectory of vocal detected by Tchernichovski et al. (2001) map onto development, incrementally increasing the match be- changes in distinct motor patterns during vocal devel- tween the acoustic features of the tutored and learned opment. It is assessment of the latter which is the es- sound. In other cases, an indirect trajectory was fol- sential behavioral data required to evaluate instructional lowed, for example when juveniles increased the fre- and selectional hypotheses. quencies of a sound over several days then abruptly Finally, the study by Tchernichovski et al. (2001) doubled the period (halving the frequency) to achieve a observed changes in vocal behavior during the day as match with a tutor syllable. Which syllables and which juvenile birds learn to sing. This is the first direct evi- components of syllables were transmuted into new ones dence in the literature for immediate, ‘‘on-line’’ learning, was more a function of the sequential position of a assuming that the reported changes occurred within a sound than how closely it already matched the tutor single session of singing and were not interrupted by material prior to transformation. These observations periods of rest. It is a provocative observation that imply that zebra finches follow an innate set of rules or Tchernichovski et al. (2001) also reported that dramatic procedures (e.g., ‘‘modify in-place rather than change changes, and the first significant changes in vocal out- the temporal sequence of the parts’’; ‘‘above a threshold put, were delayed by one full circadian cycle – the ju- slowly increase the frequency and then halve it’’) to veniles did not start exhibiting strong learning effects achieve vocal copying. An acoustic representation of the until the 2nd day after they were exposed to new songs. tutor song is insufficient to specify the mapping between Thus, it appears that the intervening period of sleep may the model and the bird’s developing song. There must have influenced the vocal behavior of the birds. This 857 interpretation must also be made cautiously, however. combine observations of juvenile RA neurons with the The study by Tchernichovski et al. (2001) was not de- paradigm of modifying a bird’s vocal output during signed to examine the role of sleep in learning, and the development, for example by sectioning the tracheosyr- possibility that this result is an artifact of some other ingeal nerves. This experiment may provide a strong test aspect of the experimental design cannot be dismissed. to favor either interpretation: (1) that song selectivity of Perhaps vocal modification during the first day of ex- young RA neurons reflects an innate, selectional mech- posure to new songs was missed. If the result stands, anism, or (2) that the selectivity reflects developmental however, it would provide strong support for the hy- expression consistent with an instructional mechanism. pothesis of ‘‘off-line’’ learning, during periods of quies- Experiments to directly examine brain mechanisms of cence or during sleep (see below). vocal development are challenging. Vocal material dur- Beyond the behavioral analyses, there have been few ing song development is complex and variable. Physio- studies that examined physiological properties of song logical experiments should optimally be conducted in system neurons in developing birds. In anesthetized young, behaving animals. A hope for the future is to white-crowned sparrows, HVc neurons showed selec- combine the recent technical advancements that enable tivity for plastic song that apparently developed as the single cell recordings in relatively unrestrained small birds engaged in singing (Volman 1993). A similar ob- birds (Dave et al. 1998a; Fee and Leonardo 2001) and servation has been made for lMAN neurons in anes- analysis of juvenile song (Tchernichovski et al. 2000). If, thetized zebra finches, with the additional important in this environment, auditory feedback can be brought point that those neurons can also exhibit strong re- under precise control, the technical requirements for sponses to tutor songs (Solis et al. 2000; see above). The conducting the experiments will be at hand. results of those studies have been interpreted in terms of This vision of the future is close to becoming a reality. instructional learning. Recent, preliminary results ex- It is technically challenging, so that until recently most tend these observations and challenge their interpreta- physiological work has been conducted on adult birds, tion (Adret and Margoliash 2000). Robust nucleus of and most behavioral experiments relied on outcomes the archistriatum (RA) neurons were recorded in anes- rather than microscopic analysis during development. thetized male and female zebra finches. Experimental The zebra finch is a good model system because adults birds were raised with siblings by their parents and depend on auditory feedback for song maintenance isolated at about 35 days of age; for each bird the data (Nordeen and Nordeen 1992). This was an important were collected during one, acute recording session after observation that severed the link suggested by prior administration of urethane. Neurons were presented experiments that only species that modify songs as with the bird’s own (plastic) songs and conspecific age- adults depend as adults on auditory feedback to main- matched plastic songs starting at around 30 days of age. tain song (e.g., Konishi 1965; Marler and Waser 1977). Testing with the bird’s own song at such an early age Nevertheless, the mechanisms for maintenance may had not been conducted in prior studies. Neurons were differ from those for development because in the former also tested with tutor, familiar, and unfamiliar adult the representations for song are already established. songs. Remarkably, RA neurons in male birds expressed a preference for the class of plastic song stimuli over adult song stimuli shortly after the age at which RA Error-driven and reinforcement learning neurons begin to express strong auditory responses, at about 38 days of age. At this age, RA neurons do not The template theory emphasizes the instructional role of distinguish between the bird’s own song and age-mat- the environment. In contrast, the notion that a bird ched conspecific songs. It cannot be ruled out that some constructs his song by choosing from a pre-ordained set unidentified feature of the bird’s own song would have of vocal primitives emphasizes selectional learning. Two more strongly stimulated the neurons (i.e., that there models of motor learning that highlight the differences was a ‘‘hidden’’ preference for the bird’s own song that between instructional and selectional learning are what was not detected), but an analysis of the variance in the was termed ‘‘error-driven’’ learning (Troyer and Bottjer response patterns does not support such a hypothesis. 2001) and reinforcement learning (Sutton and Barto The RA data are most parsimoniously explained in 1998). In error-driven learning, the errors represent the terms of a selectional model where auditory responses fine details of the mismatch in features between the ac- are first tuned to the class of plastic song stimuli and tual vocal output and the target vocalization (template). then sharpened by interaction with vocal feedback. An If, as in vocal learning, there are many features then the instructional model interpretation would require RA error signal is of high dimensionality. In error-driven neurons to acquire selective responses for plastic songs learning, the error signal defines a specific trajectory as compared with adult songs. This selectivity would be through the parameter space towards the desired vo- the result of learning to sing the bird’s own plastic song, calization. That is, the error signal instructs the system without endowing RA neurons with selectivity for the how to proceed. A fundamental issue with error-driven bird’s own song compared to age-matched plastic songs. learning is the delay between when a command is formed While it cannot be ruled out without further experi- and when the error associated with that command is ments, this is a complex model. It should be possible to received. This delay may be of the scale of, or in cases of 858 rapidly evolving behavior even longer than, the duration to these selective responses have yet to be determined, of units of behavior. If so, it is a difficult problem to however, so what features are represented in these HVc apply the high-dimensional error signal to modify the neurons in swamp sparrows is still not established. The command circuitry if that signal is received during pre-motor organization of the zebra finch HVc (Yu and control of subsequent behavior. This ‘‘temporal credit Margoliash 1996), and the representation of syllables assignment’’ problem has long plagued learning mech- and sequences of syllables in the auditory responses of anisms (Sutton 1984) and is particularly acute for vocal HVc neurons in white-crowned sparrows (Margoliash learning such as in birdsong (Troyer et al. 1996). 1983) and zebra finches (Margoliash and Fortune 1992), In contrast, in reinforcement learning, a global signal is also supports a feature-level representation in HVc. computed that biases future performance towards the Thus, the existence of a topographic representation in desired goal. The reinforcement or reward signal contains the AFP is difficult to unambiguously interpret in terms little information about the specific nature of the errors of the competing theories of the learning mechanism. (for example, it does not represent features) and is a signal There are significant theoretical differences between of low dimensionality. It does not define a trajectory error-driven and reinforcement learning, but these dif- through the parameter space, but the reward is maximized ferences may be difficult to distinguish in the brain. The when the bird has minimized the differences between the computations necessary to implement error-driven and actual and desired vocalization. The error signal does not reinforcement learning are very different and should be instruct the system how to proceed but provides a basis for reflected in different neural implementations. In practice, selecting among the natural variation that is present in the both types of learning may contribute to vocal learning, behavior. For reinforcement learning, the temporal for example, a reinforcement phase early during senso- problem is not as challenging because the low dimensio- rimotor development followed by error-driven learning. nality of the representation simplifies the problem of Differentiating between error-driven and reinforcement predicting future rewards from current actions. For bio- learning, and instructional and selectional models, is logical reinforcement learning, an important issue is what likely to be very hard, and making these distinctions may are the units of behavior that are being selected, as well as become less valuable as information about the actual the cellular implementation of the learning mechanism neural mechanisms of song learning emerges. (see Knudsen 1994; Pennartz 1996). Reinforcement learning does not preclude having high-dimensional neural representation of signals. For Behavioral state and modulation of auditory input example, the topographic organization of the AFP is to song system consistent with an error-driven learning model (Troyer and Bottjer 2001) but is also consistent with a rein- Understanding how auditory activity is processed in the forcement learning model. The AFP is thought to pro- song system is essential for a mechanistic understanding vide a correction signal that guides vocal development of song learning. It has been known for many years that and adult song maintenance (Bottjer et al. 1984; Wil- HVc neurons are selective for the bird’s own song as liams and Mehta 1999; Brainard and Doupe 2000b). compared to conspecific songs (McCasland and Konishi This pathway is topographically organized based on the 1981; Margoliash and Konishi 1985). This selectivity muscles of the syrinx starting from very early develop- arises in part from neuronal sensitivity to combinations ment (Johnson et al. 1995; Vates and Nottebohm 1995; of spectral lines and to temporal sequences of sounds Iyengar et al 1999). A myotopic organization lends itself (Margoliash 1983; Margoliash and Fortune 1992), and to the representation of fine details of vocal behavior, as HVc neurons are particularly sensitive to the overall is required for error-driven learning. In error-driven temporal structure of the bird’s own song (Margoliash learning, however, features are what are modified, and 1986; Theunissen and Doupe 1998). HVc is, in addition, features may be distributed within a topographic rep- an important site of sensorimotor integration in the song resentation. Furthermore, the inputs to the AFP are not system, projecting to RA and area X. Both HVc target organized topographically. Individual HVc fibers ramify nuclei exhibit a topographic organization, yet topo- and synapse broadly throughout area X, the first nucleus graphic organization in HVc has yet to be reported. within the AFP (Fortune and Margoliash 1995; Foster Until recently, there was little insight into HVc cir- and Bottjer 1998). The catecholaminergic input to area cuitry beyond this descriptive level of analysis. Intra- X is also presumed to be broadly distributed (Lewis et al. cellular recordings have provided the first detailed 1981). These inputs may carry information about fea- circuit-level analysis of HVc (Mooney 2000). The tures and rewards which are then mapped onto the subthreshold activity of RA-projecting HVc neurons myotopic representation in area X. The recent obser- (HVc-RAn) showed some song selectivity, whereas the vation in swamp sparrows that individual area X-pro- area X-projecting HVc neurons (HVc-Xn) had highly jecting HVc neurons respond selectively to only one song selective subthreshold potentials. A hierarchical song (comprising one syllable type) of the multiple songs relationship was proposed between HVc-RAn, inter- in a bird’s repertoire emphasizes that the AFP may re- neurons, and HVc-Xn. It would be valuable to test this ceive HVc input organized at the feature level (Mooney model with local circuit blockade, and valuable to et al. 2001). The specific acoustic features that give rise determine the functional contributions of the various 859 HVc afferents to the subthreshold properties that were Modulation of HVc auditory activity identified (Rosen and Mooney 2000b). In addition, dorsal ventricular ridge, of which HVc is a part, is characterized A recent study provides direct evidence for differential by clusters of cells (Ulinski 1983). The circuit-level anal- state-dependent modulation of auditory responses of ysis of HVc potentially provides a unique opportunity, to different classes of HVc interneurons. HVc neurons were describe the functional interaction of different HVc cells first classified by their extracellular waveforms and their within a cluster, and the functional interaction of clus- projection target; the projection targets were determined ters. This holds the potential for describing a functional by antidromic stimulation/collision experiments (Shea unit of analysis characteristic of the reptile-avian fore- et al. 2001). Those experiments could distinguish wave- brain. It could also permit assessment of whether HVc form characteristics of HVc projection neurons from two has a truly distributed representation or whether HVc distinct classes of interneurons. In chronic recordings, exhibits a spatially ordered organization based on a unit single HVc neurons were recorded in awake birds, and of organization (e.g., the cell cluster) that is too small to later the same neurons were recorded while birds were have been resolved with the anatomical techniques used sleeping. Neurons were classified based on their spike to date (Margoliash et al. 1994b). Finally, HVc multiunit waveforms as determined by the electrical stimulation auditory responses have been shown to be modulated by experiments. The data set is small because holding single behavioral state (Schmidt and Konishi 1998; Nick and cells for extended periods in behaving animals is difficult. Konishi 2001). It would be useful to extend these ob- Nevertheless, it was concluded that one class of inter- servations to single cell recordings (Rauske and Mar- neurons lost auditory responses when birds awakened, goliash 1999) to understand how HVc circuitry is and the other retained auditory responsiveness in awake modified by behavioral state. Recent observations sug- birds (Rauske et al. 2002). The loss of auditory activity gest that state-dependent modification in HVc is medi- upon wakening runs counter to the dogma that sleep in- ated by adrenergic (Dave et al. 1998b) and cholinergic volves increases in sensory thresholds, but is consistent (Shea and Margoliash 1999) agents, although to date with observations in RA as well (Dave et al. 1998b). pharmacological specificity and action of the intrinsic Different experiments have reported different results system has only been demonstrated for cholinergic concerning the prevalence of auditory activity in awake activity. These observations should be extended to male birds. This has been interpreted in terms of the strong identify the target cells and forms of action of neuro- state-dependence of auditory responses in the song sys- modulatory input to HVc, and they do not rule out yet tem, which is well established for male zebra finches, and other sources of influence on the song system. the lack of control of behavioral state in prior studies. Circuit-level analysis should be sensitive to cell types Other explanations may also obtain, however. For ex- and sub-types. A trend in recent articles is to adopt a ample, there is some controversy regarding modulation of shorthand referring to three types of HVc neurons auditory activity in HVc. Schmidt and Konishi (1998) (HVc-RAn, interneurons, and HVc-Xn). This may reported dramatic loss of auditory activity in male zebra generate confusion. In fact there are three categories of finch HVc multiunits comparing awake and anesthetized neurons and multiple morphological types within each birds. Similar results were observed in single RA neurons category (Nixdorf et al. 1989; Fortune and Margoliash (Dave et al. 1998b) and HVc multiunit recordings (Nick 1995). Recent physiological studies of HVc have em- and Konishi 2001) comparing the awake and sleeping phasized three cell classes that correspond to the cate- states, confirming the state-dependence of HVc auditory gories of HVc neurons (Dutar et al. 1998; Mooney responses in unanesthetized male zebra finches. The 2000). If variation within cell categories exists that has Schmidt and Konishi (1998) results, however, were at yet to be described, this may unintentionally highlight odds with prior studies (e.g., McCasland and Konishi the disparity between few, seemingly simple physiologi- 1981) that had described HVc auditory responses in cal classes of cells and many, seemingly complex ana- awake animals. One explanation that was offered was tomical classes of cells. Such complexity, however, state-dependent differences, on the assumption that the cannot be ignored in considering the multiple roles of prior studies did not control for state-dependency of au- HVc and its efferent targets. For example, considerable ditory activity. However, HVc auditory responses were evidence has accumulated that the AFP has an auditory reported by McCasland and Konishi (1981) using chronic component that may provide an error, reinforcement, or recording techniques in two male canaries and one male other type of signal involved in learning. The AFP also white-crowned sparrow, not two zebra finches as was re- carries premotor signals (reviewed in Solis et al. 2000). ported by Schmidt and Konishi (1998). Vigorous auditory Knowing whether these classes of signals may be carried responses were also consistently observed in male white- in separate classes of neurons is important for develop- crowned sparrows studied extensively over periods of up ing realistic theories of the action of the AFP. Similarly, to approximately 4 months in an awake-restrained para- the functional distinction of two morphologically dis- digm (Margoliash 1986), results which Schmidt and tinct classes of HVc-RAn is unknown. Multiple classes Konishi (1998) did not consider. Thus, in part these dif- of HVc interneurons have been described; these could be ferent results may simply reflect species differences. important for the complex behavioral state-dependent One study of HVc auditory neurons in awake white- processing of HVc auditory input. crowned sparrows (Margoliash 1986) recorded from 860 awake-restrained birds, which induced stress, whereas in awake birds are much less prevalent than in single unit McCasland and Konishi (1981) assessed auditory re- recordings. For these many reasons, a focus on single cell sponses in tethered birds with a technique not dissimilar recordings will continue to be necessary to make progress from that used by Schmidt and Konishi (1998). There in understanding HVc processing. does not seem to be an obvious basis for concern re- garding stress in the McCasland and Konishi (1981) experiments; for example, the birds sang of their own Modulation of RA auditory activity volition. It remains possible, therefore, that two pro- cesses are involved here. The first, as suggested by Sch- A somewhat analogous controversy concerns auditory midt and Konishi (1998) is that stress induces expression responses in RA. Some workers reported RA auditory of auditory activity in awake animals that is otherwise responses in awake birds (Vicario and Yohay 1993), suppressed. This has plausible behavioral interpreta- whereas others only observed RA auditory responses in tions. Young zebra finches, for example, copy the songs rare cases, especially when birds were restrained (Dave of the most aggressive males (Clayton 1987). Adult in- et al. 1998b; Dave and Margoliash 2000). It has been digo buntings may modify their songs following ag- difficult to resolve what methodological differences give gressive social interactions (Payne 1981). Territorial rise to the different results. Vicario and Yohay (1993) adult male white-throated sparrows learn to recognize did not report directly observing their animals during the songs of their neighbors; typically this is preceded by physiological recordings in sound isolation chambers. If a period of antagonistic encounters as territories are the animals achieved a sleep-like state during those established (Brooks and Falls 1975). times, it could help to explain the different results, but The second possible process is species differences. It is this is a conjecture. It is clear that future experiments noteworthy that McCasland and Konishi (1981) found should pay attention to behavioral state and attempt to auditory responses in awake canaries, which modify their adopt consistent methodological procedures. songs as adults based on auditory feedback. There are also State-dependency of song system auditory responses related differences with regard to the effects of HVc lesions have been described as a ‘‘gating’’ phenomenon. Gating on song perception. HVc lesions abolish the preference of implies an all-or-nothing effect. Alternatively, the song female canaries for conspecific songs or song phrases system may build an inverse or negative image to cancel (Brenowitz 1991; Del Negro et al. 1998) – post-lesion birds auditory feedback. This would be analogous to the sen- displayed promiscuously to heterospecific songs. Male sory cancellation mechanism that has been described in and female starlings in a song categorization operant task electric fish (Bell et al. 1997). The advantages of such an learned to make new associations more slowly if they re- organization is that it is explicitly dynamic, and generates ceived HVc lesions, and the reduction in learning rate was a signal proportional to the difference between the ex- related to the amount of HVc lesioned (Gentner et al. pected and perceived feedback. For such mechanisms to 2000). In contrast, female zebra finches continued to dis- be meaningful in birds, auditory activity would have to be play preferentially to conspecific songs after HVc lesions, available to the song system in the awake animal. This but lost the preference after cHV lesions (MacDougall- might occur in awake animals listening to song, or more Shackleton et al. 1998). These data suggest that recordings likely in animals engaged in singing, and might be most in a range of species may uncover differences in the apparent in young animals that have yet to establish prevalence of auditory responses in HVc that correspond crystallized songs. The data on this central point are to differences in the role that HVc plays in song perception fragmentary. Recordings from awake but quiescent birds in those species. If strong species differences are con- as young as 52 days of age failed to identify any significant firmed, this would stand in contrast to the apparently differences from the pattern of strong daytime suppression universal requirement for auditory feedback during song of RA auditory activity that had been observed in adult development, and apparently universal expression of a birds, yet confirmed auditory responses in young sleeping state-dependent auditory representation of own song in birds qualitatively similar to what had been observed in the song system of male birds. adults (Dave et al. 1998b). In intact adult birds, during A more nuanced argument revolves around HVc au- singing RA neuronal discharges are dominated by pre- ditory responses in zebra finches. Single cell auditory re- motor patterns of activity, in that a burst pattern changes sponses in HVc have been reported in awake zebra finches depending on the note that follows the burst (Dave and (Yu and Margoliash 1996; Rauske et al. 2002). One sig- Margoliash 2000). This observation has not been con- nificant methodological difference comparing studies is firmed in juvenile birds. It is also the case that relative to that Schmidt and Konishi (1998) relied on multiunit re- the number of different burst patterns an RA neuron can cordings. As described above, different classes of single express, there are relatively few cases of variable temporal HVc units are differentially modulated by behavioral sequences of syllables (e.g., A-B, A-C) in zebra finch state. It has been our experience that the HVc neurons that songs. Direct confirmation of the pre-motor nature of RA exhibit auditory responses during the day are common but activity in adult birds has been limited to these cases, are particularly difficult to maintain for the long record- although an exception to the rule has yet to be observed. ings necessary to directly compare sleep/wake properties, One model is that auditory feedback reaches HVc and that auditory responses in multiunit HVc recordings and RA in young birds as they sing. As a bird develops 861 his song, he establishes a better prediction of the ex- wakefulness has recently come under scrutiny in the pected auditory feedback, which releases RA neurons birdsong field, it is helpful to review the criteria for de- (and presumably RA-projecting HVc neurons) from fining sleep. Sleep is defined by three criteria: specific modulation via auditory feedback. The auditory re- postures, increased sensory thresholds, and specific sponses observed in RA of sleeping adult zebra finches electrophysiological properties (Hobson 1987). None of are consistent with this hypothesis. These neurons ex- the recent studies in songbirds relied on all three criteria. hibit the spike-to-spike mapping between auditory ac- Depending on the specific goals of a study, this limita- tivity and pre-motor output necessary in a ‘‘negative tion may not be critical. image’’ model. The mapping between (following) audi- Following up on prior multiunit and single unit tory activity and (leading) pre-motor activity is achieved physiological studies of RA during sleep (Dave et al. in RA neurons by regulating the burst patterns of the 1998b; Dave and Margoliash 2000), Nick and Konishi auditory responses based on the sequence of preceding (2001) studied the physiology of HVc using multiunit syllables (Dave and Margoliash 2000). This is a form of recordings during sleep in the context of simultaneous a prediction, which is presumably learned as birds de- EEG recordings. Those experiments confirmed in HVc velop their songs and establish predictable statistics of what had been demonstrated in the previous RA studies singing. The question as to why in adult birds RA au- with multiunit and single unit recordings. Nick and ditory responses are observed only in sleeping animals Konishi (2001) asserted that the prior studies ‘‘did not might be mechanistically tied to the features during sleep use an objective criterion for sleep’’ which is important if necessary for the replay phenomenon described below. physiological responses are not ‘‘directly related to In awake animals, RA neurons only burst when the sleep/wake states but to other variables such as the cir- animal sings. For RA neurons to replay during sleep cadian clock.’’ The argument fails to consider the ob- their activity during singing, the neurons have to burst. servation by Dave et al. (1998b) that during brief periods In sleeping birds, RA neurons respond to song playback. during the day when birds became quiescent and slept, Thus, RA neurons can burst as in singing and be in a RA exhibited physiological properties (e.g., low spon- state where they are sensitive to auditory input. This taneous rates, auditory responses) otherwise only ob- seems to be a strong hint that auditory activity could served during sleep at night. This is direct experimental reach RA neurons in singing birds under some condi- data to refute the subsequent hypothetical raised by tions. This model does not ‘‘beg the question’’ (Nick and Nick and Konishi (2001). Furthermore, the suite of Konishi 2001) as to the existence of auditory responses electrophysiological changes observed in RA at the in sleeping birds but makes meaningful and testable single unit and multiunit level was remarkably consis- predictions. It will be particularly valuable to assess how tent, with properties such as regular discharge and sen- RA neurons respond to changes in auditory feedback sory refractoriness (while awake) or bursting and and other challenges. auditory responsiveness (during sleep) demonstrable on The temporal correspondence between activity in a moment-by-moment basis. That is, the objective, state- sleeping and waking animals has been observed in RA but dependent RA electrophysiological properties are highly has yet to be reported in HVc. The initial HVc single-cell reliable indicators of sleep. More generally, electro- recordings from singing birds observed a hierarchical ar- physiological measures do not obviate behavioral as- rangement of activity patterns comparing HVc and RA sessment of the sleeping state any more than they recordings, including tonic and syllable-based firing pat- obviate behavioral assessment of the waking state. The terns (HVc) and phasic and note-based firing patterns earlier studies (Dave et al. 1998b; Dave and Margoliash (RA) (Yu and Margoliash 1996). The recent observations 2000) used numerical (‘‘objective’’) criteria for sound that the chronic recording technique that was used amplitude to assess cage noises associated with move- strongly biased recordings towards interneurons (Shea ments, but also behavioral observations (eye closure, et al. 2001) raises the question as to whether aspects of the posture, breathing rate, etc.). Sleep is associated with transformation from the HVc-type of firing pattern to the specific postures and movements, and may be released RA-type of firing pattern might not occur in the RA- by sign stimuli, which are the proper subject of etho- projecting HVc neurons (cf. Yu and Margoliash 1996). logical analysis. Postures are one of the defining criteria This raises the further possibility, that HVc-RAn also of the sleeping state; this criteria should not be aban- exhibit the sensorimotor mapping reported for RA neu- doned. rons (Dave and Margoliash 2000).

Spontaneous activity during sleep: what is replayed? The possible roles of sleep RA neurons in sleeping birds have recently been shown How do we define sleep? to exhibit patterns of activity during sleep that suggest that birds spontaneously replay fragments of songs A central question regarding state-dependency of audi- while they sleep (Dave and Margoliash 2000). The full tory activity in the song system is its non-paradigmatic extent of what is replayed, however, has yet to be well expression across behavioral state. Since sleep and established. During undisturbed sleep, RA neurons 862 exhibit two patterns of activity. One pattern is a more It remains unclear how to interpret non-matching variable version of the regular, oscillatory ongoing ac- bursts in the midst of a sequence of matching bursts. tivity observed in awake animals. RA neurons also One plausible model is that both the matching bursts burst, a pattern which is observed in intact awake ani- and the sequences of matching bursts have behavioral mals only when they sing. The two patterns of RA significance. This is consistent with the hierarchical or- activity in sleeping animals results in a bimodal peak in ganization of pre-motor properties observed during the distribution of inter-spike intervals (ISIs), with the singing (McCasland and Konishi 1981), where notes are short-ISI peak related to the bursts. To analyze the represented in RA, syllables are represented in HVc, and patterns of RA activity in sleeping animals, the temporal larger sequences of sounds are represented in Uva structure of spontaneous bursts were matched to burst (Williams and Vicario 1993; Vu et al. 1994; Yu and patterns the same neurons had produced earlier during Margoliash 1996). Under the proposed model, during the diurnal period when animals had sung. Bootstrap sleep, sequences of bursts would be replayed at higher and across-neuron analyses were used to establish the levels of the hierarchy (e.g., Uva, NIf, or HVc) and in- statistical significance of those matches. Only RA dividual bursts would be replayed in RA. How closely spontaneous activity during sleep that qualified as a coupled the two patterns of activities (burst sequences burst was considered for analysis, where a burst was and individual bursts) are could be investigated by defined as (1) a sequence of ISIs all of which fell outside comparing ‘‘errors’’ in sequences with ‘‘errors’’ in indi- the normal (non-bursting) distribution of ISIs, and (2) as vidual bursts. For example, when a burst in RA (rep- having sufficient number of spikes for within-burst sta- resenting one note of a syllable) is misrepresented in a tistical analysis (eight spikes). Longer intervals were sequence, are all the bursts in the syllable also misrep- removed from the beginning and ends of bursts, and resented? Are the patterns of replay in HVc dependent bursts with intervals greater than the duration between on the syllable that is being represented, independent of syllable onsets were split. which (if any) notes within the syllable are misrepre- Comparison of spontaneous neural activity to activ- sented in RA? Are non-veridical sequences of syllables ity recorded at other times in the context of observable replayed, with the pattern of notes of each syllable behavior is statistically challenging, and criteria for faithfully replayed in RA? Is there temporal coincidence which analysis techniques apply and how to assess sta- across neurons in the timing of errors within a sequence tistical significance is under active investigation (e.g., akin to the temporal coincidence in veridical replay Nadasdy et al. 1999). For example, bootstrap proce- across neurons that was preliminarily reported (Dave dures that shuffle spikes between bursts destroy the and Margoliash 2000)? higher-order statistics of the original signal. This may It also remains unclear whether the apparent ‘‘errors’’ demonstrate statistical significance of the spike timing or misrepresentations in burst patterns do not in fact structure in original data as compared to the shuffled represent accurate information about variation in the data but the hypothesis being tested is very weak. Thus, daytime singing patterns. Perhaps variation regarding a conservative posture was adopted by Dave and Mar- singing behavior that is stored during the day is ampli- goliash (2000); only individual spontaneous bursts were fied in replay during sleep to provide a larger error sig- compared to pre-motor activity, and only spikes within nal. Understanding if there is information coded in these individual bursts were shuffled. This not only simplified non-veridical burst patterns, and what that information the analysis but also its interpretation, yet still estab- is, will be a central feature of proving or disproving any lished the original result. This conservative approach theory of behavioral significance of song replay during may, however, potentially exclude many bursts from sleep. Finally, it remains unclear whether individual consideration (bursts with fewer than eight spikes or spontaneous bursts in RA are as well-regulated in their bursts that contained long as well as short ISIs), and timing as are the corresponding pre-motor bursts. may label as non-matching some bursts that might Clearly, there is still much to learn about spontane- match under other, less-stringent but still statistically ous activity of RA neurons in adult zebra finches. It is a valid criteria. Perhaps most importantly, Dave and premature and unsubstantiated conclusion that the RA Margoliash (2000) did not examine sequences of bursts. ‘‘information that is putatively ‘replayed’ is not a new Thus, the statistical analysis they applied could not have memory but rather the bird’s crystallized, or stable, detected the case where one or more individual sponta- adult song’’ (Nick and Konishi 2001). Replay in adults neous bursts could fail to significantly match any has been demonstrated relative to temporal spike pat- premotor burst, while a sequence of bursts including terns of the stable adult song for technical reasons, but non-matching bursts would exhibit statistically signifi- this finding does not preclude the possibility that replay cant matches to a corresponding sequence of pre-motor of new memories occurs, even in adults. Zebra finches bursts. Indeed, in the data set collected, visual examin- require auditory feedback to maintain adult song ation of longer stretches of spontaneous activity resulted (Nordeen and Nordeen 1992). Adult song maintenance in compelling examples of matches that nevertheless is a dynamic process (Leonardo and Konishi 1999). included abnormal patterns of bursting never observed Whether sleep is involved in this process has yet to be in the singing animal (Fig. 4 in Dave and Margoliash established, but the multiple weeks it takes for adult 2000). birds to exhibit deterioration of song after deafening is 863 consistent with a slow, potentially off-line process of forward prediction observed for RA neurons, and sen- assessing feedback in the adult. sorimotor match (Dave and Margoliash 2000), and de- tails of HVc circuitry (Mooney 2000) are not predicted by this model. Nevertheless, the Troyer and Doupe Conclusions – models of birdsong learning (2000a, 2000b) effort is the most comprehensive model of birdsong learning yet proposed, including not only a Recent electrophysiological studies of bird song have conceptual-level model but a comprehensive quantita- begun to emphasize models that differ from the template tive model. It is a significant attempt to explain online model. Several models based on reinforcement learning learning and incorporates much of what is currently have been proposed. The primary conceptual differences known at a systems level about the birdsong system. are the emphasis on online (Doya and Sejnowski 1995; It provides a framework in which new facts can be Troyer and Doupe 2000a, 2000b) and offline (Dave and considered. Such modeling is likely to be increasingly Margoliash 2000; Margoliash 2001b) modes of learning. important as this field matures. The offline model proposes a simple solution to the problem of timing delay between sensory feedback and Acknowledgements I thank Amish S. Dave and Timothy Q. Gentner motor output (temporal credit assignment). For offline for useful discussions of an earlier version the manuscript, and Eliot A. Brenowitz and anonymous reviewers for valuable critiques. All learning this problem is solved by hypothesizing for- the experiments from the Margoliash laboratory that were reviewed mation of a memory during singing and access of that comply with the ‘‘Principles of animal care’’ publication No. 86-23, memory later in such a way as to bring signals into revised 1985 of the National Institute of Health and the laws of the temporal register. In the specific model proposed, the United States of America. Supported by grants Nos. 59831 and central assumption in the functional circuitry is that the 60276 from the National Institute of Mental Health. nucleus DLM exhibits a synaptic delay that changes depending on behavioral state. A long synaptic delay for the medial subdivision of the dorsal lateral nucleus of References anterior thalamus (DLM) is known (Luo and Perkel 1999). State-dependent changes in the synaptic delays Adret P (1993) Operant conditioning, song learning and imprinting to taped song in the zebra finch. Anim Behav 46:149–159 are not known for DLM, but DLM shares properties Adret P, Margoliash D (2000) Early development of auditory se- with mammalian thalamocortical cells (Luo and Perkel lectivity in the song control nucleus robustus archistriatalis. Soc 1999; Perkel and Farries 2000) that exhibit state- Neurosci Abstr 26:2031 dependent properties. The Dave and Margoliash (2000) Anderson SE, Dave AS, Margoliash D (1996) Template-based automatic recognition of birdsong syllables from continuous model also relies on the RA to DLM projection (Wild recordings. J Acoust Soc Am 100:1209–1219 1993) as a means of conveying spontaneous activity of Baptista LF, Petrinovich L (1984) Social interaction, sensitive RA neurons (during sleep) back into the forebrain song phrases and the song template hypothesis in the white-crowned system. The RA-DLM projection has been little studied, sparrow. Anim Behav 32:172–181 and its functional significance has not been tested. Off- Baptista LF, Petrinovich L (1986) Song development in the white- crowned sparrow: social factors and sex differences. Anim Be- line models of birdsong learning posit that motor pro- hav 34:1359–1371 gramming should change during sleep, and indeed Basham ME, Nordeen EJ, Nordeen KW (1996) Blockade of recently it has been reported that RA burst patterns NMDA receptors in the anterior forebrain impairs sensory change after the bird experiences sleep (Rauske et al. acquisition in the zebra finch (Poephila guttata). Neurobiol Learn Mem 66:295–304 2001). It remains to be seen, however, whether these Beecher MD (1996) Birdsong learning in the laboratory and field. changes reflect activity of specific song system nuclei In: Kroodsma DE, Miller EH (eds) Ecology and evolution of (e.g., lMAN) or general circadian mechanisms that re- acoustic communication in birds. Cornell University Press, side outside of the song system. This should be testable Ithaca, pp 61–78 Bell C, Bodznick D, Montgomery J, Bastian J (1997) The genera- by examining the effects of lMAN lesions on circadian tion and subtraction of sensory expectations within cerebellum- changes in RA activity patterns. like structures. Brain Behav Evol 50 [Suppl 1]:17–31 For online learning, the requirements are more Boco T, Margoliash D (2001) NIf is a major source of auditory and challenging, and Troyer and Doupe (2000a, 2000b) spontaneous drive to HVc. Soc Neurosci Abstr 27:381.2 propose as yet unsubstantiated properties of HVc neu- Bottjer SW, Miesner EA, Arnold AP (1984) Forebrain lesions disrupt development but not maintenance of song in passerine rons. Notably, these include a temporally asymmetric birds. Science 224:901–903 Hebbian association between HVc-RAn and HVc-Xn, Brainard MS, Doupe AJ (2000a) Alteration of auditory feedback and a prediction that HVc-Xn should carry signals causes both acute and lasting changes to Bengalese finch song. during singing comparable to the neurons’ auditory re- Soc Neurosci Abstr 26:269.266 Brainard MS, Doupe AJ (2000b) Interruption of a basal ganglia- sponses but projected forward in time by approximately forebrain circuit prevents plasticity of learned vocalizations. 60 ms. (Such signals would represent predictions that Nature 404:762–766 result from learning the statistics of singing.) The com- Brenowitz EA (1991) Altered perception of species-specific song by putations and mechanisms proposed by Troyer and female birds after lesions of a forebrain nucleus. Science 251:303–305 Doupe (2000a, 2000b) are plausible, yet some may be Brooks RJ, Falls JB (1975) Individual recognition by song in white- difficult to convincingly verify or falsify. Recent results throated sparrows. I. Discrimination of songs of neighbors and such as the behavioral state dependency of RA activity, strangers. Can J Zool 53:879–888 864

Clayton N (1987) Song tutor choice in zebra finches. Anim Behav ing the sensitive period for song learning. J Neurosci 19:6037– 35:714–721 6057 Clayton NS (1989) The effects of cross-fostering on selective song Janata P, Margoliash D (1999) Gradual emergence of song selec- learning in estrildid finches. Behaviour 109:163–175 tivity in sensorimotor structures of the male zebra finch song Dave AS (2001) Mechanisms of sensorimotor vocal integration. system. J Neurosci 19:5108–5118 Ph.D. Thesis, University of Chicago Johnson F, Sablan MM, Bottjer SW (1995) Topographic organi- Dave AS, Margoliash D (2000) Song replay during sleep and zation of a forebrain pathway involved with vocal learning in computational rules for sensorimotor vocal learning. Science zebra finches. J Comp Neurol 358:260–278 290:812–816 Karten H (1968) The ascending auditory pathway in the pigeon Dave A, Yu AC, Gilpin JJ, Margoliash D (1998a) Methods for (Columba livia). II. Telencephalic projections of the nucleus chronic neuronal ensemble recordings in singing birds. In: Ni- ovoidalis thalami. Brain Res 11:134–153 colelis MAL (ed) Methods for neuronal ensemble recordings. Kelley DB, Nottebohm F (1979) Projections of a telencephalic CRC Press, Boca Raton, pp 101–120 auditory nucleus-field L in the canary. J Comp Neurol 183:455– Dave A, Yu AC, Margoliash D (1998b) Behavioral state modula- 469 tion of auditory activity in a vocal motor system. Science Knudsen EI (1994) Supervised learning in the brain. J Neurosci 282:2250–2254 14:3985–3997 Del Negro C, Gahr M, Leboucher G, Kreutzer M (1998) The se- Kogan J, Margoliash D (1998) Automated recognition of bird song lectivity of sexual responses to song displays: effects of partial elements from continuous recordings using dynamic time chemical lesion of the HVC in female canaries. Behav Brain Res warping and hidden Markov models: a comparative study. 96:151–159 J Acoust Soc Am 103:2185–2196 Dooling RJ, Searcy MH (1980) Early perceptual selectivity in the Konishi M (1965) The role of auditory feedback in the control of swamp sparrow. Dev Psychobiol 13:499–506 vocalization in the white-crowned sparrow. Z Tierpsychol Doupe AJ (1997) Song- and order-selective neurons in the songbird 22:770–783 anterior forebrain and their emergence during vocal develop- Konishi M (1978) Auditory environment and vocal development in ment. J Neurosci 17:1147–1167 birds. In: Walk RD, Pick HLJ (eds) Perception and experience. Doupe AJ, Konishi M (1991) Song-selective auditory circuits in the Plenum Press, New York, pp 105–118 vocal control system of the zebra finch. Proc Natl Acad Sci Konishi M (1985) Birdsong: from behavior to neuron. Annu Rev USA 88:11339–11343 Neurosci 8:125–170 Doya K, Sejnowski TJ (1995) A novel reinforcement model of Leonardo A, Konishi M (1999) Decrystallization of adult birdsong birdsong vocalization learning. In: Tesauro G, Touretzky DS, by perturbation of auditory feedback. Nature 399:466–470 Leen TK (eds) Advances in neural information processing Lewis JW, Ryan SM, Arnold AP, Butcher LL (1981) Evidence systems. MIT Press, Cambridge, pp 101–108 for a catecholaminergic projection to area X in the zebra finch. Dutar P, Vu HM, Perkel DJ (1998) Multiple cell types distin- J Comp Neurol 196:347–354 guished by physiological, pharmacological, and anatomic Luo M, Perkel DJ (1999) A GABAergic, strongly inhibitory pro- properties in nucleus HVc of the adult zebra finch. J Neuro- jection to a thalamic nucleus in the zebra finch song system. physiol 80:1828–1838 J Neurosci 19:6700–6711 Emlen ST (1972) An experimental analysis of the parameters of MacDougall-Shackleton S, Hulse S, Ball G (1998) Neural bases of bird song eliciting species recognition. Behavior 41:130–171 song preferences in female zebra finches (Taeniopygia guttata). Fee MS, Leonardo A (2001) Miniature motorized microdrive and Neuroreport 9:3047–3052 commutator system for chronic neural recording in small ani- Margoliash D (1983) Acoustic parameters underlying the responses mals. J Neurosci Methods 112:83–94 of song-specific neurons in the white-crowned sparrow. Fortune ES, Margoliash D (1992) Cytoarchitectonic organization J Neurosci 3:1039–1057 and morphology of cells of the field L complex in male Margoliash D (1986) Preference for autogenous song by auditory zebra finches (Taenopygia guttata). J Comp Neurol 325: neurons in a song system nucleus of the white-crowned spar- 388–404 row. J Neurosci 6:1643–1661 Fortune ES, Margoliash D (1995) Parallel pathways and conver- Margoliash D (1987) Neural plasticity in birdsong learning. In: gence onto HVc and adjacent neostriatum of adult zebra finches Rauschecker JP, Marler P (eds) Imprinting and cortical plas- (Taeniopygia guttata). J Comp Neurol 360:413–441 ticity. Wiley, New York, pp 23–54 Foster EF, Bottjer SW (1998) Axonal connections of the high vocal Margoliash D (2001a) The song does not remain the same. Science center and surrounding cortical regions in juvenile and adult 291:2559–2561 male zebra finches. J Comp Neurol 397:118–138 Margoliash D (2001b) Do sleeping birds sing? Population coding Gentner TQ, Margoliash D (2001) Perception in songbirds: defin- and learning in the bird song system. Prog Brain Res 130:319– ing a role for the forebrain region cHV. Soc Neurosci Abstr 331 27:381.7 Margoliash D, Fortune ES (1992) Temporal and harmonic com- Gentner TQ, Margoliash D (2002) The neuroethology of vocal bination-sensitive neurons in the zebra finch’s HVc. J Neurosci communication: perception and cognition. In: Megela-Sim- 12:4309–4326 mons A, Popper AN, Fay RR (eds) Acoustic communication. Margoliash D, Konishi M (1985) Auditory representation of aut- Springer, Berlin Heidelberg New York, pp. 324–386 ogenous song in the song-system of white-crowned sparrows. Gentner TQ, Hulse SH, Bentley GE, Ball GF (2000) Individual Proc Natl Acad Sci USA 82:5997–6000 vocal recognition and the effect of partial lesions to HVc on Margoliash D, Staicer CA, Inoue SA (1991) Stereotyped and discrimination, learning, and categorization of conspecific song plastic song in adult indigo buntings, Passerina cyanea. Anim in adult songbirds. J Neurobiol 42:117–133 Behav 42:367–388 Hobson JA (1987) Sleep. In: Adelman G (ed) Encyclopedia of Margoliash D, Staicer CA, Inoue SA (1994a) The process of syl- neuroscience. Birkha¨user, Boston, pp 1097–1100 lable acquisition in adult indigo buntings, (Passerina cyanea). Hultsch H, Todt D (1989) Memorization and reproduction of Behavior 131:39–64 songs in nightingales (Luscinia megarhynchos): evidence for Margoliash D, Fortune ES, Sutter ML, Yu AC, Wren-Hardin BD, package formation. J Comp Physiol A 165:197–203 Dave A (1994b) Distributed representation in the song system Ito K, Mori K, Iwasaki S-I (1996) Application of dynamic pro- of oscines: evolutionary implications and functional conse- gramming matching to classification of budgerigar contact calls. quences. Brain Behav Evol 44:247–264 J Acoust Soc Am 100:3947–3956 Marler P (1970) A comparative approach to vocal learning: song Iyengar S, Viswanathan SS, Bottjer SW (1999) Development of development in white-crowned sparrows. J Comp Physiol Psy- topography within song control circuitry of zebra finches dur- chol 71:1–25 865

Marler P (1997) Three models of song learning: evidence from Perkel DJ, Farries MA (2000) Complementary ‘bottom-up’ and behavior. J Neurobiol 33:501–516 ‘top-down’ approaches to basal ganglia function. Curr Opin Marler P, Nelson DA (1993) Action-based learning: a new form of Neurobiol 10:725–731 developmental plasticity in bird song. Neth J Zool 43:91–103 Rauske PL, Margoliash D (1999) Does behavioral state modulate Marler P, Peters S (1981) Sparrows learn adult song and more from sensorimotor properties in HVc? Soc Neurosci Abstr 25:624 memory. Science 213:780–782 Rauske PL, Dave AS, Margoliash D (2001) Sleep in adult zebra Marler P, Peters S (1982a) Developmental overproduction and finches functionally rewires the song system nucleus RA. Soc selective attrition: new processes in the epigenesis of birdsong. Neurosci Abstr 27:381.383 Dev Psychobiol 15:369–378 Rauske PL, Shea SD, Margoliash D (in press) State and neuronal Marler P, Peters S (1982b) Structural changes in song ontogeny in class dependent reconfiguration in the avian song system. the swamp sparrow, Melospiza georgiana. Auk 99:446–458 J Neurophysiol Marler P, Pickert R (1984) Species-universal microstructure in the Rosen MJ, Mooney R (2000a) Intrinsic and extrinsic contributions learned song of the swamp sparrow (Melospiza georgiana). to auditory selectivity in a song nucleus critical for vocal plas- Anim Behav 32:673–689 ticity. J Neurosci 20:5437–5448 Marler P, Sherman V (1983) Song structure without auditory Rosen MJ, Mooney R (2000b) Local and extrinsic contributions to feedback: emendations of the auditory template hypothesis. song-selectivity in the zebra finch song nucleus HVc. Soc J Neurosci 3:517–531 Neurosci Abstr 26:758.712 Marler P, Sherman V (1985) Innate differences in singing behavior Saini KD, Leppelsack H-J (1977) Neuronal arrangement in the of sparrows reared in isolation from adult conspecific song. auditory field L of the neostriatum of the starling. Cell Tissue Anim Behav 33:57–71 Res 176:309–316 Marler P, Waser MS (1977) Role of auditory feedback in canary Schmidt MF, Konishi M (1998) Gating of auditory responses in the song development. J Comp Physiol Psychol 91:8–16 vocal control system of awake songbirds. Nat Neurosci 1:513– McCasland JS, Konishi M (1981) Interaction between auditory and 518 motor activities in an avian song control nucleus. Proc Natl Searcy WA, Marler P, Peters SS (1985) Songs of isolation-reared Acad Sci USA 78:7815–7819 sparrows function in communication but are significantly less Mooney R (2000) Different subthreshold mechanisms underlie effective than learned songs. Behav Ecol Sociobiol 17:223–230 song selectivity in identified HVc neurons of the zebra finch. Sen K, Theunissen FE, Doupe AJ (2001) Feature analysis of nat- J Neurosci 20:5420–5436 ural sounds in the songbird auditory forebrain. J Neurophysiol Mooney R, Hoese W, Nowicki S (2001) Auditory representation of 86:1445–1458 the vocal repertoire in a songbird with multiple song types. Proc Shea SD, Margoliash D (1999) Multiple neuromodulators may Natl Acad Sci USA 98:12778–12783 gate auditory responses in the song motor system. Soc Neurosci Nadasdy Z, Hirase H, Czurko A, Csicsvari J, Buzsaki G (1999) Abstr 25:249.17 Replay and time compression of recurring spike sequences in Shea SD, Rauske PL, Margoliash D (2001) Identification of HVc the hippocampus. J Neurosci 19:9497–9507 projection neurons in extracellular records by antidromic Nelson DA (1992a) Song overproduction and selective attrition stimulation. Soc Neurosci Abstr 27:381.386 during song development in the field sparrow (Spizella pusilla). Solis MM, Doupe AJ (1997) Anterior forebrain neurons develop Behav Ecol Sociobiol 30:415–424 selectivity by an intermediate stage of birdsong learning. Nelson DA (1992b) Song overproduction, song matching and se- J Neurosci 17:6447–6462 lective attrition during development. In: McGregor PK (ed) Solis MM, Doupe AJ (1999) Contributions of tutor and bird’s own Playback and studies of animal communication. Plenum Press, song experience to neural selectivity in the songbird anterior New York, pp 121–133 forebrain. J Neurosci 19:4559–4584 Nelson DA, Marler P (1993) Innate recognition of song in white- Solis MM, Brainard MS, Hessler NA, Doupe AJ (2000) Song se- crowned sparrows: a role in selective vocal learning? Anim lectivity and sensorimotor signals in vocal learning and pro- Behav 46:806–808 duction. Proc Natl Acad Sci USA 97:11836–11842 Nelson DA, Marler P (1994) Selection-based learning in bird song Sutton RS (1984) Temporal credit assignment in reinforcement development. Proc Natl Acad Sci USA 91:10498–10501 learning. PhD Thesis, University of Massachusetts Nelson DA, Khanna H, Marler P (2001) Learning by instruction or Sutton RS, Barto AG (1998) Reinforcement learning: an intro- selection: implications for patterns of geographic variation in duction. MIT Press, Cambridge, MA bird song. Behaviour 138:1137–1160 Tchernichovski O, Mitra PP (2002) Towards quantification of Nick TA, Konishi M (2001) Dynamic control of auditory activity vocal imitation in the zebra finch. J Comp Physiol A (in press) during sleep: correlation between song response and EEG. Proc DOI 10.1007/s00359-002-0352-4 Natl Acad Sci USA 98:14012–14016 Tchernichovski O, Lints T, Mitra PP, Nottebohm F (1999) Vocal Nixdorf BE, Davis SS, DeVoogd TJ (1989) Morphology of Golgi- imitation in zebra finches is inversely related to model abun- impregnated neurons in the hyperstriatum ventralis, pars cau- dance. Proc Natl Acad Sci USA 96:12901–12904 dalis in adult male and female canaries. J Comp Neurol Tchernichovski O, Nottebohm F, Ho CE, Pesaran B, Mitra PP 284:337–349 (2000) A procedure for an automated measurement of song Nordeen KW, Nordeen EJ (1992) Auditory feedback is necessary similarity. Anim Behav 59:1167–1176 for the maintenance of stereotyped song in adult zebra finches. Tchernichovski O, Mitra PP, Lints T, Nottebohm F (2001) Dy- Behav Neural Biol 57:58–66 namics of the vocal imitation process: how a zebra finch learns Nottebohm F (1991) Reassessing the mechanisms and origins of its song. Science 291:2564–2569 vocal learning in birds. Trends Neurosci 14:206–211 Theunissen FE, Doupe AJ (1998) Temporal and spectral sensitivity Nottebohm F, Kelley DB, Paton JA (1982) Connections of vocal of complex auditory neurons in the nucleus HVc of male zebra control nuclei in the canary telencephalon. J Comp Neurol finches. J Neurosci 18:3786–3802 207:344–357 Theunissen FE, Sen K, Doupe AJ (2000) Spectral-temporal re- Payne RB (1981) Song learning and social interaction in indigo ceptive fields of nonlinear auditory neurons obtained using buntings. Anim Behav 29:688–697 natural sounds. J Neurosci 20:2315–2331 Payne RB, Payne LL, Doehlert SM (1988) Biological and Thompson WL (1970) Song variation in a population of indigo cultural success of song memes in indigo buntings. Ecology buntings. Auk 87:58–71 69:104–117 Todt D, Hultsch H, Heike D (1979) Conditions affecting song Pennartz CMA (1996) The ascending neuromodulatory systems in learning in nightingales. Z Tierpsychol 51:23–35 learning by reinforcement: comparing computational conjec- Troyer TW, Bottjer SW (2001) Birdsong: models and mechanisms. tures with experimental findings. Brain Res Rev 21:219–245 Curr Opin Neurobiol 11:721–726 866

Troyer TW, Doupe AJ (2000a) An associational model of birdsong Volman S (1993) Development of neural selectivity for birdsong sensorimotor learning. I. Efference copy and the learning of during vocal learning. J Neurosci 13:4737–4747 song syllables. J Neurophysiol 84:1204–1223 Vu ET, Mazurek ME, Kuo Y-C (1994) Identification of a forebrain Troyer TW, Doupe AJ (2000b) An associational model of birdsong motor programming network for the learned song of zebra sensorimotor learning. II. Temporal hierarchies and the learn- finches. J Neurosci 14:6924–6934 ing of song sequence. J Neurophysiol 84:1224–1239 West MJ, King AP (1988) Female visual displays affect the devel- Troyer T, Doupe AJ, Miller KD (1996) An associational hypoth- opment of male song in the cowbird. Nature 334:244–246 esis for sensorimotor learning of birdsong. In: Bower JM (ed) Wild JM (1993) Descending projections of the songbird nucleus Computational neuroscience. Academic Press, San Diego, pp robustus archistriatalis. J Comp Neurol 338:225–241 409–414 Williams H, Mehta N (1999) Changes in adult zebra finch song Ulinski PS (1983) Dorsal ventricular ridge. Wiley, New York require a forebrain nucleus that is not necessary for song pro- Vates GE, Nottebohm F (1995) Feedback circuitry within a song- duction. J Neurobiol 39:14–28 learning pathway. Proc Natl Acad Sci USA 92:5139–5143 Williams H, Vicario DS (1993) Temporal patterning of song pro- Vates GE, Broome BM, Mello CV, Nottebohm F (1996) Auditory duction: participation of nucleus uvaeformis of the thalamus. pathways of caudal telencephalon and their relation to the song J Neurobiol 24:903–912 system of adult male zebra finches (Taenopygia guttata). J Comp Yu AC, Margoliash D (1996) Temporal hierarchical control of Neurol 366:613–642 singing in birds. Science 273:1871–1875 Vicario DS, Yohay KH (1993) Song-selective auditory input to a Zoloth SR, Petersen MR, Beecher MD, Green S, Marler P, Moody forebrain vocal control nucleus in the zebra finch. J Neurobiol DB, Stebbins W (1979) Species-specific perceptual processing of 24:488–505 vocal sounds by monkeys. Science 204:870–873