Downloaded by guest on September 30, 2021 in otnbt) orsl rmteatvt fnuosand neurons of activity simula- the through from or result analytically to either both), shown, (often be tions can experiments, in they observed and assemblies of properties known reflect merge, associate, ones: project, existing rocal modify the and/or assemblies are new assemblies create while 8), ref. e.g., Gy of extensively (see, theme studied brain central been animal have the called behavior in (sometimes dynamic assemblies their were and recently, they bles) imag- More calcium until 7). through (6, ago 5), decade ing ref. a than e.g., more identified (see, decades clearly the prediction Assembly-like during Hebb’s idea. researchers by or following sought concept the been with particular whose have coterminous a formations neurons is of of pattern thinking sets a O. subject in interconnected Donald firing densely by synchronized be ago loosely to decades (4) seven new. hypothesized Hebb not course, first of were is, assemblies They of idea The neurons. excitatory models 3). whole-brain and 2 the refs. of (e.g., science level cognitive the in individual useful and of synapses level Calculus and the Assembly between neurons the intermediate granularity, position and a the detail occupies system of this terms call In we culus. neurons; of assemblies on “logic”? sought the neural as of qualify realities the would the as of activity, abstracting kind logic and What embodying [this] neuroscience.” system, discerning of formal view direction activ- I future neural important action. of most transformation and the thought for into logic a ity put have recently not (1) do Axel answer: “we Richard it, an As development frameworks. to the conceptual necessitate closer new to of seems us and bring formidable, theoretical– is accelerating to gap and The seem and experimental not does both remarkable computational, neuroscience, The in language? progress and reasoning, H brain the in language assemblies neuronal with consistent is which cortex, results. experimental in recent language syn- of the implementing processing for tactic plausible assemblies a on propose based we architecture particular, reason- brain In as language. such and functions planning, cognitive ing, like human higher something underlie that may hypothesize it be We carrying can of computations. capable system arbitrary principle, out in computational be, resulting to assumptions, The under brain. shown, that and whole synapses and the neurons of spiking interme- of detail level of the level which between a diate brain occupying Calculus, the Assembly of the call model their we computational and a Assemblies constitute inhibition. and operations plasticity neu- Hebbian of populations with connected rons randomly generic, to by simulations, through realizable experi- and be in analytically opera- shown, observed be These assemblies can and of assemblies. ments, properties on to operations correspond of tions We repertoire Poggio) information. A. a Tomaso cognitive imprint and identify other to Friederici and D. believed Angela words, neurons by concepts, reviewed of memories, 2020; populations 6, February large review are for Assemblies (sent 2020 27, March Papadimitriou, H. Christos by Contributed a neurons Papadimitriou H. of Christos assemblies by computation Brain www.pnas.org/cgi/doi/10.1073/pnas.2001893117 and eateto optrSine oubaUiest,NwYr,N 10027; NY York, New University, Columbia Science, Computer of Department u aclsott sebiswt eti prtosthat operations certain with assemblies outfits calculus Our of assembly the is system our of object elementary basic The based brain the of model computational formal a propose We c nttt fTertclCmue cec,Ga nvriyo ehooy 00Ga,Austria Graz, 8010 Technology, of University Graz Science, Computer Theoretical of Institute wde h ri ee h id o omolecules, do How intelligence, mind? behavior, cognition, the effect beget synapses brain and cells, the does ow | sebyCalculus Assembly | ryBuzs orgy ¨ admgraph random a,1 n e tes hs operations These others. few a and ´ k’ eetbo (9). book recent aki’s ats .Vempala S. Santosh , | computation | sebyCal- Assembly rjc,recip- project, b ailMitropolsky Daniel , ensem- b olg fCmuig eri nttt fTcnlg,Alna A30332; GA Atlanta, Technology, of Institute Georgia Computing, of College apn oohrpisti sawl-tde rmwr usu- framework well-studied as a proba- to is small referred pairs—this same ally other the has to area happens an in neurons bility of pair ordered connections. of recurrent population random a with contains cortex area the Each of 1). (Fig. partition meaningful functionally number and finite denoted anatomically simulation a areas assume our We brain synapses. of of and plausi- neurons most yet of tractable, model as analytically ble, and well simplified a as employ results, results, mathematical Our programming Model high-level which code. machine in into way translated the are synapses— languages of and neurons reminiscent of fact world the a system to this down” of “compiled operations be high-level can the words, other In synapses. doi:10.1073/pnas.2001893117/-/DCSupplemental at online information supporting contains article This 1 BY-NC-ND) (CC 4.0 NoDerivatives License distributed under is article access open This model the Since neurons. of assemblies of manipulation latter. and the to former the from connection synaptic a is in neuron from say interconnections areas, synaptic of afferent pairs ordered certain for aadpsto:Orsmlto ytmi vial nie aigi osbeto possible it making online, available is results: system simulation our simulation extend and reproduce Our deposition: Data and interest.y Sciences; competing no Brain declare and authors The Cognitive Human Technology. for y of Institute Institute Massachusetts T.A.P., D.M., paper. Planck y the S.S.V., Max wrote C.H.P., A.D.F., D.M. and research; Reviewers: S.S.V., C.H.P., designed and research; W.M. performed W.M. and and S.S.V., M.C., C.H.P., contributions: Author * † owo orsodnemyb drse.Eal [email protected]. y Email: addressed. be may correspondence whom To nlss lgtymr ope eso ol aeparameters have would version parameter complex area numerical more basic slightly the A provides analysis. and convenience, mathematical, not otyspotorfaeokin framework of support—our uniformity mostly the from brain animal the u supinta l ra oti h aenme fnuosi n fexpository, of one is neurons of number same the contain areas all that assumption Our eea xeietlrsls(1 2 ugs eitoso h yatccnetvt of connectivity synaptic the of deviations suggest 12) (11, results experimental Several ytmb rpsn ri rhtcuefrsnatcpro- recent syntactic with for results. compatible language, experimental architecture of this production brain of the in reach a level cessing the proposing the demonstrate by at We system synapses. realizable and plausibly as neurons well be of as to cog- analytically shown, simulations, in be implicated through can be and to project, phenomena, appear nitive as which such merge, neurons, and of encom- associate, It assemblies Calculus. on Assembly operations the passes which gap: system this bridge computational to a phenom- promises introduce these we of between cognitive Here gap level of scales. formidable a two level the leaves language, the at as such and brain ena synapses, the and of neurons understanding expanding Our Significance eueti oe ofto,qatttvl,tecreation the quantitatively, fathom, to model this use We A. p fbigcnetdb yas,idpnetyo what of independently synapse, a by connected being of A a ihe Collins Michael , n vr ernin neuron every and Erd sRnigraph os–Renyi ˜ A, Discussion. B . y , a C G raieCmosAttribution-NonCommercial- Commons Creative n ofagMaass Wolfgang and , n,p t. neddt tn o an for stand to intended etc., , B http://github.com/dmitropolsky/assemblies edsushwsc eitosaffect— deviations such how discuss We . hr sachance a is there , . y ∗ yti,w enta each that mean we this, By https://www.pnas.org/lookup/suppl/ or (A, A NSLts Articles Latest PNAS G to B n n ,p hr r random are there ), B xiaoyneurons excitatory 1) naddition, In (10). hti,frevery for is, that ; n A and p † htthere that c k A | n o each for four of f9 of 1 . y

NEUROSCIENCE Fig. 1. A mathematical model of the brain. Our analytical results, as well as most of our simulations, use a formal model of the brain intended to capture cognitive phenomena in the association cortex. Our model encompasses a finite number of brain areas denoted A, B, ..., each containing n excitatory neurons. We assume that any pair of neurons in each area have a recurrent synaptic connection independently with probability p, and that, for certain pairs of areas (A, B), there are also afferent connections from any neuron in A to any in B, also independently and with probability p. We assume that firing of neurons proceeds in discrete steps, and, at each step, k of the neurons in each area fire: namely, those k neurons which have the largest sum of presynaptic inputs from neurons that fired in the previous step. Synaptic weights are modified in a Hebbian fashion, in that synapses that fired (i.e., the presynaptic neuron fired, and, in the next step, the postsynaptic neuron did) have their weight multiplied by (1 + β), where β is a plasticity constant. In our simulations, we typically use n = 107, k = 104, p = 0.001, and β = 0.1.

is probabilistic (by virtue of the random synaptic connectivity), a fixed number k of the n excitatory neurons in any area fire—in our analytical results postulating the effectiveness of the vari- particular, those neurons which receive the k highest excitatory ous operations must contain the clause “with high probability,” inputs. where the event space is implicitly the underlying random graph. The four basic parameters of our model are these: n (the We assume that all cells in an assembly x belong to the same number of excitatory neurons in an area, and the basic param- brain area, denoted area(x). eter of our model), p (the probability of recurrent and afferent Our model also encompasses simplified forms of plasticity synaptic connectivity), k (the maximum number of firing neu- and inhibition. We assume multiplicative Hebbian plasticity: If rons in any area), and the plasticity coefficient β. Typical val- at a time step neuron i fires and, at the next time step, neu- ues of these parameters in our simulations are n = 106−7, p = ron j fires, and there is a synapse from i to j , the weight of 10−3, k = 102−3, and β = 0.1. this synapse is multiplied by (1 + β), where β > 0 is the final Our model, as described so far, would result, through plastic- parameter of our model (along with n, p, and k). Larger values ity, in gigantic synaptic weights after a long time of operation. of the plasticity coefficient β result in the operations converg- We further assume that synaptic weights are renormalized, at ing faster, and render many of our proofs simpler. Finally, we a slower time scale, so that the sum of presynaptic weights at model inhibition and excitatory–inhibitory balance by postulat- each neuron stays relatively stable. This process of homeosta- ing that neurons fire in discrete time steps, and, at any time, only sis through renormalization is orthogonal to the phenomena we

2 of 9 | www.pnas.org/cgi/doi/10.1073/pnas.2001893117 Papadimitriou et al. Downloaded by guest on September 30, 2021 Downloaded by guest on September 30, 2021 area area in area which neurons from in many connectivity case the synaptic for once, 14, fired ref. of section A discussion the in dicted of firing repeated the of “copy” time a every as fire of henceforth thought be can that area By object. whole a cortex, bly inferotemporal as projection sub- the face the in this a by perhaps encoding in created population, neuronal face is a (MTL) familiar of lobe a temporal that example, medial 13) for ref. ject’s e.g., imprinting, (see, hypothesized assembly been an has It about? come tex Projection. Assembly assembly. an of firing repeated simple certain the of as the consequence events—such certain of almost an selection as the input synaptic and network randomness the the from of Its operation—emerges areas. assembly imple- of an as such pairs menting task, selected each for between apparatus are needed and functionality—the area tasks an various excitatory within between to connectivity neurons specific random beyond circuits place, neural in already that assumed not is and arguments our with simulations. minimally interferes it and here, describe expect area from strengthened neurons already from has which connections plasticity, the is this for reason extra of firing of framework, our acltdi u ahmtclmdlt eroughly If be parameters. {y of to range model broad mathematical a our in calculated aaiiro tal. et Papadimitriou exclusively powered are Primitive. here, Cap described as and operations, assembly Projection Random The higher be plasticity. should of synapses because these average of than weights the addition, of In dom. connectivity recurrent synaptic of the union the from assembly input explanation: synaptic of an elements provides paragraphs the previous Since computa- projection the through difficult in assembly this outlined an as with of seem creation may deal The assembly brain sci- problem? an tional computer the of in can existence problem How very difficult the surprising. known thus and a (20), is size ence certain graph a sparse of surprising, subgraph a rather dense in is a finding this of computation, problem of the 2C). because view (Fig. of hypothesis point this not— the do verify larger From they simulations significantly when and than analysis is assembly our same and connection the to synaptic belong they a that when have chance the neurons is, two intraconnected—that densely are assemblies Assemblies. of Intraconnection Dense write we the place, Once taken has 2B4. analysis, Fig. improved as in (18, well presented as simulations are model, detailed our in through results and simulation 19); 17) (16, analytically both project(x fast, exponentially assembly plastic- converge an enough to create projection. high to proved for probability, be high and can with parameters process of this range ity, large a For o stenwassembly new the is How it that sense the in generic is model our that emphasize We ednt h rjcinpoesdsrbdaoeas above described process projection the denote We t steofcoybl and bulb olfactory the is } x B ema h raino nassembly an of creation the mean we , fst fnuoso size of neurons of sets of y oasalrsto ern,ltu alit call us let neurons, of set smaller a to 2 x ooelpsbtnilywith substantially overlap to , and B , y y sebypoeto a endemonstrated been has projection Assembly ). 1 B rae togrectto narea in excitation stronger a creates k o oasmle nteascaincor- association the in assemblies do How ilb eetdt r,cl it call fire, to selected be will .Nx,tesimultaneous the Next, 2A). (Fig. neurons x x B narea in to niiinwl onlmttefiigin firing the limit soon will Inhibition . B x B y y = fires. 1 omdi ontemarea downstream a in formed ,ad sarsl,anwstof set new a result, a as and, ), area(y steprfr otx Once cortex. piriform the is h rcs a iil pre- vividly was process The A? y x x k eeslce ohv strong have to selected were and otne rn,asequence a firing, continues eb()hpteie that hypothesized (4) Hebb project(x narea in ) and y y y n nutvl expects intuitively one , k projection 1 ob ihrta ran- than higher be to x ti vra a be can overlap —this ern ihhighest with neurons x n uhthat such and , tcnb adthat said be can It A B = y y h euto the of result the , oarea to nadownstream a in parent(y , y ilb created. be will B 1 y ossig in consisting, , , 2 fa assem- an of y n would One . ) operation B 50% B ). excites y x B (one will has for by k h te hl te el ev h sebist ananits maintain to to assemblies migrating the each leave to from cells size other cells while with imprint other they overlap, the entities their two cooc- the increasing that of the affinity, to is by observed adapt results other entities or similar different currence, many familiar parsimo- two and compelling particular imprinting A this assemblies place. a of that of explanation in has person nious image image this of combined the shown a Pyramids— the been to sibling—once the to subject’s respond as the responding person—say, place—such also consistently familiar to MTL particular starts subject’s a of a image in neuron that revealed a subjects human of recordings electrocorticography Completion. Pattern and Association perform to typically seems unit these. it with linear that competitive rectified found level we a the at and or leaning, sigmoid machine network, the in neural used of deep a place of the experi- layer in each in in RP&C nonlinearity used the we as Finally, first ments 21). (as ref. well in remarkably empirically assemblies noticed of that overlap For simulations, preserves properties. through RP&C useful and potentially analytically of can establish, number we and example, a generally, have to com- more shown neural learning be of computational primitive important and an putation, is RP&C that believe We call of we applications which operator, of this consists here of described area, computation each assembly is in that inhibition all through the selection, place; and special in recurrently areas, and be No within, between, to connectivity plasticity). synaptic ran- required random to is generally: are needed more addition circuits life neural (in for purpose crucial selection be and to domness known forces two by h eetieo h sebyCluu smerge. is Calculus Assembly the merge(x of repertoire the Merge. further be should which pursued. computation direction research probabilistic a of assemblies, genre through a for possibilities fascinating 2 very striking (Fig. complete, assembly to rather able whole being the assembly a accurately, the of in parts our small In happens with way, firing. completion pri- subset a pattern particular the certain experiments, on a depending with probability happens ori completion such presumably, (8); bly is assemblies 22. ref. see view, this of of aspects discussion asso- a mathematical complex for certain of assemblies; arena assemblies area’s the the between 10% between as patterns area association to overlap ciation brain The a 8 an of assembly. about conception suggest an a of evokes of 15 MTL size ref. the the in of of In assemblies results cells. associated The few between substantial: very operation quite be the should is from any, resulting if overlap overlap, contrast, size may random assembly area, of this same the root tions, the If square in chance. if the them, by than of little two a any result, overlap a as and, rons, plasticity networks timing-dependent of spiked (18). simulations through (STDP) by neurons obtained spiking been of have results similar the of of parents firing two sustained simultaneous the that simulations, through associate(x denoted is area assemblies area two brain of same association The another. one ebleeta soito n atr opeinoe up open completion pattern and association that believe We involving phenomenon well-studied and important One neu- of sets random ways, many in and, large are Assemblies k x area(x u of out nrsos otefiigo ml usto t cells its of subset small a of firing the to response in h otspitctdadcmlxoeainin operation complex and sophisticated most The k , esyta h w sebisare assemblies two the that say we ; y , A, n = ) h rn ftewoeassem- whole the of firing the completion: pattern el urnl eevn ihs yatcipt All input. synaptic highest receiving currently cells x z area(y hsoeainsat ihtoassemblies two with starts operation this ), and y osefc uhices noelp while overlap, in increase such effect does ) mlct ecnso,aayial and analytically show, can We implicit. n sw fe suei u simula- our in assume often we as , admpoeto n cap and projection random narcn xeiet(15), experiment recent a In F1 NSLts Articles Latest PNAS , and y ihtecommon the with ), F2). associate(x associating k x saotless about is and (RP&C). Denoted y | nthe in x f9 of 3 with and , y )

NEUROSCIENCE Fig. 2. Assembly operations. (A) An RP&C: If an assembly x fires in area A, and there is afferent connectivity to area B, the ensuing synaptic input will cause a set of k neurons in area B, call it y, to fire. The set y contains the k neurons in area B that receive the highest synaptic input from assembly x. This is an important primitive called RP&C. (B1–B3) If assembly x fires again, afferent synaptic input from area A will be combined with recurrent synap- tic input from y1 in area B to cause a new set of k neurons in area B, y2, to fire. Continued firing of x will create a sequence of sets of firing neurons in area B: y1, y2, y3, .... In the presence of Heb- bian plasticity, this sequence converges exponentially fast to a set of neurons, an assembly y which we call the projection of x to B. If, subsequently, x fires, y will fol- low suit. (B4) Exponential convergence of assembly projection: The horizontal axis is the number of times assembly x fires; the vertical axis is the total number of neu- rons in area B that fired in the process. Different color lines represent different values of the plasticity coefficient β; for higher levels of β, convergence is faster. (C) Synaptic density within the resulting assembly increases with higher values of β (blue line), and is always higher than the baseline random synaptic connectivity p (dashed red line). (D) Preservation of over- lap: Assembly overlap is preserved under RP&C. The x axis represents the overlap in two assemblies that are then projected to a new area once (RP&C); the y axis repre- sents the percentage overlap of the two resulting assemblies in B.(E) Association: If two assemblies in two different areas have been independently projected in a third area to form assemblies x and y, and, subsequently, the two parent assem- blies fire simultaneously, then each of x, y will respond by having some of its neu- rons migrate to the other assembly; this is called association of x and y. Such over- lap due to association may reach 8 to 10% (15). The figure shows results from simu- lation: first, two separate, stable assem- blies are created in two areas, A and B. Then, the assemblies in A and B are pro- jected simultaneously into C for x time steps, represented by the x axis; the y axis shows the resulting overlap in the pro- jected assemblies of A and B after the simultaneous firing. (F1) Pattern comple- tion: The firing of a few cells of an assembly results in the firing of the whole assembly. (F2) An assembly is created in an area A by repeated firing of its parent; the number of times the parent fires is depicted in the horizontal axis. Next, 40% of neurons of the assembly are selected at random and fire for a number of steps; the y axis shows the overlap of the resulting assembly with the original assembly. With more reinforcement of the original assembly, the subset recovers nearly all of the original assembly. (G1) Merge: The most advanced, and demanding in its implementation, operation of the Assembly Calculus involves five brain areas; areas B and C contain the assemblies x, y to be merged, while areas B0 and C0 contain the parents of these assemblies. The simultaneous spiking of the parents induces the firing of x and y, which then initiates a process whereby two-way afferent computation between areas B and A, as well as areas C and A, results in the adjustment of both x and y and the creation of an assembly xy in area A having strong synaptic connections to and from both x and y.(G2) Speed of convergence depends critically on β.

y, in different brain areas, and creates a new assembly z, in a can, in turn, be combined with others. This is a crucial step in the third area A, such that there is ample synaptic connectivity from creation of the hierarchies (trees of entities) that seem necessary x and y to z, and also vice versa, from z to both x and y. for the syntactic processing of language, but also for hierarchical Linguists had long predicted that the human brain is capable thinking more generally (deduction, discourse, planning, story- of combining, in a particularly strong sense, two separate entities telling, etc.). Recent fMRI experiments (25) have indicated that, to create a new entity representing this specific combination (23, indeed, the completion of phrases and sentences (the completion 24), and that this ability is recursive in that the combined entity of auditory stimuli such as “hill top” and “ship sunk”) activates

4 of 9 | www.pnas.org/cgi/doi/10.1073/pnas.2001893117 Papadimitriou et al. Downloaded by guest on September 30, 2021 Downloaded by guest on September 30, 2021 iihbto–niiinpiiie sage nrf 6i em of (27). terms cells in peptide 26 intestinal ref. vasoactive in argued is operation primitives disinhibition–inhibition the by marked excita- is not the does operation default, the this by by area firing, that, an in assume assembly cells of also tory whole fragment We the tiny implausible. to pattern a seem of leads firing phenomenon assembly which the in an of view above, in discussed it; completion assembly activating for an place time in the at say, eepiil asdt r yteoperation the by fire assembly to an caused explicitly that be assume We operations. area in sys- fired assembly just the otherwise. has the Accordingly, which assembly operation action. of the an fies further state contains appropriate current Calculus Assembly mechanisms trigger the readout and sense useful, tem that functionally Buzs exist be assemblies, must to on state assemblies simple paper for the a review affecting First, tial and system. sensing the of operations, control low-level ob bet rt ennflporm—nfc,parallel read segments fact, if containing programs—in example, high- meaningful for which as write system, such within to want this we system able programs—in Ultimately, formal expressed. be be a to can behavior provide cognitive to level is simplified Operations. Calculus our Control and in Readout as well as (19), of networks STDP model. of with simulations neurons detailed ref. spiking (see through mice” verified chase of imentally “cats plausibility sentence the The of 19). for subject designating the instrumental as as be brain—such “cats” the to in binding hypothesized variable been implementing has projection rocal aaiiro tal. et Papadimitriou resulting project(x is in plexity reported are 2 results Fig. also simulation see plasticity our 2; stronger Section operations; requires other and them, than recipro- between ample with connectivity involves areas it brain cal as different work, five this of in complex coordination most the the far constituent by is Methods, the and two-way and have assembly must merge new that respec- in pro- the assemblies. assume generation, the both lan- between to in language natural traversed connections (e.g., in and is direction be parsing it selection downward tively), language to the word of need in cesses and in hierarchies upward reside as these implicated the represent- assemblies such As area word words assemblies guage. the for of Wernicke’s presumably, assemblies merge in “sunk”); is, the and area (that of Broca’s “ship” constituents results in their assemblies the ing by are represented that are sentences and has of processing area syntactic Broca’s the in that implicated language. Recall be to do area. sentences. believed ship” Broca’s been for long “hill activate 45 as to BA such BA seem sequences triangularis not opercularis word pars unstructured pars the contrast, the and In phrases, particular, for area—in 44 Broca’s of parts ial,teAsml aclscnan w ipecontrol simple two contains Calculus Assembly the Finally, additional certain introduce next we mind, in goal this With ial,asmlroeainwt iia e idrcom- milder yet similar with operation simpler a Finally, in explained merge, implementing for algorithm Our phrases that is findings these of explanation parsimonious A y (A) reciprocal.project(x , a togbcwr yatccnetvt to connectivity synaptic backward strong has A, snl hnproject(x then null is y ihteadtoa ucinlt htthe that functionality additional the with ), disinhibit(A) A r niie,uls xlctydisinhibited explicitly unless inhibited, are reciprocal.project G1 h luiiiyo the of plausibility the inhibit(A); and x read scetd ehns sset is mechanism a created, is G2. h ups fteAssembly the of purpose The o iie eid hs end whose period, limited a for , A, prto.I ninfluen- an In operation. y , ti netninof extension an is It ): ´ k 7 rpssthat, proposes (7) aki A, read(A) x y fire(x n eun null returns and A, ). na area an in a enexper- been has hti to is That ). htidenti- that x Recip- . Results A can repeat the in assemblies disinhibit(A) two of overlap the area increase same to is effect whose inhibit(B fire(x repeat disinhibit(B function the on relying program, this about after formed is bly inhibit(B repeat disinhibit(B following: the to equivalent rpsdipeetto fteceto of creation the of implementation proposed a inhibit(A). command the Calculus, Example. ne prpit supin,abtaycmuain on implementing, computations of arbitrary capable assumptions, appropriate is under above defined as Calculus Assembly Power. Computational in described is language, natural of generation samcielann rmwr—nprd fcus,b the by course, pre- of framework—inspired, and learning model machine to a computing assembly meant hyperdimensional is brain, While the are a in ends. function here as similarity cognitive dict introduced the of where as thought is operations its be this of vector but can characteristic support; assembly the namely, an vector, high-dimensional Indeed, dimensions. permutation and of multiplication, component-wise addition, component- wise typically operations, algebraic with equipped of vectors tradition research the computing to resemblance tain pro- are explain to operations, intended system of previous computational phenomena. suite a of cognitive of a aware basis the with not as assemblies, are posed which we However, in 30). work ref. com- to (see connection putation in (29) vivo occasionally, extensively in decades—even, discussed past identified and the 5), over (8), (4, experimentally studied hypothesized 7), been (6, have They new: not Work. Related planning, reasoning, as such language. brain, and human the in func- especially cognitive be tions, advanced can way—underlies however, this which, to precise, abstracted and one usefully neu- structured a embolden less rather far may system—or system ral computational Calculus a computa- Assembly such ensuing that the hypothesize The neurons of fir- plasticity. excitatory power the Hebbian of tional to and populations reduced inhibition connected ultimately with randomly be can of where operations operations, ing sophisticated these rather of of each repertoire a with tem Hypothesis. Assembly The be principle, in Calculus. can, Assembly registers, the paral- hundred by few out hundred carried a few using a and with steps, lel computation parallel computation any space-bounded equivalence (28)—that and well-established computation the parallel of between view suggests—in This bits. iso eoyad ne uhwae assumptions, weaker much under memory—and, of bits o nte ipeeape h command the example, simple another For assem- stable a parameters, typical with that, show Simulations a oeeaoaepormi h sebyCluu,for Calculus, Assembly the in program elaborate more far A sebisadteroeain stetdhr eracer- a bear here treated as operations their and Assemblies fire(parenty T T o ipeeapeo rga nteAssembly the in program a of example simple a For ie:d nparallel in do times: fire(x times: stnaon othis: to tantamount is A, serf.3–3,sseso ihdmninlsparse high-dimensional of systems 31–33), refs. (see ). ). sebiso xiaoynuosae fcourse, of are, neurons excitatory of Assemblies ) ) ) ni read(B until ))} htthe that Appendix) (SI show can We project(x h sebyCluu safra sys- formal a is Calculus Assembly The T ) 10 = ) sntnull not is tp.A lentv eso of version alternative An steps. , B {fire stefollowing: the is read, , y NSLts Articles Latest PNAS ) where (parent((x ytxtrees syntax associate(x Discussion. hyperdimensional area(x )), = ) O | nthe in O ( f9 of 5 is A, √ , ( k y n n k ), ) )

NEUROSCIENCE brain, like many other such frameworks—and used successfully, Since association between assemblies is thought to capture for example, for learning semantics in natural language (33). In affinity of various sorts, the question arises: If two associated sparse vector systems, there is no underlying synaptic connectiv- assemblies x and y are both projected in another area, will the ity between the dimensions, or partition of the dimensions into size of their overlap be preserved? Our results, both analytical brain areas. Finally, the operations of the Assembly Calculus and through simulations, strongly suggest that assembly overlap (project, associate, and merge) are very different in nature, style, is indeed conserved reasonably well under projection (see Sec- detail, and intent from the operations in sparse vector systems tion 2 and Fig. 2). This is important, because it means that the (add, multiply, and permute). signal of affinity between two assemblies is not lost when the two Assembly computation is closer in spirit to Valiant’s model assemblies are further projected in a new brain area. of neuroids and items (34), which was an important source of Pattern completion involves the firing, with some probability, inspiration for this work. One difference is that, whereas, in of an assembly x following the firing of a few of its neurons (8, Valiant’s model, the neuron (called neuroid there) has con- 35). Simulations in our model (Fig. 2F2) show that, indeed, if the siderable local computational power—for example, to set its creation of an assembly is completed with a sufficient number of presynaptic weights to arbitrary values—in our formalism, com- activations of its parent, then firing fewer than half of the neu- putational power comes from a minimalistic model of inhibi- rons of the assembly will result in most, or essentially all, of the tion and plasticity; both models assume random connectivity. assembly firing. Another important difference is that, in contrast to an item in Valiant’s theory, an assembly is densely intraconnected, while the Reciprocal Projection and Merge. Reciprocal projection, denoted mechanism for its creation is described explicitly. reciprocal.project(x, B, y), is a more elaborate version of projection. It involves parent(x) firing, which causes x in area Results and Methods A to fire, which, in turn, causes, as with ordinary projection, a set Projection. The operation project(x, B, y) entails activating, y1 in B to fire. The difference is that now there is synaptic con- repeatedly, assembly x while B is disinhibited. Such repeated nectivity from area B to area A (in addition to connectivity from activation creates, in the disinhibited area B, a sequence of sets A to B), which causes, in the next step, x to move slightly to a of k cells, let us call them y1, y2, ... , yt , .... The mathematical new assembly x1, while y1 has become y2. This continuing inter- details are quite involved, but the intuition is the following: Cells action between the xt and the yt±1 eventually converges, albeit in B can be thought of as competing for synaptic input. At the slower than with ordinary projection, and under conditions of first step, only x provides synaptic input, and thus y1 consists ampler synaptic connectivity and plasticity. The resulting assem- of the k cells in B which happen to have the highest sum of bly y has strong synaptic connectivity both to and from x (instead synaptic weights originating in x—note that these weights are of only from x to y, as is the case with ordinary projection). That subsequently increased by a factor of (1 + β) due to plasticity. At reciprocal projection works as described above has been shown the second step, neurons in both x and y1 spike, and, as a result, both analytically and through simulations in our model, as well a new set y2 of “winners” from among cells of B is selected; as in simulations in a more realistic neural model with explicit for typical parameters, y2 overlaps heavily with y1. This contin- inhibitory neurons and STDP in ref. 26. ues as long as x keeps firing, with certain cells in yt replaced by The operation merge(x, y, A, z) is essentially a double recip- either “new winners”—cells that never participated in a yt0 with rocal projection. It involves the simultaneous repeated firing, in t 0 < t—or by “old winners”—cells that did participate in some different areas, of the parents of both x and y, which causes 0 yt0 with t < t. We say that the process has converged when there the simultaneous repeated firing, also in two different areas B are no new winners. Upon further firing of x, yt may evolve and C , of x and y. In the presence of enabled afferent two- further slowly, or cycle periodically, with past winners coming way connectivity between A and B, and also between A and in and out of yt ; in fact, this mode of assembly firing (cells of C , this initiates a process whereby a new assembly z is even- the assembly alternating in firing) is very much in accordance tually formed in area A, which, through its firing, modifies the with how assemblies have been observed to fire in Ca+ imaging original assemblies x and y. In the resulting assemblies, there is experiments in mice; see, for example, ref. 35. It is theoretically strong two-way synaptic connectivity between x and z, as well as possible that a new winner cell may come up after convergence; between y and z. Analytical and simulation results are similar to but it can be proved that this is a highly unlikely event, and those for reciprocal projection (Fig. 2G1). we have never noticed it in our simulations. The number of steps required for convergence depends on the parameters, but, Simulations. We gain insights into the workings of our model, most crucially, on the plasticity coefficient β; this dependence is and validate our analytical results, through simulations. In a typ- fathomed analytically and through simulations in Fig. 2B4. ical simulation task, we need to simulate a number of discrete time steps in two or three areas, in a random graph with ∼ n2p Density. It was postulated by Hebb (4) that assemblies are nodes (where the notation ∼ f means ”a small constant multi- densely interconnected—presumably such density was thought to ple of f ). Creating this graph requires ∼ n2p computation. Next, cause their synchronized firing. Since assemblies are created by simulating each firing step entails selecting, in each area, the k projection, increased density is intuitively expected: Cells in yt out of n cells that receive that largest synaptic input. This takes are selected to be highly connected to yt−1, which is increasingly ∼ knp computations per step. Since the number of steps needed closer to yt as t increases. It is also observed in our simulations for convergence is typically much smaller than the ratio n/k, the (Fig. 2C) and predicted analytically in our model. n2p computation for the creation of the graph dominates the computational requirements of the whole simulation (recall that Association and Pattern Completion. Our simulations (Fig. 2E), as n >> k). well as analytical results, show that the overlap of two assem- In our simulator, we employ a maneuver which reduces this blies x, y in the same area increases substantially in response to requirement to ∼ knp, enabling simulations of the required scale simultaneous firing of the two parent assemblies parent(x) and on a laptop. The trick is this: We do not generate all ∼ n2p edges parent(y) (assumed to be in two different areas). The amount of of the random graph a priori, but generate them “on demand” postassociation overlap observed in our simulations is compati- as the simulation proceeds. Once we know which cells fired at ble with the estimates in the literature (36, 37), and increases the previous step, we generate the k cells in the area of inter- with the extent of cooccurrence (the number of consecutive est that receive the most synaptic input, as well as their incoming simultaneous activations of the two parents). synapses from the firing cells, by sampling from the tail of the

6 of 9 | www.pnas.org/cgi/doi/10.1073/pnas.2001893117 Papadimitriou et al. Downloaded by guest on September 30, 2021 Downloaded by guest on September 30, 2021 aaiiro tal. et Papadimitriou pro- the once model, our extent, in large even a jection But, to behavior. suppresses, sequential steps this the time experiments, discrete our and of proofs assumption analytical our times. in different and at formalism, firing a neurons an in different fire of with instead “pattern” but neurons simultaneously, observa- the activated current all (18), not as are simulations assembly well neurorealistic as and (8) (4) firing. tions prediction synchronous of Hebb’s of capable set to area a According brain simply a is in assembly neurons far, excitatory so narrative our In conundrum. may system brain. human formal the this in especially to functions, is akin cognitive work underlie something our that to speculation Central fraught the power. is computational operations, sys- considerable control formal simple with certain resulting and with The equipped plasticity gen- connectivity. tem, through is synaptic rely probability, of apparatus not high randomness their do with instead, operations on-the-fly, These circuits; erated ones. neural two to original preexisting connectivity furthermore, two on ample and, the with similarity, assembly from cooccur- new other and a observed or create reflect to world to assemblies assemblies the order two in in projection, overlap through rence area their “copy” brain a increase new create to a to The assembly in firing. an itself enable near-simultaneous of system of this capable of and operations area, brain com- object same the elementary assembly, model Its an to functions. is intended cognitive system underlying formal putations a defined have We Discussion results: simulation our extend com/dmitropolsky/assemblies. and reproduce to the select to area same next. fire the will previ- in with cells compare generated then ously we distribution; binomial appropriate ocue nrf 1 eedn ntepeievle fthe of values precise the on Depending 11. parameters ref. edges in directed suppose concluded two example, of be For existence can and the regimes. assemblies on such of conditioned conclu- that, stability in our and fact, strengthened density In further the kind. this regarding nonuniform of to sions routinely connectivity extended synaptic be random can randomness). results from analytical not (but 12) Our uniformity (11, from departure measurements a experimental suggest however, connectivity; tic syllables of below). language frequency on the discussion Hz, the 4 (see and at language formed brain in be human can the In assemblies by representations. that modified conjecture stable we fairly here typ- as contrast, assemblies, assembly on implicitly, in discourse considered, the usual do ically the computational at to a from of departs place hypothesized basis also the not takes system as is assemblies are plasticity of it use we Our as which operations. happens, scale, in learning time models and subsecond many works brain of aspect the important aware an way as the studied well of is plasticity While plasticity. much not is an probability) than nonzero larger in with neurons assembly of set An (the support this: is assembly set area an spiking the of of definition out principled and in coming keep often B ‡ nmsclmtpo,a sebyi huh ob o notv u odsequence. cord a but octave an not be to thought is assembly an metaphor, musical In hti nasml,eaty h rcs ento sa is definition precise The exactly? assembly, an is What possible it making online, available is system simulation Our u eut n iuain sueuiomyrno synap- random uniformly assume simulations and results Our of role crucial the is operations assembly our of aspect One r hthdnvrfie eoe,asalnme fneurons of number small a before), fired never had that fire (a B , project(x sasas itiuino h ern farea of neurons the on distribution sparse a is c h rsneo edge of presence the ), k n . , k , e fecttr ern l eogn othe to belonging all neurons excitatory of set a p hswudlkl rge brha paradox” “birthday a trigger likely would this , , B , y ) a tblzd(onwnuosi area in neurons new (no stabilized has (b , c ) smc oelkl,as likely, more much is y t n possible One . http://github. k B el that cells ‡ whose , nour In (a , b k ) )Asqec fsnecscnitn ffu ooylbcwords monosyllabic four of consisting sentences of sequence A 3) marked is presented sentences and phrases of completion The 2) temporal superior left the of areas brain different Two 1) ol tb htasmle n hi prtospa hsrole? this play operations their and other.” assemblies each that with be communicate it regions Could brain call distinct I two phrases, which what “for are (words, that, there proposes language 134) language p. of 38, (ref. Friederici constituents Angela sentences)? rep- the of these on-the-fly, of out? processing, carried through and resentations events generation transmission the of involve sequence and it orchestrated Does this areas is these How fibers. of sequence complex activations a and entail to of 45, Syn- seems axon hemisphere. language BA as left of the well processing an in tactic as all 44 areas, gyrus, four temporal BA these connecting superior fibers areas the Broca’s in point area MTL, Wernicke’s results the experimental involving other ing, of chap- plethora 38, a a ref. to and to these According accomplished? 4, this ter must is one How sentences, parsing—and ask, the generation—of next during the brain during our also in presumably constructed is trees syntax recent informative and subject. of comprehensive the correlates extremely of see treatment an brain clues; for the some 38 are provide experiments ref. which Recent then Merge? correct, 3A). and (Fig. syntax is sentence theory a a of this create structure If the can captures Merge one which of form tree, applications to syntax recursive units unit; syntactic minimalistic two syntactic a combines new as Merge proposed syntax. the been of particular, basis In has 24). Merge (23, ability primitive this a linguistic of in core sentences the of at structure (i.e., lies the syntax language) govern for that capability rules the brain’s mastering human for how the that on and humans, below to speculate we language. in ending, implicated cognitive be In may higher assemblies humans. mediating in in involved functions be may operations their Language. and Assemblies interesting. very be would effect leading same operations the control to plausible Replacing biologically system. more Assem- computational with the them programmable rendering a of purpose Calculus the bly justifi- for such added were in and lack fire, cation, elementary, operations—read, and control simple and merge read disinhibit—however the language the of contrast, discussion In (for the below). parsi- see results operations, provide project experimental reciprocal 2) and of and analysis, explanations mathematical monious simulations are in by reproduced 1) predicted be can which and they that events sense the population in plausible, neural and to projection, merge—correspond reciprocal association, here—projection, sented the hence and density, synaptic assemblies. of the stability, cell enhance further same would the that cells from assembly connections two tic of (existence phenomenon foeacpstehptei ht ned oehn knto akin something indeed, that, hypothesis the accepts one If unique is faculty language the that argued long have Linguists pre- as Calculus Assembly the of operations basic The t1 ,ad4H,cnitn ihtehptei htsyn- that the hypothesis during constructed the being with are consistent peaks sentences (41). Hz, presentation the energy for 4 with trees four responses and tax of brain 2, rate of 1, sylla- the pattern articulated at of a at frequency elicits presented natural bles) fish”) (the second eat per cats words “Bad as (such known (40), hemisphere syntax. left in the implicated in be area to Broca’s in presented activity a language) by of in object words the of vs. subject use the (39). the sentence to in respond to implicated 19th seem the be since to known area, century Wernicke’s contains (which gyrus ucinlcria network cortical functional ehpteie htasmle and assemblies that hypothesized We o eia n ytci process- syntactic and lexical for irrnua ensembles neural mirror a fteprn assembly) parent the of NSLts Articles Latest PNAS b , c ihsynap- with through | f9 of 7

NEUROSCIENCE articulate—a particular fact. The raw fact to be put together is denoted here by Im—an image, sensed or mental. In the first line, a brain area containing the lexicon is searched in order to identify the verb (the action in the fact relayed in Im), the sub- ject (the agent of the fact), and the object (the patient of the fact). Here we are assuming that the lexicon is a searchable collection of tens of thousands of assemblies in the left MTL of the subject, representing words in the language; this is in rough agreement with current views (42), even though much still remains to be understood about the representation of words and the associated information. We also assume that the corresponding brain func- tions find-verb, etc., are already in place. For the example “The boy kicks the ball,” the three assemblies x, y, and z are identi- fied after the first step, encoding the words “kick”, “boy”, and “ball”, respectively. In the second line, these three assemblies are projected to the three subareas of Wernicke’s area special- izing in the verb, subject, and object of sentences, respectively. In fact, instead of ordinary projection, the reciprocal.project operation is used, as first proposed in ref. 26, for reasons that will become clear soon. Next, an assembly p is formed in the pars opercularis of Broca’s area (BA 44) representing the verb phrase “kicks the ball” through the merge operation applied to 0 0 Fig. 3. A proposed cortical architecture for syntax in language (see also the two assemblies x and z encoding the constituent words of ref. 38). In order to generate a simple sentence such as “The boy kicks the the phrase in Wernicke’s area. Finally, an assembly s correspond- ball,” the subject must first find, in the left MTL, the representations of ing to the whole sentence is formed in the pars triangularis of the verb, subject, and object of the sentence. Next, these three assemblies Broca’s area (BA 45) via the merge of assemblies p and y 0, com- are reciprocal-projected to the corresponding subareas of Wernicke’s area. pleting the construction of the rudimentary syntax tree of the Next, the verb and the object are merged in BA 44 of Broca’s area to form a whole sentence. representation p of the verb phrase “kicks the ball,” and, finally, p is merged The Assembly Calculus program above accounts only for the with the subject to create a representation s of the whole sentence in BA 45. This concludes the phase of building of the sentence’s syntax tree. Now, first phase of sentence production, during which the syntax tree in order to articulate the sentence, s fires, and this results in the firing of the of the sentence is constructed. Next, the sentence formed may constituents of the tree, until, eventually, the three words in the MTL fire be articulated, and we can speculate on how this process is car- in the correct order for the subject’s language—an order learned at infancy ried out: Assembly s is activated, and this eventually causes the (in English, subject–verb–object). This latter activation of the three words assemblies x 0, y 0, z 0—the leaves of the syntax tree—to fire. The mobilizes the corresponding motor functions resulting in the production of activation of these three assemblies is done in the order spe- sound. cific to the speaker’s language learned at infancy (for example, in English, subject–verb–object). Note that the first phase of build- ing the syntax tree was largely language independent. Eventually, We propose a dynamic brain architecture for the generation the lexicon assemblies will be activated in the correct order (this of a simple sentence, powered by assemblies and the opera- activation was the purpose of using the reciprocal.project tions reciprocal.project and merge, and consistent with the operation), and these, in turn, will activate the appropriate motor experimental results and their interpretations outlined above functions which will ultimately translate the word sequence into (Fig. 3). In particular, we propose that the construction of the sounds. syntax tree of a simple sentence being generated by a subject The above narrative is only about the building of the basic syn- can be implemented by the following program of the Assembly tactic structure—the “scaffold”—of extremely simple sentences, Calculus: and does not account for many other facets: how the three find- tasks in the first line are implemented; how the noun phrases do in parallel: are adorned by determiners such as “the,” and how the verb find-verb(Im, MTL, x), is modified to reflect person and tense (“kicks” or “kicked”), find-subj(Im, MTL, y), and the important inverse processes of parsing and comprehen- find-obj(Im, MTL, z); sion, among a myriad of other aspects of language that remain a do in parallel: mystery. reciprocal.project(x, WVb, x 0), reciprocal.project(y, WSubj, y 0), ACKNOWLEDGMENTS. We are grateful to Larry Abbott for his insightful 0 ideas and comments concerning this work. We also wish to thank several reciprocal.project(x, WObj, z ); colleagues for many helpful discussions and useful feedback: Richard Axel, merge (x 0, z 0, Broca44, p); Bob Berwick, Costis Daskalakis, Boris Harizanov, Pentti Kanerva, Robert merge (y 0, p, Broca45, s). Legenstein, Chris Manning, Bruno Olshausen, Les Valiant, and Rafa Yuste. C.H.P.’s research was partially supported by NSF Awards CCF1763970 and CCF1910700, and by a research contract with Softbank; S.S.V.’s research was partially supported by NSF Awards 1717349, 1839323, and 1909756; and The generation of a sentence such as “The boy kicks the ball” W.M.’s research was partially supported by the Human Brain Project Grant starts with a desire by the subject to assemble—and possibly 785907 of the European Union.

1. R. Axel, Q & A. Neuron 99, 1110–1112 (2018). 4. D. O. Hebb, The Organization of Behavior: A Neuropsychological Theory (Wiley, New 2. S. T. Piantadosi, J. B. Tenenbaum, N. D. Goodman, Bootstrapping in a language of York, NY, 1949). thought: A formal model of numerical concept learning. Cognition 123, 199–217 (2012). 5. M. Abeles, Corticonics: Neural Circuits of the Cerebral Cortex (Cambridge University 3. S. T. Piantadosi, J. B. Tenenbaum, N. D. Goodman, The logical primitives of thought: Press, 1991). Empirical foundations for compositional cognitive models. Psychol. Rev. 123, 392–424 6. K. D. Harris, Neural signatures of cell assembly organization. Nat. Rev. Neurosci. 6, (2016). 399–407 (2005).

8 of 9 | www.pnas.org/cgi/doi/10.1073/pnas.2001893117 Papadimitriou et al. Downloaded by guest on September 30, 2021 Downloaded by guest on September 30, 2021 4 .D asr .Cosy .T ic,Tefclyo agae hti t h a it, has who it, is What language: of faculty The Fitch, T. W. Chomsky, N. Hauser, D. M. Chomsky, N. 24. Berwick, C. R. 23. 1 .Dsut,C .Sees .Nvah,Anua loih o fundamental a for algorithm neural A Navlakha, Anari N. S. 22. Stevens, F. C. Dasgupta, S. 21. 0 .Fie .Plg .Gy h es -ugahproblem. k-subgraph dense The Guy, for K. Peleg, pointers D. Feige, Assembly U. Maass, 20. W. Vempala, S. Papadimitriou, H. C. Legenstein, R. 19. Pokorny C. 18. computation and brain the in projection “Random Vempala, S. S. Papadimitriou, H. C. memory 17. “Long-term Vempala, S. S. Papadimitriou, H. C. individual Maass, by W. memories new Legenstein, of R. encoding Rapid 16. Fried. I. Quiroga, Quian R. Ison, J. M. 15. 4 .M Franks M. K. memory. 14. and perception visual for codes Neuronal Quiroga, Quian R. 13. Schl J. A. Guzman, S. 12. Sj J. P. Song, S. 11. Erd P. 10. aaiiro tal. et Papadimitriou .J-.K ilr .Aznha,L arloRi,R ut,Vsa tml eri intrin- recruit Buzs stimuli G. Visual 9. Yuste, R. Carrillo-Reid, L. Ayzenshtat, I. readers. Miller, and K. J.-e. synapsembles, assemblies, 8. Cell syntax: Neural Buzsaki, G. 7. n o i tevolve?Science it did how and 2018, 10880–10890. pp. NeurIPS 2018), 2018, Associates, (Curran Systems Eds. Wallach, Processing M. Information H. Neural on ference in neurons” of optn problem. computing (2001). aibebnigi ewrso pkn ern.aXv11.39 1 November (11 arXiv:1611.03698 neurons. spiking of networks 2016). in binding variable 2017). September (14 bioRxiv:188938 STDP. pp. through model 2019), circuit Informatik, fuer (Leibniz-Zentrum Ed. Blum, A. 57:1–57:19. 2019, ITCS in neurons” Conference, of assemblies with Institute 57:1–57:15. (Massachusetts pp. Ed. 2018) Karlin, MA, R. Cambridge, Technology, A. of Conference, (ITCS) in Science problem” Computer retical k-subgraph densest the and brain. human the in neurons cortex. network. ca3 hippocampal the in completion (2014). ensembles. cortical generated sically (2010). 362–385 gia circuits. cortical local in connectivity synaptic of features Sci. Acad. 2–4 (2016). 227–241 83, s .Rni nteeouino admgraphs. random of evolution the On Renyi, A. os, ˝ aki, Neuron ´ Sote nlsso iceetno eopsto n assemblies and decomposition tensor discrete of analysis “Smoothed al., et 76 (1960). 17–61 5, h ri rmIsd Out Inside from Brain The soitosbtenmmr rcseeg nagnrcneural generic a in emerge traces memory between Associations al., et eurn icir yaial hpsteatvto fpiriform of activation the shapes dynamically circuitry Recurrent al., et 95 (2011). 49–56 72, dacsi erlIfrainPoesn ytm 1 nulCon- Annual 31: Systems Processing Information Neural in Advances ostr ¨ m .Ril .Nlo,D .Ckosi,Hgl nonrandom Highly Chklovskii, B. D. Nelson, S. Reigl, M. om, ¨ Science l .Foshr .Jns yatcmcaim fpattern of mechanisms Synaptic Jonas, P. Frotscher, M. gl, o ¨ h nyU:Lnug n Evolution and Language Us: Only Why 9–9 (2017). 793–796 358, Neuron 5917 (2002). 1569–1579 298, 0hInvtosi hoeia optrScience Computer Theoretical in Innovations 10th 2–3 (2015). 220–230 87, rc al cd c.U.S.A. Sci. Acad. Natl. Proc. Ofr nvriyPes 2019). Press, University (Oxford rceig f9hInvtosi Theo- in Innovations 9th of Proceedings Science 1712 (2016). 1117–1123 353, LSBiol. PLoS ul ah nt Hungary. Inst. Math. Publ. Algorithmica MTPes 2016). Press, (MIT E4053–E4061 111, 6 (2005). e68 3, Neuropsycholo- 410–421 29, Neuron .Bengio, S. 68, 8 .H Papadimitriou, H. C. 28. 2 .F a,C hlis .Pepl otclntokfrsmnis(e osrcigthe constructing semantics:(de) hierarchical for network of based cortical tracking A Poeppel, sub-region Cortical D. Phillips, Poeppel, C. A D. Lau, F. Tian, E. brain: X. 42. human Zhang, H. the Melloni, L. in Ding, Merge N. Friederici, 41. D. in A. meaning Zaccarella, sentence E. encoding for 40. architecture An Greene, D. J. Frankland, M. S. 39. and Friederici, personal D. of A. coding Long-term 38. Quiroga, Quian R. Fried, I. Ison, J. M. Falco, De E. 37. visually-guided Triggering Rey Yuste, G. H. R. 36. Akrouh, A. Yang, W. Han, S. Carrillo-Reid, L. 35. Valiant, G. L. utterances. for 34. spaces semantic distributed High-dimensional dis- Kanerva, P. in Karlgren, J. computing 33. to introduction An computing: Hyperdimensional Kanerva, P. 32. composi- for algebra Convolution representations: reduced “Holographic Plate, T. 31. computing. assembly Neural Ranhel, J. 30. feature of notion the abandon to time it is When Hebb: versus Barlow Eichenbaum, variations. H. and 29. Themes circuit: neocortical The Shepherd, G. M. G. Harris, D. K. 27. neural The M syntax: G. M. by Legenstein, Building R. Friederici, D. 26. A. Makuuchi, M. Meyer, L. Zaccarella, E. 25. Neurosci. eairb oorpi ciaino atr opeinnuosi cortical in lobe. neurons temporal medial completion human pattern of 2018). August activation (17 bioRxiv:394999 holographic ensembles. by behavior Eng. Lang. Nat. vectors. random (2009). high-dimensional 139–159 with representation tributed (Morgan Eds. Reiter, R. Mylopoulos, 30–35. J. pp. (IJCAI), 1991), Kaufman, Intelligence Artificial on in Conference representations” distributed tional (2012). 927 cognition? of unit the as assembly (2018). cell 88–93 the adopt and detectors spiking of networks in 2016). concepts November to (11 roles arXiv:1611.03698 thematic neurons. of assignment the support jections structures. linguistic minimal of basis n400. speech. connected in structures linguistic opercularis. pars left the in investigation functional cortex. (2015). temporal mid-superior left 2017). Press, (MIT brain. (2016). human 13408 the 7, in web memory the underlying associations universal a.Rv Neurosci. Rev. Nat. tal. et 7–8 (2015). 170–181 18, iciso h Mind the of Circuits noigo ogtr soitostruhnua ntzto nthe in unitization neural through associations long-term of Encoding , 0–1 (2019). 503–517 25, agaei u ri:TeOiiso nqeyHmnCapacity Human Uniquely a of Origins The Brain: Our in Language opttoa Complexity Computational le,C .Ppdmtiu .Vmaa .Mas sebypro- Assembly Maass, W. Vempala, S. Papadimitriou, H. C. uller, ¨ 2–3 (2008). 920–933 9, a.Commun. Nat. Ofr nvriyPes 1994). Press, University (Oxford rc al cd c.U.S.A. Sci. Acad. Natl. Proc. eer Cortex Cerebr. EETas erlNt.Lan Syst. Learn. Netw. Neural Trans. IEEE rceig fte1t nentoa Joint International 12th the of Proceedings a.Neurosci. Nat. 32(2018). 4372 9, Wly 2003). (Wiley, rn.Psychol. Front. NSLts Articles Latest PNAS 1–2 (2017). 411–421 27, 5–6 (2016). 158–164 19, ersi Lett. Neurosci. 11732–11737 112, on Comput. Cogn. 88(2015). 1818 6, a.Commun. Nat. | 916– 23, f9 of 9 680, Nat. 1,

NEUROSCIENCE