Proc. Nati. Acad. Sci. USA Vol. 89, pp. 5774-5778, July 1992 Psychology On the definition of the concepts thinking, consciousness, and conscience (artificial intelligence/mind/cognltion/perception) ANDREI S. MONIN P. P. Shirshov Institute of Oceanology, Russian Academy of Sciences, 117218 Moscow, Commonwealth of Independent States Contributed by Andrei S. Monin, February 3, 1992
ABSTRACT A complex system (CS) is defined as a set of chess-playing computer: its data base necessarily contains a elements, with connections between them, singled out of the selection of standard openings, end-games, and multimove environment, capable of getting information from the environ- checkmates which constitute the chess semantics itself (if it ment, capable of making decisions (i.e., of choosing between exists at all). alternatives), and having purposefulness (i.e., an urge towards Generalizing, we go over to data base semantics in the preferable states or other goals). Thinking is a process that theory of games and in its numerous practically important takes place (or which can take place) in some of the CS and applications. Or, considering the most rigorous problems of consists of (i) receiving information from the environment (and thinking, it must be admitted that all the semantics of the from itself), (it) memorizing the information, (iii) the subcon- mathematical group theory and other abstract ("formal") scious, and (iv) consciousness. Life is a process that takes place mathematical constructions are created by using initial def- in some CS and consists of functions i and i, as well as (v) initions and axioms as the data bases and the rules of reproduction with passing of hereditary information to prog- mathematical logic as the programs to prove theorems. eny, and (vi) oriented energy and matter exchange with the However, the broadening ofthe definition of a computer is environment sufficient for the maintenance ofall life processes. not the principal aim of this paper: I am about to elaborate a Memory is a complex of processes of placing information in definition of the concept "thinking" suitable for a vast class memory banks, keeping it there, and producing it according to of systems (or "subjects") which includes particularly hu- prescriptions available in the system or to inquiries arising in mans and broadly defined computers as well-for the so- it. Consciousness is a process of realization by the thinking CS called "complex systems" [see, for instance, Fleishman (2)]. of some set of algorithms consisting of the comparison of its knowledge, intentions, decisions, and actions with reality- Complex Systems i.e., with accumulated and continuously received internal and external information. Conscience is a realization of an algo- DEFINITION 1. A complex system (CS) is a set ofelements, rithm of good and evil pattern recognition. with connections between them, singled out of the environ- ment, capable ofgetting information from the environment, The theory of artificial intelligence (AI) (and that of human capable of making decisions (i.e., of choosing between intelligence as well) lacks a constructive (that is, functional) alternatives), and having purposefulness (i.e., an urge to- definition of the concept "intelligence." This may be a wards preferable states or other goals). reason why the concept of Al is not widely accepted in Let me mention some of CS qualities which seem to be society, even among scientists (while, curiously, the concept necessary while considering possibilities of CS thinking. The of human intelligence even in the absence of definition is first ofthese is reliability or stability (R-quality) ofa given CS never considered as questionable). This of course restrains or perhaps of a larger CS which includes the given CS as one practical steps towards the realization and development ofAI of its subsystems. The R-quality may be provided for, in of higher and higher levels. particular, by CS structure (preprogrammed, for instance, by The proverbial question "Can a machine think?" obvi- means ofthe genetic code) or by its behavior, including, in the ously is meaningless until a definition of the concept "think- case of a biological CS, instincts or unconditioned reflexes ing" is given. The Alan Turing test-"if an expert cannot (i.e., aggregates of innate complex reactions or acts of distinguish the performance of a machine from that of a behavior arising as a rule in a constant form in response to human who has a certain cognitive ability, then the machine internal or external stimuli, such as defensive or self- also has that ability"-does not give a constructive defini- preservation instincts, feeding, sexual or reproductive in- tion. Substitution of a computer (defined as a machine for stincts, and parental and population-preservation instincts). manipulation of formal symbols) for a machine in general The second is noise-stability while receiving information enabled Searle (1) to conclude that computers, as defined from the environment, including the normal functioning of above, do not think (or, in other words, that "strong AI" is organs of sense (I-quality). The third is controllability (C- impossible) because computer programs contain only syntax quality), and the fourth is self-organization or self-learning while thinking is not limited by formal symbol manipula- (L-quality), including learning by being taught (which plays a tion-it needs semantics. major role in many biological CS). All four of these qualities However, there is no necessity to restrict ourselves to the are obviously feasible in cases of several nonbiological CS. narrow definition of a computer given above. For instance, a The structure ofa CS is understood usually as a graph, that computer usually has a data base ordered according to some is, a set of elements ("vertexes" or "nodes") and their pairs indications, and this ordering introduces a kind of semantics. (nonordered "ribs" and ordered "arcs") with adjacency and In Searle's favorite example of the "Chinese room" com- incidence relations between them. Complex structure is puter it includes a data base of numbered hieroglyphs that necessary for the provision of a CS with R-, I-, Q-, and actually contains some semantics in the form of interconnec- L-qualities defined above. For example the R-quality may be tions between hieroglyphs designating, say, "color" and "blue" or "green." Another example is presented by a Abbreviations: Al, artificial intelligence; CS, complex system(s). 5774 Downloaded by guest on September 23, 2021 Psychology: Monin Proc. Natl. Acad. Sci. USA 89 (1992) 5775 provided for by duplicating a function in different subsystems receiving informationfrom the environment (andfrom the CS in the same fashion as the reconstruction of the image of a itself), (ii) memorizing the information, (iii) the subcon- whole object imprinted on a hologram by any part of the scious, and (iv) consciousness. hologram, or doing without a damaged electrical line in a DEFINITION 3. Life is a process which takes place in some parallel electrical network. These analogies may have a literal of the CS and consists of functions i and ii above, (v) meaning while modeling neural networks of human brain or reproduction with passing ofhereditary information to prog- organs of sense. eny, and (vi) oriented energy and matter exchange with the A complexity of a set may be measured by its dimension- environment sufficient for the maintenance of all life pro- i.e., an exponent d in the power law N - Ed which expresses cesses. the minimum number N of spheres ofdiameter E covering the Here v appears to be the principal function or purport oflife set when E is small (but not too small; this means that the (there exist minor exceptions; for instance, some hybrids, power law is an intermediate asymptotic). If d exceeds the such as the mule, are undoubtedly alive but have no repro- usual (topological) dimension of the set, then the set is called duction abilities-this is an aimless life). afractal. There exist some grounds to suspect that for neural Function iv is obviously not necessary for life. The same networks d > 1-i.e., they are fractals. appears to be basically true for function iii ifinnate hereditary It appears, however, that the most complex structures are instincts are not included in this At the same time the random ones [for the theory of random graphs see, for concept. instance, Gilbert (3)]. They are not unusual (for example, function v is obviously not necessary for thinking, especially polycrystalline structures and ferromagnetic domains are for Al, at least if AI has no reproduction abilities [although widespread in nature and in engineering). The same is true for sexual reflections may play a significant role in the mental life the brain cellular microstructure. The formation of the brain of biological subjects, ascending from innate instincts, to structure on higher, supercellular, levels, including cytoar- subconscious motivations discovered by Sigmund Freud (4), chitectonics of cerebral cortex, is of course determined by and up to the idealized concept of personal love at the very the genetic program. However, the human genome (several summit known at present]. The same is true for function vi million genes) is very far from sufficient for encoding all the (the energy necessary for the maintenance of the thinking neurons (1010 to 1011), synapses (1014 to 1015), and their processes may be borrowed from some internal source, for combinations. The complexity of the neural network struc- instance from some quantity of a radioactive isotope). ture necessary for the thinking processes exceeds by many Thus thinking and life processes are by definition not orders of magnitude the possibilities of transmitting the interconnected: a thinking CS may be either alive or lifeless, genetic information. and a living CS may be either thinking or thoughtless. In the It is clear, therefore, that the cellular microstructure of second of these four cases the CS would constitute Al. different brain regions is being formed during the ontogenesis Let us also mention lifeless and thoughtless CS implement- processes not preprogrammedly but individually-i.e., in a ing prescribed complex programs, such as the program of sense, randomly (it appears that in these circumstances an exploration of the outer planets and satellites of the solar exhaustive genetic program of the system functioning, espe- system, carried out by CS "Voyager 1-Voyager 2." The aim cially on the cellular level, is hardly possible; therefore of this CS had been to collect several specified kinds of teaching and self-learning should play a great role in the information (including some current self-inspection informa- program-it reveals prospects for elaboration ofprograms for tion on its own orientation and internal state) and to transmit CS with more and more complex structure-i.e., with very it back to the Earth. The performance was not quite auto- large amounts of fast-acting elements and connections in matic; it was radio-controlled from the Earth. The system small volumes). might be organized closer to a thinking CS, were its perfor- The fortuity of formation of neurons and synapses in the mance more reliable. ontogenesis of an individual human brain determines the Another example is given by chess-playing "foreseeing" inimitability of its cellular structure and therefore the ob- computers, which have reached at present the level of world served scatter ofparameters and abilities of individuals. That championship and can exceed this level ifit is desirable. They is why human individuals are inimitable and every person is have shown that automatically going over all the possible priceless. Just this is the principal difference between human chessboard positions for a certain number of moves forward beings and modern computers, the structure of which, down plus a reasonable cost function of positions successfully to its smallest elements and connections between them, is replaces the incomplete memory and intuitive preferences of rigidly preprogrammed, so that all computers of the same a human player. This may mean that chess-playing itself in its series have an identical structure. A wide scatter of abilities pure form (if we disregard the partners' behavior and the of human individuals includes geniuses-very rare largest environment) is not a thinking process. favorable deviations of some ability from the norm situated, We have included function i (receiving information) into so to speak, on the tails of the Gaussian curves (occurrence the general definition as a necessary condition for thinking ofgeniuses of a certain ability may be estimated by the order processes (and for life processes as well). Information may be of magnitude as one per billion). This interpretation, in received by a CS by means of suitable sensors (in biological contrast to the mutation hypothesis, explains why genius is CS, organs of sense) or in recorded form (written, photo- not a hereditary virtue. The cloning of geniuses seems to be graphic, video- and audiotapes, etc). In the case of humans in principle beyond the possibilities of genetic engineering. there are five classical senses-sight (visible light or electro- Finally, let us discuss in this section the question of the magnetic waves with a length of 700-400 nm), hearing purposefulness of a CS. It may be formalized in particular as (acoustic waves with frequencies of 10-20,000 Hz), touch optimization-that is, an urge towards extreme values of (tactile, pressure, warm-cold, pain), smell, and taste (sour, some purpose functionals of the CS states. The multiformity salty, sweet, bitter, etc); in addition there exists a sense of of goals may lead to relative optimization in regard to some acceleration (vestibular apparatus) and also baro-, mechan- target or other and to a corresponding multitude of possible ical-, chemical-, thermo-, and osmotic interoreceptors sup- CS decisions. plying some information on the internal state of the human body. Organs of sense include primary receptors and encod- Thinking and Life ing devices transforming external influences into signals that are transmitted through suitable channels (nerves in the case DEFINITION 2. Thinking is a process which takes place (or of humans) to specific analysis in a central processor (the which can take place) in some of the CS and consists in (i) brain in the case of humans). Downloaded by guest on September 23, 2021 5776 Psychology: Monin Proc. Natl. Acad. Sci. USA 89 (1992) The ability to receive and store sensory information on the In literal translation from Russian, "consciousness" environment is very important for thinking systems. Just this means "coknowledge," that is, "existing together with gives semantics to thinking CS [but of course not the "spe- knowledge" [see Shreider (8)]. Consciousness algorithms are cific biochemical properties of brains," which mystically realized by means ofa working memory using current sensual "enable them to cause consciousness and other sorts of information and chronological and semantic data bases (while mental phenomena"; quotations from Searle (1)]. In some program libraries are used subconsciously). Thereby con- cases it is sufficient for thinking (and for consciousness) to sciousness transforms the available knowledge into the com- have even a very narrow channel of sensory information on municable form, which may be fixed or registered by output the environment. Thus, cases are known of blind, deaf, and devices and, therefore, transmitted to the outer world. dumb individuals who have been taught by using the sense of According to Kihlstrom (6), consciousness is not identical touch and the Braille alphabet and have reached rather high to any functions ofperception and cognition such as memory, levels of intellectual activities. It seems strange that Searle selective responses to stimuli, problem solving, etc.-all of denies that the computer receiving video images has seman- them may be performed subconsciously but may also be tics. I believe, on the contrary, that experiments with com- accompanied by consciousness. In other words, quality iv of puters of this kind (supplied with programs of pattern rec- Definition 2 is not necessary for most of the thinking pro- ognition and self-learning) would be very fruitful in Al cesses. But it is necessary for some of them, such as construction. communicating information of one's own states to the outer world, voluntary control, etc; that is why we have included Memory quality iv in Definition 2 (though in the case of Al the definition of a subconsciously thinking CS may be construc- DEFINITION 4. Memory is a complex of processes of tive). placing the information in memory banks, keeping it there, of available in the The very first consciousness algorithm consists singling and producing it according to prescriptions oneself out of the environment-i.e., in self-identification or system or to inquiries arising in it. work of William The contents of the memory store may be divided into data self-reference. According to the classical bases ["declarative knowledge", according to Anderson (5) James (9), the key to the consciousness is self-awareness or and Kihlstrom (6)] and program libraries ("procedural knowl- self-reference-i.e., the presence of the concept "I": the edge" in refs. 5 and 6). Data bases are further divided into universal conscious thought is not "the feelings exist" and chronological (with memorized circumstances of their acqui- "the thoughts exist" but "I think" and "I feel." This sition by the CS) and semantic ones (which represent the algorithm may conditionally be called "Soul" (understood "mental lexicon" of the abstract knowledge memorized differently than the religious concept of soul, which refers without obligatory connection to the circumstances of their only to living thinking CS or even only to humans, and implies acquisition). The program library contains prescriptions of a combination of life and mentality, the latter in a broader algorithms for the solution of some problem or for actions in sense than mere self-reference). some conditions. Reminder: An algorithm is a sequence of The algorithm of self-identification may be realized in transformations of discrete constructive objects (in the prac- particular by fixation of some characteristic or other of one's tically important case, words of some alphabet)-i.e., the own states at present or in the past, carrying out self- exact prescription defining the calculation process which inspection test programs prescribed earlier or formed in the starts from arbitrary initial data and is directed to getting the processes of learning or self-learning, continually or at some result determined by these data; the process is characterized specific moments of time prescribed or worked out by the by sets of possible initial data, intermediate and final results, system itself. The conscious system may nevertheless have and rules of beginning, direct transformation, ending, and a very incomplete and distorted view about itself (like an extraction of the result. illiterate human, often having no clear view about his or her Memory is also divided into short-term ("working") and anatomy, physiology, and psychology, for instance, not long-term memories [in the "parallel distributed processing" knowing up to now what is consciousness), but still it theory (7) there exist a number of working memories]. A identifies itself. working memory may apparently have a relatively limited For example, a computer may extract and give out to a volume and may contain a number of notions on the state of printer or to a display data on missteps which have happened the system and its aims at a given moment of time, and also in it during a specified period of time, with indication of their knowledge from the data bases and the program library, nature and causes. This may be done by means of a specif- activated by actions of the working memory itself or by ically prescribed program or following the prescription of external stimuli. A long-term memory may in principle be self-learning. The latter means, first, the need to form the practically limitless-i.e., may have room for all the infor- concept of a misstep as some definite class of computer's mation acquired by the system during its lifetime (this is actions, having adequate general and in their totality specific apparently so in the case of humans; in this case long-term features; second, to form algorithms of misstep pattern memory includes also innate-i.e., inherited, genetically recognition consisting of recognition operators and solving encoded programs; there is not much point in including into rules; third, to realize these algorithms according to some their number the programs of ontogenesis contained in the chosen time-table; fourth, to give out the results in some genes in all chromosomes of all cells, but there is every chosen form. Similarly a computer may realize self- reason to include instincts or unconditioned reflexes, defined inspection test programs or construct them for itself, follow- and listed above; in some lifeless thinking CS these instincts ing the prescription of self-learning. or a part of them may be absent. Self-identification (on a rather low level-i.e., highly spe- cialized) is probably performed by the chess-playing com- Consciousness puter: it receives external information (partner moves); it has working and long-term memories, a program library, includ- DEFINITION 5. Consciousness is a process ofrealization by ing the rules of the game, prescribed relative costs of chess the thinking CS of some set of algorithms consisting of the pieces, algorithms of standard openings and end-games, and comparison of its knowledge, intentions, decisions, and rules for estimating chessboard positions; and it is able to give actions with reality-i.e., with accumulated and continuously out all the information on its state. But of course it "does not received internal and external information. know" what things the chess pieces are; there is no need of Downloaded by guest on September 23, 2021 Psychology: Monin Proc. Natl. Acad. Sci. USA 89 (1992) 5777 this for the game. In all of this there is apparently no higher levels it may acquire practical value. In the future, difference from a human chess player, say, playing blindly. while constructing AI systems, the problem of introducing Self-learning programs appear to be especially important conscience into them should be taken seriously, as has for consciousness algorithms. Let us emphasize, however, already been suggested by I. Asimov in the form ofthe "laws that not only self-learning but also direct teaching of a given of robotechnics" (10): the Al should have the concepts of thinking CS by another CS plays a great role in the devel- good and evil, the latter including everything harmful for opment processes of any intelligence-for instance, in bio- humans (with some priorities in cases of alternatives). logical populations, teaching of offspring by parents and in a human society also by teachers and/or by computers. Teach- The Subconscious ing of computers by humans and/or by other computers at first glance appears to be trivial, but in principle it may help Most ofthe thinking processes are performed without the use in developing new educational methods in general. For of the working memory and consciousness. Let us try to example, attempts at teaching computers to translate texts classify them, basically following Kihlstrom (6) and keeping from one language into another have shown that some in mind the possibilities of their algorithmization. revisions of the rules of grammar are desirable. Then, of course, the automatic processes, using "proce- It appears that the concept "I"-i.e., the program of dural knowledge" (the program library), come first. They self-identification, may be worked out for any modern com- include in particular the acquisition of sensual information puter. The elaboration of alternative programs of this kind in (through sight, hearing, etc.) which goes on without any the near future may promote the development of new gen- intention on the part of the subject and without consuming erations of computers and their interactions with humans- any of one's attentional resources-i.e., unconsciously or i.e., the creation of human-computer civilization. Compila- nonconsciously in the strict sense ofthe word. Some of these tion of the more and more complete lists of consciousness automatic are are "hard- seems to have great prospects. processes innate-i.e., hereditarily algorithms wared" into the system-while others are acquired through experience, such as skill learning, repetition, routinization, Conscience rehearsal, and training. The tendency to liberate conscious- ness from repetitive routine actions may be the result of the DEFINITION 6. Conscience is a realization ofan algorithm of the of the good and evil pattern recognition. limited "power" (i.e., volume and speed of actions) It is one of consciousness algorithms. According to Shrei- working memory. der (8) conscience is a realization of the meaning of one's In the second place, let us mention the "preconscious" actions and the following moral responsibility for them. In processes such as subliminal perception and implicit mem- literal translation from Russian it is a "co-notice"-that is, ory, producing some latent data bases activated below the "existing together with notification" (from society); the threshold of conscious awareness, but still able to influence prefix "co-" emphasizes social conditionality of the crite- thinking processes and actions of the CS. An example of rion. Conscience is difficult to define for a single CS because subliminal perception is given by cinema advertisements it is not equivalent to the concepts of expediency, efficiency, using sequences too short (say, less than 5 msec) to be usefulness, reliability, etc. In human society it might be perceived consciously but still influencing purchasers' pref- different in various populations and in different historical erences. Other examples are the so-called Poctzel phenom- periods; let us note, for example, the social attitude towards enon (reappearance of subliminal stimuli in a subject's sub- cannibalism and concentration camps. sequent dreams) and recognition experiments of the type In the presence of the concepts of good and evil in the data stimulus word (prime)-random mask-target with timing pre- base of a CS (introduced there, for example, most probably venting conscious detection of the masked prime, which by teaching and possibly always adapted to some extent to nevertheless helped in detecting the target. the general structure and the volume of the data base of the Implicit memory is revealed by a change in actions attrib- given CS, including available instincts) a conscience may be utable to the information acquired during prior experience defined as an algorithm of good and evil pattern recognition which itself was not consciously registered. An example is in the actions performed by the system or analyzed as given by the so-called Korsakoffs syndrome, consisting in an possible. The results of the conscience algorithm may be anterograde amnesia-i.e., inability to recall events occur- realized by a system in very different ways (and not only ring since the onset of an illness due to the cut-off of the consciously; one can, for instance, perspire or blush with chronological memory, while all the semantic memory and shame). program library remain intact. It appears that the definition of conscience for a system The third class of subconscious processes is revealed by without consciousness is impossible. In contrast, conscious- the phenomena of hypnosis-alterations of the thinking pro- ness without conscience is possible (for instance, unfortu- cesses induced by another person. For example, hypnotic nately, in the case of some human individuals). analgesia consists in an induced loss of the conscious aware- The Holy Bible testifies that there was a time when ness of pain, while all the physiological reactions to pain everything was good and the consciousness of humans was (such as an increase in heart-beat rate) are present. This serene, and the only evil existing potentially was the eating phenomenon is somewhat similar to the catatonia syndrome, of the Tree of the Knowledge of Good and Evil, which was when a person stays conscious but does not react to pain or forbidden by the Lord God: if there is no evil, no conscience any other stimuli and does not move at all. Another example is necessary. But in the presence of evil the CS without is a posthypnotic amnesia, which is a selective disruption of conscience seem frightening, even ones like simple robots the conscious memory retrieval. and the simplest automatic machines. Because oftheir "soul- It appears that all these subconscious processes may be in lessness" they are usually perceived by the human con- principle algorithmized; it is quite obvious for the establish- sciousness negatively, cautiously, or sneeringly. This is the ment of selective thresholds of getting into the working root of the frightening fantasies on Frankenstein, Golem, memory and of putting off the consciousness according to Terminator, etc. and Chaplin's warning sequences with con- prescriptions introduced from without. Therefore, again, veyor lines and automatic machines. thinking and life are not interconnected. The elaboration of the simplest conditional "conscience" algorithms for any computer does not seem to be difficult. At The author thanks Dr. V. B. Kuznetsov for reading the text and Downloaded by guest on September 23, 2021 5778 Psychology: Monin Proc. Natl. Acad. Sci. USA 89 (1992)
Mrs. R. L. Leonova and Mrs. N. I. Solntseva for preparing the 5. Anderson, J. R. (1983) in The Architecture ofCognition (Har- manuscript. vard Univ. Press, Cambridge, MA). 6. Kihlstrom, J. F. (1987) Science 237, 1445-1452. 1. Searle, J. R. (1990) Sci. Am. (Intl. Ed.) 262 (1), 20-25. 7. Hinton, G. E. & Anderson, J. A., eds. (1981) in Parallel 2. Fleishman, B. S. (1982) in Fundamentals ofSystemology (Ra- Models ofAssociative Memory (ErIbaum, Hillsdale, NJ). dio and Communication, Moscow), pp. 17-54 (in Russian). 3. Gilbert, E. N. (1959) Ann. Math. Stat. 30, 1141-1144. 8. Shreider, Yu. (1989) The New World 11, 244 (in Russian). 4. Freud, S. (1966-1969) in Gesammelte Werke (G. Fischer, 9. James, W. (1890) in Principles ofPsychology (Holt, New York). Stuttgart, F.R.G.), Vols. 1-18. 10. Asimov, I. (1970) I, Robot (Fawcett, Greenwich, CT). Downloaded by guest on September 23, 2021