Knowledge Representation: How Far We Have Come? Daniel Khashabi University of Illinois, Urbana-Champaign
Total Page:16
File Type:pdf, Size:1020Kb
Knowledge Representation: How far we have come? Daniel Khashabi University of Illinois, Urbana-Champaign TechReport, Spring 2015 “Concepts are the glue that holds our mental world together”–Gregory Murphy Abstract ogy about how humans solve problems. Having multiple trends in interpreting the form of under- The theory of Knowledge Representation standing, comprehension and reasoning in the hu- has been one of the central focuses in Arti- man mind has created a diverse set of options for ficial Intelligence research over the past 60 knowledge representation. years which has seen numerous propos- In this work we give a relatively comprehensive als and debates ranging from very philo- review on the long-standing works on knowledge sophical and theoretical level to practical representation with emphasis on natural language issues. Although a lot of work has been systems. The review will start from relatively fun- done in this area, we believe currently, it damental issues on representation, and will con- does not receive enough attention from the tinue to exemplary practical advancements. Due research community, except in very lim- to the close ties between representation and rea- ited perspectives. The goal of this work soning, various reasoning frameworks have ap- is to give a unified overview of a set of peared; we briefly explore them in conjunction select works on the foundations of knowl- with representation techniques. edge representation and exemplar practi- Key terms: Before starting our main conversation cal advancements. we define the terminology we will be using in this summary. We start with proposition. 1 Introduction Proposition: Propositions are judgments or opin- Knowledge Representation (KR) is the subfield of ions which can be true or false. A proposition Artificial Intelligence (AI) devoted to formalizing is not necessarily a sentence, although a sentence information about the world in a form that com- can express a proposition. For example the sen- puter systems can use in order to solve complex tence that “Cats cannot fly” contains the proposi- tasks, such as a robot moving objects in an envi- tion that cats are not able to fly. Similarly this sen- ronment or answering a question in the form of tence “Cats are unable to fly” and this declarative natural language. The work in knowledge repre- phrase with logic can-fly(cats)=false sentation goes beyond creating formalism for in- convey the same proposition. A claim or a sen- formation, and includes issues like how to acquire, tence can contain multiple atomic propositions. encode and access it, which sometimes is called For example “Many think that cats can fly”. knowledge engineering. Knowledge acquisition is Concept: Propositions can be about any physi- is the process of identifying and capturing the rel- cal object (like a tree, bicycle, etc) and any ab- evant knowledge, according to the representation. stract idea (like happiness, thought, betrayal, etc), The discussion of knowledge representation has which all are (usually) called concepts. a long history in AI and ranges very fundamental Belief: Belief is an expression of faith and/or trust problems like, whether knowledge representation in a proposition, although a belief might not nec- is necessary for an AI system? to relatively practi- essarily be true. For example “Homer believed cal questions like whether classical logical repre- that the earth is flat” is an proposition which con- sentation is complete? Many of the ideas in repre- tains the belief of “earth” being “flat”, which is sentation stemmed from observations in psychol- not true. “The weather forecast predicts a tornado for tomorrow.” is another proposition which ex- Natural Input presses another belief which might turn out to be true or false. Cognition: Discussion of knowledge representa- Intermediate Input tion is closely related to the mental picture of the objects and concepts around us. Cognition is the mental action or process of acquiring knowledge AIComputer System Brain and understanding through thought, experience, and the senses. Although cognition involves ab- Knowledge Base Knowledge stract mental representations of concepts, it can Intermediate Output have an outer physical source (like observation, hearing, touching, etc), or inner source (like think- ing), or a combination of inner and outer sources. Natural Output Knowledge: Knowledge is information, facts, un- derstanding, and skills acquired through experi- Figure 1: A conventional structure of an AI sys- ence or education; the theoretical or practical un- tem. The “intermediate” representation is the mid- derstanding of a subject. The philosophical nature dleman between the reasoning engine and the ac- of knowledge has long been studied in Epistemol- tual input information. In addition to information ogy (Steup, 2014), where it has been been classi- in the input, for many problems reasoning engine fied into different categories based on its nature, demands knowledge beyond the problem defini- and its inherent connection to belief, truth, justifi- tions itself, which is provided by a knowledge cation, etc is analyzed. base. Therefore the discussion of representation Representation: To solve many problems there is can be about either the input information, or the a need to use some form of information which internal knowledge of the reasoning system. might be implicit in the problem definition or even not mentioned; therefore this knowledge must be provided to the solver (which can be a computer edge representation is necessary or not. In a lot of or any other machine) in order to be used when problems, since dealing with the raw input/output needed. In other words, representation is a surro- complicates the reasoning stage, historically re- gate for a concept, belief, knowledge or proposi- searchers have preferred to devise intermediate in- tions. For example, the number 5 could be rep- put/output to be middleman between the natural resented with the string “5”, or with bits 101, information and the reasoning engine (Figure 1). or Roman numeral “V”, etc. As part of design- Therefore the need for intermediate level seems ing a program to solve a problem, we must de- to be essential. In addition, in many problems fine how to represent the required knowledge. The there is a significant amount of knowledge which theory of Knowledge Representation is concerned either is not mentioned, or it is implicit. Somehow with the formalization of how to represent propo- the extra information needs to be provided to the sitions, beliefs, etc. A representation scheme de- reasoning system, which is usually modeled as a fines the form of the knowledge saved in a knowl- knowledge base. The issue of representations ap- edge base. Any representation is an abstraction plies to both the input level information and the of the world and its objects. The abstraction level internal knowledge of the reasoning system. of a representation might be too detailed, or too Such separation of knowledge from reasoning coarse with only high-level information. system is inherited to us from the very early AI Reasoning: Reasoning is the heart of AI, the de- projects and nowadays many accept it as the right cision making system for solving a problem or the way of modeling problems. Whether this is def- action of thinking about something in a logical initely the right way or not, there are debates; at and sensible way in order to form a conclusion. least this is the popular way of doing it. We refer Why Knowledge Representation?: It is not a to some of the relevant debates in the forthcoming trivial question to answer whether having knowl- sections. 2 2 A brief history of Knowledge 1978) was one of the first working expert sys- Representation tems for hypothesis formation and discovery of unknown organic molecules, using its knowledge In the early history of AI, there was big series of chemistry. of oppositions to the idea of “abstract represen- There were parallel trend inspired by neurons tation of thought” among psychology researchers. in the human (Rosenblatt, 1958). This movement Perhaps one of the earliest works that created a started by modeling mental or behavioral phenom- separate representation of the knowledge from the ena, emergent from interconnected networks of solver engine was the General Problem Solver some units. It lost many of its fans after Min- (GPS) system of Newell and Simon (1961). This sky and Papert (1969) showed fundamental lim- system was intended to be a universal problem itations of low layer networks in approximating solver which could be formalized in symbolic some functions. However, a series of following form, like chess playing or theorem proving. The events gave another energy to neurally inspired idea of symbolic reasoning was developed cen- models; Rumelhart et al. (1988a) found a formal- turies earlier by Descartes, Pascal, Leibniz and ized way to train networks with more than one some other pioneers in philosophy of mind. The layer. Funahashi (1989) showed the universal ap- use of symbols as the fundamental elements of proximation property for feedforward networks, the representation, or the physical symbol system i.e. any continuous function on the real num- hypothesis (Newell and Simon, 1961) is a debat- bers can be uniformly approximated by neural net- able assumption which has continued its way until works; Rumelhart et al. (1988b) emphasized on now. the parallel and distributed nature of processing, After the early scalablity issues of GPS, it soon which gave rise to the name “connectionism”. became clear that a system designed for solving Much progress has been made, but still many any formalized problem is impractical due to the problems are unsolved to a large extent. Some is- combinatorial explosion of intermediate states for sues are far more foundational in AI, like authors numerous problems. Also, it became apparent who have claimed that human-level reasoning is early that one of the main problems was how to not achievable via purely computational means; represent the knowledge needed to solve a prob- e.g.