Frameworks for Intelligent Systems
CompSci 765 Meeting 3
Pat Langley Department of Computer Science University of Auckland Outline of the Lecture
• Computer science as an empirical discipline • Physical symbol systems • List structures and list processing • Reasoning and intelligence • Intelligence and search • Knowledge and intelligence • Implications for social cognition
2 Computer Science as an Empirical Discipline!
In their Turing Award article, Newell and Simon (1976) make some important claims: • Computer science is an empirical discipline, rather than a branch of mathematics. • It is a science of the artificial, in that it constructs artifacts of sufficient complexity that formal analysis is not tractable. • We must study these computational artifacts as if they were natural systems, forming hypotheses and collecting evidence. They propose two hypotheses based on their founding work in list processing and artificial intelligence.!
3 Laws of Qualitative Structure!
The authors introduce the idea of laws of qualitative structure, which are crucial for any scientific field’s development: • The cell doctrine in biology • Plate tectonics in geology • The germ theory of disease • The atomic theory of matter They propose two such laws, one related to mental structures and another and the other to mental processes.
4 Physical Symbol Systems!
Newell and Simon’s first claim, the physical symbol system hypothesis, states that: • A physical symbol system has the necessary and sufficient means for general intelligent action. They emphasize general cognitive abilities, such as humans exhibit, rather than specialized ones. This is a theoretical claim that is subject to empirical tests, but the evidence to date generally supports it.!
5 More on Physical Symbol Systems!
What do Newell and Simon mean by a physical symbol system? • Symbols are physical patterns that are stable unless modified. • Symbol structures or expressions are organized sets of symbols. • A physical symbol system create, modifies, copies, and destroys symbol structures in order to: • Maintain structures that designate other objects or processes; • Interpret expressions that designate such processes. These ideas are agnostic about the nature of physical patterns; they can reside in neurons, in silicon chips, or on paper.!
6 Development of the Hypothesis!
Four historical developments during the 20th Century led to the physical symbol system hypothesis: • Studies in formal logic and symbol manipulation • Turing machines and digital computers • The concept of a stored program • List processing and languages like IPL and Lisp Later work on computer systems that designated and interpreted symbol structures built on these advances.!
7 What is List Processing?!
Newell, Shaw, and Simon introduced list processing in 1956; this paradigm involved three key ideas: • Dynamic memory structures and mechanisms to alter them • Data types and operations for different types • Designation and manipulation of symbol structures These support abstraction of structures and processes beyond the specific hardware on which they are implemented. They demonstrated these ideas in IPL, the first list-processing language, although Lisp soon became more widely used.!
8 Why is List Processing Important?!
This new framework was very important to AI’s development because it could: • Encode arbitrarily complex structural descriptions; • Create new structural descriptions dynamically; • Use such structures to designate other structures; and • Interpret these structures to produce behavior. As we will see, each of these abilities plays a crucial role in the construction of intelligent systems.!
9 Lists and List Structures!
List-processing techniques can encode very complex structural descriptions using: • Symbols such as on, A, and B ; • Lists such as (on A B) and (eats agent John object soup) ; • List structures such as (goal me (not (on ?any B))) . We can use such structures to represent an agent’s beliefs and goals, as well as its rules and other forms of knowledge. Most classical AI systems have depended on list structures to encode such symbolic content.
10 Three Metaphors for Intelligence
Most research on AI adopts one of three distinct metaphors for understanding the mind:
• Intelligence as multi-step reasoning • Intelligence as search through a maze • Intelligence as retrieval of knowledge from memory None of these metaphors is right or wrong, but each one offers important insights into the nature of mental processing.
11 Metaphor 1: Reasoning and Intelligence
One metaphor for intelligence emphasizes the human ability to carry out complex reasoning, including:
• Deduction, which draws logical conclusions from givens; • Abduction, which finds plausible accounts of observations; • Analogy, which maps new situations onto known ones. AI researchers have developed systems that exhibit each of these cognitive capabilities. Despite differences in operation, they all operate over structured, relational encodings, stated as list structures.
Some of the earliest AI research dealt with automated reasoning:
• Newell, Shaw, Simon’s (1957) Logic Theorist: The first AI system, based on studies of human reasoning, proved theorems in logic; Gelernter’s (1959) Geometry Theorem Machine: Proved geometry • theorems with diagrams, some requiring constructions;
• Slagle’s (1963) SAINT: Solved final exam problems in symbolic integration from a first-year Calculus course. Automated reasoning methods are now mature enough to appear in software like Maple and to be used in hardware verification. Propagation of values in Excel is an even more widespread use.
13 SHRDLU (1970)
SHRDLU was an early AI system that Person: Pick up a big red block. Computer: OK. interacted with users in natural language. Person: Grasp the pyramid. It inferred sentence meanings to answer Computer: I don't understand which pyramid you mean. queries and executed complex commands. Person (changing their mind): Find a block which is taller than the one you are Although limited in scope, SHRDLU had holding and put it into the box. many features of a cognitive system. Computer: By "it", I assume you mean the block which is taller than the one I am holding. Computer: OK. Person: What does the box contain? Computer: The blue pyramid and the blue block. Person: What is the pyramid supported by? Computer: The box. Person: How many blocks are not in the box? Computer: Four of them. Person: Is at least one of them narrower than the one which I told you to pick up? Computer: Yes, the red cube. 14 Metaphor 2: Intelligence and Search
A second metaphor for intelligence rests on an analogy with finding one’s way through a physical maze.
This view is commonly adopted for problem solving tasks. These require one to find some sequences of steps that solves an unfamiliar problem. Instances include generating new plans, schedules, and designs.
The search view is not antithetical to the reasoning metaphor, but they emphasize different aspects of intelligence.
15 Problem Solving and Symbol Systems!
Human intelligence includes the ability to solve novel problems. But how can we find solutions to problems when we do not already know the answers? • This is the Meno Paradox that Plato first posed. Fortunately, we can separate generators for candidate solutions from mechanisms for testing them. • This division eliminates the apparent paradox. But it requires the ability to represent candidate solutions and to search through the resulting problem space.!
16 The Heuristic Search Hypothesis!
This insight led Newell and Simon to propose their heuristic search hypothesis: • A problem solver represents candidate solutions in terms of symbol structures; • Problem solving involves a search process that generates and modifies these structures; • The problem solver tests candidates to determine whether they are acceptable. This process relies on heuristics because, in practice, one cannot search most problem spaces exhaustively.
17 Search and the Tower of Hanoi
Consider a puzzle that is known as the Tower of Hanoi, which involves three pegs and N disks.
The task is to move all disks from the initial peg to another peg, which involves problem solving and search. 18 Problem Space for Tower of Hanoi
Starting from a single Initial state state, we can apply operators repeatedly to generate the entire problem space, or we can explore it more selectively.
Goal state
19 The Logic Theorist!
In 1956, Newell, Shaw, and Simon developed the Logic Theory Machine, the first running AI program. • This system proved theorems in propositional logic, finding different proofs from those of Russell and Whitehead. • The Logic Theorist demonstrated the use of list processing, heuristic search, and goal-directed reasoning. • This was first of an entirely new breed of computer programs. It would difficult to overestimate the system’s impact on the new field of artificial intelligence.!
20 AI Planning Systems
One important use of search mechanisms is to generate a plan for achieving some goal. AI planning methods have become mature enough to support a variety of applied systems, including: • Automobile navigation systems that generate routes to follow; • Orbitz and other travel sites, which propose airline itineraries; • DART, which generated logistical plans for the US military; • The Hubble space telescope, Mars rovers, and even copiers. These systems are much less flexible than human planners, but they find solutions that people might overlook.
21 Game-Playing Programs
Early work on games like chess, one of AI’s original challenge problems, led to many insights about representation and search.
• In 1957, Herbert Simon predicted that “within ten years, a computer will be the world’s chess champion”. • Simon’s prediction was too optimistic, but only his timing was off.
• In 1997, Deep Blue, a hardware-aided chess program that searched deeper than humans, beat Gary Kasparov, the human champion. Similar results have occurred for checkers, backgammon, and other complex games that humans find challenging. However, these systems are tuned for specific games and lack the generalized abilities of human players.
22 Metaphor 3: Knowledge and Intelligence
A third metaphor for intelligence emphasizes people’s ability to draw upon large amounts of knowledge in memory about: • The structure and behavior of physical environments; • Topics from science, engineering, arts, and humanities; and • Relations and events that arise in social interactions. AI researchers have developed systems that store, retrieve, and use knowledge in each of these areas. This paradigm led to the first widespread applications of AI technology in commerce and industry.
23 Expert Systems
The expert systems movement built upon insights about the role of knowledge in human expertise. This paradigm encodes knowledge as if-then rules or as stored cases that are matched to draw conclusions. • Hundreds of such expert systems have been deployed since the 1980s, making or saving substantial money. However, they are costly to maintain as conditions change, and • their reasoning is often routine and shallow.
Successes of machine learning and data mining are best viewed as the automated construction of expert systems.
24 The TurboTax System
One widely adopted expert system is TurboTax, which helps its users prepare their US tax returns.
TurboTax has been purchased by millions of customers, reducing their time and effort to file taxes. 25 Case-Based Systems
An alternative approach to mimicking expert behavior involves case-based systems that: • Store / index specific instances or cases in long-term memory. • Upon encountering some new problem, retrieve a similar case. • Use the retrieved case to draw inferences or propose a response. • Adapt this case to the new situation if needed (optional). Case-based systems are arguably easier to construct, understand, and maintain than rule-based expert systems. They have also been quite successful in commercial settings, including interactive ones like help desks.
26 Recommender Systems
In the 1990s, increasing availability of Web content, including products on the Web, led to recommender systems that:
• Propose, rank, or otherwise present selected items that they predict will interest the user. • Utilize profiles for users that describe their preferences, which are often learned from explicit or implicit feedback.
• Amazon, Tivo, and many other companies use techniques of this sort to increase sales and customer satisfaction. Recommender systems have shallow user models and limited interaction abilities, but they are used widely.
27 Summary Remarks!
We have reviewed some broad claims that appear relevant to interactive cognitive systems: 1. Computer science and AI are empirical disciplines that study the behavior of computational artifacts. 2. Physical symbol systems are necessary and sufficient for intelligent behavior (physical symbol system hypothesis). 3. Problem solving involves heuristic search in a space of states encoded as symbol structures (heuristic search hypothesis). 4. AI has explored two other main metaphors for intelligence: as reasoning and as expertise. These insights and frameworks have become so basic that many AI researchers do not realize they were once unknown.
28 Implications for Social Cognition!
How are these ideas relevant to the social cognition hypothesis proposed earlier? They suggest that we should: • Develop computational artifacts to understand the social character of intelligence; • Represent our agents’ beliefs, goals, and knowledge using symbolic structures; • View social understanding and planning in terms of heuristic search, reasoning, and knowledge use. However, interaction raises new issues; e.g., in social planning, should problem states include models of others’ beliefs / goals?
29 Reading Assignments
We will have two reading assignments for the next meeting: • Young, R. M. (2001). Production systems in cognitive psychology. In N. J. Smelser & P. B. Baltes (Eds.), International Encyclopedia of the Social and Behavioral Sciences. Pergamon. • Jones, G., & Ritter, F. E. (2003). Production systems and rule- based inference. Encyclopedia of Cognitive Science. London: Macmillan. These papers provide brief overviews of production systems, an important class of formalisms for high-level cognition. Please read both papers and come prepared to discuss them.
30 End of Lecture!
31