Artificial Intelligence Illuminated Ben Coppin JONES AND BARTLETT PUBLISHERS World Headquarters Jones and Bartlett Publishers Jones and Bartlett Publishers Jones and Bartlett Publishers 40 Tall Pine Drive Canada International Sudbury, MA 01776 2406 Nikanna Road Barb House, Barb Mews 978-443-5000 Mississauga, ON L5C 2W6 London W6 7PA [email protected] CANADA UK www.jbpub.com Copyright © 2004 by Jones and Bartlett Publishers, Inc. Cover image © Photodisc Library of Congress Cataloging-in-Publication Data Coppin, Ben. Artificial intelligence illuminated / by Ben Coppin.--1st ed. p. cm. Includes bibliographical references and index. ISBN 0-7637-3230-3 1. Artificial intelligence. I. Title. Q335.C586 2004 006.3--dc22 2003020604 3382 All rights reserved. No part of the material protected by this copyright notice may be reproduced or utilized in any form, electronic or mechanical, including photocopying, recording, or any information storage or retrieval system, without written permission from the copyright owner. Acquisitions Editor: Stephen Solomon Production Manager: Amy Rose Marketing Manager: Matthew Bennett Editorial Assistant: Caroline Senay Manufacturing Buyer: Therese Bräuer Cover Design: Kristin E. Ohlin Text Design: Kristin E. Ohlin Composition: Northeast Compositors Technical Artist: George Nichols Printing and Binding: Malloy, Inc. Cover Printing: Malloy, Inc. Printed in the United States of America 08 07 06 05 04 10 9 8 7 6 5 4 3 2 1 For Erin This page intentionally left blank Preface Who Should Read This Book This book is intended for students of computer science at the college level, or students of other subjects that cover Artificial Intelligence. It also is intended to be an interesting and relevant introduction to the subject for other students or individuals who simply have an interest in the subject. The book assumes very little knowledge of computer science, but does assume some familiarity with basic concepts of algorithms and computer systems. Data structures such as trees, graphs, and stacks are explained briefly in this book, but if you do not already have some familiarity with these concepts, you should probably seek out a suitable book on algorithms or data structures. It would be an advantage to have some experience in a programming lan- guage such as C++ or Java, or one of the languages commonly used in Arti- ficial Intelligence research, such as PROLOG and LISP, but this experience is neither necessary nor assumed. Many of the chapters include practical exercises that require the reader to develop an algorithm or program in a programming language of his or her choice. Most readers should have no difficulty with these exercises. How- ever, if any reader does not have the necessary skills he or she simply should describe in words (or in pseudocode) how his or her programs work, giving as much detail as possible. vi Preface How to Read This Book This book can be read in several ways. Some readers will choose to read the chapters through in order from Chapter 1 through Chapter 21. Any chapter that uses material which is presented in another chapter gives a clear refer- ence to that chapter, and readers following the book from start to finish should not need to jump forward at any point, as the chapter dependencies tend to work in a forward direction. Another perfectly reasonable way to use this book is as a reference. When a reader needs to know more about a particular subject, he or she can pick up this book and select the appropriate chapter or chapters, and can be illumi- nated on the subject (at least, that is the author’s intent!) Chapter 12 contains a diagram that shows how the dependencies between chapters work (Section 12.6.2). This diagram shows, for example, that if a reader wants to read Chapter 8, it would be a good idea to already have read Chapter 7. This book is divided into six parts, each of which is further divided into a number of chapters. The chapters are laid out as follows: Part 1: Introduction to Artificial Intelligence Chapter 1: A Brief History of Artificial Intelligence Chapter 2: Uses and Limitations Chapter 3: Knowledge Representation Part 2: Search Chapter 4: Search Methodologies Chapter 5: Advanced Search Chapter 6: Game Playing Part 3: Logic Chapter 7: Propositional and Predicate Logic Chapter 8: Inference and Resolution for Problem Solving Chapter 9: Rules and Expert Systems Preface vii Part 4: Machine Learning Chapter 10: Introduction to Machine Learning Chapter 11: Neural Networks Chapter 12: Probabilistic Reasoning and Bayesian Belief Networks Chapter 13: Artificial Life: Learning through Emergent Behavior Chapter 14: Genetic Algorithms Part 5: Planning Chapter 15: Introduction to Planning Chapter 16: Planning Methods Part 6: Advanced Topics Chapter 17: Advanced Knowledge Representation Chapter 18: Fuzzy Reasoning Chapter 19: Intelligent Agents Chapter 20: Understanding Language Chapter 21: Machine Vision Each chapter includes an introduction that explains what the chapter cov- ers, a summary of the chapter, some exercises and review questions, and some suggestions for further reading. There is a complete bibliography at the back of the book. This book also has a glossary, which includes a brief definition of most of the important terms used in this book. When a new term is introduced in the text it is highlighted in bold, and most of these words are included in the glossary. The only such terms that are not included in the glossary are the ones that are defined in the text, but that are not used elsewhere in the book. The use of third person pronouns is always a contentious issue for authors of text books, and this author has chosen to use he and she interchangeably. In some cases the word “he” is used, and in other cases “she.” This is not intended to follow any particular pattern, or to make any representations about the genders, but simply is in the interests of balance. viii Preface The first few chapters of this book provide introductory material, explain- ing the nature of Artificial Intelligence and providing a historical back- ground, as well as describing some of the connections with other disciplines. Some readers will prefer to skip these chapters, but it is advis- able to at least glance through Chapter 3 to ensure that you are familiar with the concepts of that chapter, as they are vital to the understanding of most of the rest of the book. Acknowledgments Although I wrote this book single-handedly, it was not without help. I would like to thank, in chronological order, Frank Abelson; Neil Salkind and everyone at Studio B; Michael Stranz, Caroline Senay, Stephen Solomon, and Tracey Chapman at Jones & Bartlett; also a number of peo- ple who read chapters of the book: Martin Charlesworth, Patrick Coyle, Peter and Petra Farrell, Robert Kealey, Geoffrey Price, Nick Pycraft, Chris Swannack, Edwin Young, my parents, Tony and Frances, and of course Erin—better late than never. Thanks also to: The MIT Press for the excerpt from ‘Learning in Multiagent Systems’ by Sandip Sen and Gerhard Weiss, © 2001, The MIT Press. The MIT Press for the excerpt from ‘Adaptation in Natural and Artificial Systems’ by John H. Holland, © 1992, The MIT Press. The MIT Press for the excerpt from ‘The Artificial Life Roots of Artificial Intelligence’ by Luc Steels, © 1994, the Massachusetts Institute of Technol- ogy. The IEEE for the excerpt from ‘Steps Towards Artificial Intelligence’ by Marvin Minsky, © 2001, IEEE. I have attempted to contact the copyright holders of all copyrighted quotes used in this book. If I have used any quotes without permission, then this was inadvertent, and I apologize. I will take all measures possible to rectify the situation in future printings of the book. Contents Preface v PA R T 1 Introduction to Artificial Intelligence 1 Chapter 1 A Brief History of Artificial Intelligence 3 1.1 Introduction 3 1.2 What Is Artificial Intelligence? 4 1.3 Strong Methods and Weak Methods 5 1.4 From Aristotle to Babbage 6 1.5 Alan Turing and the 1950s 7 1.6 The 1960s to the 1990s 9 1.7 Philosophy 10 1.8 Linguistics 11 1.9 Human Psychology and Biology 12 1.10 All Programming Languages 12 1.10.1 PROLOG 13 1.10.2 LISP 14 1.11 Chapter Summary 15 1.12 Review Questions 16 1.13 Further Reading 17 x Contents Chapter 2 Uses and Limitations 19 2.1 Introduction 19 2.2 The Chinese Room 20 2.3 HAL—Fantasy or Reality? 21 2.4 AI in the 21st Century 23 2.5 Chapter Summary 24 2.6 Review Questions 24 2.7 Further Reading 25 Chapter 3 Knowledge Representation 27 3.1 Introduction 27 3.2 The Need for a Good Representation 28 3.3 Semantic Nets 29 3.4 Inheritance 31 3.5 Frames 32 3.5.1 Why Are Frames Useful? 34 3.5.2 Inheritance 34 3.5.3 Slots as Frames 35 3.5.4 Multiple Inheritance 36 3.5.5 Procedures 37 3.5.6 Demons 38 3.5.7 Implementation 38 3.5.8 Combining Frames with Rules 40 3.5.9 Representational Adequacy 40 3.6 Object-Oriented Programming 41 3.7 Search Spaces 42 3.8 Semantic Trees 44 3.9 Search Trees 46 3.9.1 Example 1: Missionaries and Cannibals 47 3.9.2 Improving the Representation 49 3.9.3 Example 2: The Traveling Salesman 50 3.9.4 Example 3: The Towers of Hanoi 54 3.9.5 Example 4: Describe and Match 56 3.10 Combinatorial Explosion 57 3.11 Problem Reduction 57 Contents xi 3.12 Goal Trees 58 3.12.1 Top Down or Bottom Up? 60 3.12.2 Uses of Goal Trees 61 Example 1: Map Coloring Example 2: Proving Theorems Example 3: Parsing Sentences 63 Example 4: Games 3.13 Chapter Summary 64 3.14 Review Questions 65 3.15 Exercises 65 3.16 Further Reading 66 PA R T 2 Search 69 Chapter 4 Search Methodologies