DOCUMENT RESUME'. ED 150 955 IR 005 492 AUTHOR Lehnert, Wendy TITLE The Process'of Question Answering. Research Report No. 88. ..t. SPONS AGENCY Advanced Research Projects Agency (DOD), Washington, D.C. _ PUB DATE May 77 CONTRACT ,N00014-75-C-1111 . ° NOTE, 293p.;- Ph.D. Dissertation, Yale University 'ERRS' PRICE NF -$0.83 1C- $15.39 Plus Post'age. DESCRIPTORS .*Computer Programs; Computers; *'conceptual Schemes; *Information Processing; *Language Classification; *Models; Prpgrai Descriptions IDENTIFIERS *QUALM; *QuOion AnsweringSystems . \ ABSTRACT / The cOmputationAl model of question answering proposed by a.lamputer program,,QUALM, is a theory of conceptual information processing based 'bon models of, human memory organization. It has been developed from the perspective of' natural language processing in conjunction with story understanding systems. The p,ocesses in QUALM are divided into four phases:(1) conceptual categorization; (2) inferential analysis;(3) content specification; and (4) 'retrieval heuristict. QUALM providea concrete criterion for judging the strengths and weaknesses'of store representations.As a theoretical model, QUALM is intended to describ general question answerinlg, where question antiering is viewed as aerbal communicb.tion. device betieen people.(Author/KP) A. 1 *********************************************************************** Reproductions supplied'by EDRS are the best that can be made' * from. the original document. ********f******************************************,******************* 1, This work-was presented to thd Graduate School of Yale University in candidacy for the'degree o.CDoctOr of Philosophy.' U.S. DEPARTMENT OF HE ALTH. EDUCATION & WELFARE NATIONAL IMSTITUTIE OF EDUCATION THIS DOCUMENT HASBEEN REPRO. OUCED EXACTLY ASRECEIVED Fnom THE PERSON OR ORGANIZATIONORIGIN. ATING tT POINTS OF VIEWCR OPINIONS STATEO 00 NOT NECESSARILYREPRE INSTITUTE OF SENT OFFKIAL NATIONAL EDUCATION POSITION ORPOLICY THE PROCESS OF QUESTION ANSWERING May 1977 Research Report #88 Wendy Lehnert This work was supported in part by the Adiranced ReSearch Projects Agency, of the Department of Defenseland monitored under the Office of Naval Research under contract Np0014-,v-c-1111. 2 o ABSTRACT Problems in computational question answering assume *a new perspectivewhen question answering is viewed as a problem in natural language processing.A theory of question answering has been proposed from this viewpointwhich relies on ideas in conceptual information processing and theories of human memory organization. This theory of question answering has been implemented in a computer program, QUALM. QUALM-is currently used by two story understanding systems (SAM and PAM) to complete a natural language processing system which needs stories and answers questions about whatwas read. - -W IL The processes in QUALMare divided into four phases: (1) ConceptualCategoriiation, 42) ,Inferential Analysis, (3)- Content Specification, and (4) 1- Retrieval Heuristics. Conceptual categorizationl guides subsequent processing by dictating which . specific inference mechanismS, and memory retrieval strategies should be invoked in the course' ofanswering a question. Inferential ,Aalysis is responsible for understanding wheat the questioner really meantwhen a question should not be takenliterally. Content \ Specification determines how much of an answer should be returned in terms of detail and elaborations.Retrieval Heuristics do the actual digging in order to extract an answer frommemory% All of the inferenceprocesses within these four phasesare 'independent of language, operating within conceptual representations. QUALMrepresentsa; theory of questionanswering which is motivated by theories Ofnatural language processing. Within the contextif story understanding, 'QUALM has provided a concrete criterion for judging the strengths and weaknesses, of story representations generated by SAM and PAM.If a systemunderstands a story, it should be able to answer questions about that story in the same sway that people do. Although the computer -implementation of QUALM is currently limited to the application of answering questions about stories, the theoreticalmodel goes beyond this particular, context. As a theoretical model QUALM is intended to describe general question answering, witcre question .answering in its most generalform is viewed as a verbal communication device between people. , c 3 "PREFACE When aperson understands astory, he can demonstrate his understanding by answering questions about the story. Since questions can be devised to query any aspect of text-cOmprehension, theability to answer questions is the strongest possibledemonstration understanding. of Question answering istherefore a task criterion for evaluating reading' skills. If a computer issaid to understand a story,wemust demand of thecompUter the same demonstrations of understanding thatwe require of people. such demands are met,we have no way of evaluating text understanding programs. Any computerprogrammer can write a Trogram Which /inputs text. If the programmer assures us thathis program 'understands' teNt, it is a bit like being !reassured bya used car salesman about a susAciously low speedometer reading. Only when wecan ask a program to.answer questions about what it reads willwe be able to begin to assessthat program's comprehension. r a 4 1 ACKNOWLEDGEMENTS 'I would like to thank my 'thesis advisor, Professor RogerSchenk, whosecreativity and teaching style have transformed the...problems natural language processing dnto intensely challenging and gratifying researchefforts. He taught me to conduct research with a delicate balance of disciplined restraint and inspired atandor.. But lest we get tpo carried away, it also helps to be slightly manic; d to have a , big- computer. 0 Special thanks go to ,Professor Robert Abelson, social', psychologist, member of the Yale A. I. Project, and an endless,source of ideas that tend to be about ten years ahead of their time. A. Dr. ChristopherRiesbeck's ideas and intuitions havebeen another major ,and valuable influence.gis remarkable sense of humor 4 counterbalances the pressures 'of impossibledeadlines and other bothersomedistractions for everyone who has the pleasure of working with him. The preparation of this manuscript was greatly aided by Dr. Drew McDermott who read draft version 2.5 ofthis own free will. I am grateful for hismany extensive and thoughtful comments which faCilitated the final rewriting (polishing not quite the word) of this thesis. I want to thank ProfeestirAlan Perlis forbring on myreading commitee alang with Professors Robert Abelson and Roger Schenk. j I would also like to acknowledge the cordial efforts of Dr. Daniel Bobrow who invited me to spend a summer at Xerox PARC while he and Terry Winograd collaborated on KRL. While we did.not always agree on issues in natural language processing, our arguments were often enlightening, and the opportunity to experienceKRL first hand was both stimulating and instructive.,One of the computer programs (COIL) described in this thesis was written in an experimental 'implementation of KRL during my stay at Xerox. 1 The other members of the Understander Groupat 'XetOk-- PARC all contributed to makingmy stay atXerox,a pleasant land. pioductive .experience: Ronald Kaplan, 'Martin Kay, David Levy, Paul' Martin, and henryThompson. Special thanks go to David Levy,l'aul Martin, and Henry Thompson for patiently, helping me unravel the mysteries of INTERLISP and KRL. 0 .I am especially grateful to all the past and present student membersof theYale A.r. Project 'who.4ave made beinga, graduate student much more fun than it's supposed to be: Jaime Carbonell, /Richard Cullingford, GeraldDeJong, Anatole Gershman, Richard Granger, Leila Habib, James Meehan, RichardProudfoot, Mallory Selfridge, Walter Stutzman,, and Robert Wiiensky. Finally, Imust thank my. parents who have held uprather admirably throughout all my successive childhoods. And a very special word of appreciation goes to my htisband, Richard Goldstein, for convincing me that two free spirits,,are better than one. This work was supported in part by the Advanced Research ProjeCts Agencyof the DepartmentofDefense and monitored under the Office of Naval Research under contract N00014- 75 -C -11 t. 6 5 A .4) TABLE OF CONTENTS ABSTRACT PREFACE. ii ACKNOWLEDGEMENTS A. ill' TABLE OF VATENTS' v CHAPTER 1: PROBLEMS, PREVIEWS AND PROGRAMS 1.0 Introduction , 1 1.1 What's Hard About Answering Questions 1 1.1.1 Conceptual Analysis; 1 1.1.2 Refetence Recognition 2 1.1.3 Conceptual Categorization 3 1.1.4 Understanding in Context 3 1.1.5 Conversational Interpretation . 4 4' 1.1.6 Inferences 5 1.1.7 Knowledge-Based Interpretation . .. 6 1.1.8 Memory -Based Interpretation 7 1.1.9 Se4pting the Best Answer 8 1.1.10 State bescriptions 9 1.1.11 Content Specification . A 11. 1.1.12 Accessing Memory 12 1.1.13 Constructing Information 13 1.1.14 Finding Only What's Needed 14 1.2 Where We're Going '-\ 15 1.2.1 How to Survive the Trip 17 1.3 QUALM 17 1.4 Computer Programs Using QUALM . 18 1.4.1 SAM 10 . 1.4.2 PAM 25 1.4.3 ASP 26 1.4.4 COIL ' 28 'PREFACE TO INTERPRETATION: UNDERSTANDING QUESTIONS O. Introduction 30 _ 1. Four Levels of Understanding 30 2. The Conceptual Parse 31 3. Memory Internalization 32 4. Conceptual Categorization 34 5. Inferential Analysis 34 . -vi- -v CHAPTER 2: CONCEPTUAL; CATEGORIES FOR QUESTIONS I Introduction 37 2.1 Causal Antecedent 42 2.2 Goal Orientaelon 44 2.3 Enablement 46 2.4 Causal Consequent' 49 2.5 Verification.