University of Pennsylvania ScholarlyCommons
IRCS Technical Reports Series Institute for Research in Cognitive Science
October 1993
A Computational Model of Syntactic Processing: Ambiguity Resolution from Interpretation
Michael Niv University of Pennsylvania
Follow this and additional works at: https://repository.upenn.edu/ircs_reports
Niv, Michael, "A Computational Model of Syntactic Processing: Ambiguity Resolution from Interpretation" (1993). IRCS Technical Reports Series. 184. https://repository.upenn.edu/ircs_reports/184
University of Pennsylvania Institute for Research in Cognitive Science Technical Report No. IRCS-93-27.
This paper is posted at ScholarlyCommons. https://repository.upenn.edu/ircs_reports/184 For more information, please contact [email protected]. A Computational Model of Syntactic Processing: Ambiguity Resolution from Interpretation
Abstract Syntactic ambiguity abounds in natural language, yet humans have no diffculty coping with it. In fact, the process of ambiguity resolution is almost always unconscious. But it is not infallible, however, as example 1 demonstrates. 1. The horse raced past the barn fell.
This sentence is perfectly grammatical, as is evident when it appears in the following context:
2. Two horses were being shown to to a prospective buyer. One was raced past a meadow and the other was raced past a barn.
Grammatical yet unprocessable sentences such as 1. are called 'garden-path sentences.' Their existence provides an opportunity to investigate the human sentence processing mechanism by studying how and when it fails. The aim of this thesis is to construct a computational model of language understanding which can predict processing difficulty. The data to be modeled are known examples of garden path and non-garden path sentences, and other results from psycholinguistics.
It is widely believed that there are two distinct loci of computation in sentence processing: syntactic parsing and semantic interpretation. One longstanding controversy is which of these two modules bears responsibility for the immediate resolution of ambiguity. My claim is that it is the latter, and that the syntactic processing module is a very simple device which blindly and faithfully constructs all possible analyses for the sentence up to the current point of processing. The interpretive module serves as a filter, occasionally discarding certain of these analyses which it deems less appropriate for the ongoing discourse than their competitors.
This document is divided into three parts. The first is introductory, and reviews a selection of proposals from the sentence processing literature. The second part explores a body of data which has been adduced in support of a theory of structural preferences - one that is inconsistent with the present claim. I show how the current proposal can be specified ot account for the available data, and moreover to predict where structural preference theories will go wrong. The third part is a theoretical investigation of how well the proposed architecture can be realized using current conceptions of linguistic competence. In it, I present a parsing algorithm and a meaning-based ambiguity resolution method.
Comments University of Pennsylvania Institute for Research in Cognitive Science Technical Report No. IRCS-93-27.
This thesis or dissertation is available at ScholarlyCommons: https://repository.upenn.edu/ircs_reports/184 The Institute For Research In Cognitive Science
A Computational Model of Syntactic Processing: Ambiguity Resolution from Interpretation P (Ph.D. Dissertation)
by E Michael Niv
University of Pennsylvania Philadelphia, PA 19104-6228 N October 1993
Site of the NSF Science and Technology Center for Research in Cognitive Science N
University of Pennsylvania IRCS Report 93-27 Founded by Benjamin Franklin in 1740
A Computational Model of Syntactic Processing
Ambiguity Resolution from Interpretation
Michael Niv
A Dissertation
in
Computer and Information Science
Presented to the Faculties of the UniversityofPennsylvania in Partial Ful llmentofthe
Requirements for the Degree of Do ctor of Philosophy
Mark J Steedman
Sup ervisor of Dissertation
Mark J Steedman Graduate Group Chairp erson
c
Copyright
Michael Niv
Acknowledgements
My deep est feelings of gratitude and indebtedness are to my advisor and mentor Mark Steedman
Mark has taught me not just linguistics and cognition but also ab out thinking and b ehaving like
a scientist He tirelessly and carefully read through each of the many many drafts of pap ers I
have given him including this thesis providing meticulous and insightful comments and sp ent
countless hours explaining and debating these comments with me Iwould guess that every
single p oint I discuss in this thesis re ects his contributions
This thesis like me is a pro duct of a community The Cognitive Science communityatPenn
brings forth a b eautifully dissonant buzz of intellectual activity The diversity of outlo oks
paradigms metho ds results and opinions ab out the study of the mind provides an ideal en
vironment in which to go shopping for one s own direction It is imp ossible to identify the
individual memb ers and visitors to IRCS Institute for Research in Cognitive Science that have
shap ed my p ersp ective so I ll just thank the two p eople who are resp onsible for enabling the
tremendous growth of IRCS during mystayatPenn Aravind Joshi and Lila Gleitman
Iamvery grateful to my do ctoral committee Janet Fodor Aravind Joshi Mitch Marcus and
Ellen Prince for their helpful comments on this do cument in particular to Ellen and Janet for
long and fruitful discussions ab out the rst parts of the thesis and to Mitch for making the
Penn Treebank available to me
My colleagues Breck Baldwin Barbara Di Eugenio Bob Frank Dan Hardt Jamie Henderson
oung Suk Lee Owen Rambow Phil Resnik Rob ert Rubino Lyn Walker Beth Ann Ho ckey Y
and many other memb ers of the Computational Linguistics group at Penn provided me with
much help supp ort and friendship throughout my graduate studies IRCS p ostdo cs have b een
particularly helpful I have learned a great deal from Sandeep Prasada Je Siskind and Mark
Hepple
This work has b ene tted greatly from suggestions and advice by Ellen Bard Julie Boland Lyn
Frazier Susan Garnsey Mark Johnson Rob ert Ladd Don Mitchell Dick Oehrle Stu Shieb er
Val Tannen Henry Thompson AmyWeinb erg Steve Whittaker and Bill Wo o ds Rich Pito
was very helpful in extending his treebank searching program tgrep to accommo date my needs
I am grateful to Bonnie Webb er for o erring me my rst opp ortunity to do research the
TraumAID pro ject and for her nancial supp ort during my rst few years at Penn The
research rep orted in this do cumentwas supp orted by the following grants DARPA N
J ARODAAL C NSF IRI Ben Franklin S C I thank the
Computational Linguistics facultymemb ers for their sustained nancial supp ort My thanks
also go to Jow and Oyi Ali Baba Yue Kee and the Kims for sustenance iii
Iamvery grateful to Barbara Ramesh Bob Sandeep and to the o cial memb ers of the late
nightWawa crew Patrick Anuj Tilman esp ecially to Young Suk for the b eginnings of life long
friendships
I ve had a few really wonderful teachers Ruta in kindergarten Mme O Connor for Frenchin
high scho ol Jo e O Rourke in college and Val Tannen in graduate scho ol They each p ossess
the rare and precious talent of sparking curiosity and intellectual excitement in their students
My most imp ortant teachers havebeenmy parents Ya a and Avigdor I thank them and my
two sisters Adi and Tamar for encouraging my curiosity and coming to terms with just how
curious I have b ecome iv
Abstract
A Computational Mo del of Syntactic Pro cessing
Ambiguity Resolution from Interpretation
Michael Niv
Mark J Steedman Sup ervisor
Syntactic ambiguity ab ounds in natural language yet humans have no di culty coping with it
In fact the pro cess of ambiguity resolution is almost always unconscious But it is not infallible
however as example demonstrates
The horse raced past the barn fell
This sentence is p erfectly grammatical as is evident when it app ears in the following context
Two horses were b eing shown o to a prosp ective buyer One was raced past a meadow
and the other was raced past a barn
Grammatical yet unpro cessable sentences such as are called garden path sentences Their
existence provides an opp ortunitytoinvestigate the human sentence pro cessing mechanism by
studying how and when it fails The aim of this thesis is to construct a computational mo del
of language understanding which can predict pro cessing di culty The data to b e mo deled
are known examples of garden path and non garden path sentences and other results from
psycholinguistics
It is widely b elieved that there are two distinct lo ci of computation in sentence pro cessing
syntactic parsing and semantic interpretation One longstanding controversy is which of these
two mo dules b ears resp onsibility for the immediate resolution of ambiguity My claim is that it
is the latter and that the syntactic pro cessing mo dule is a very simple device which blindly and
faithfully constructs all p ossible analyses for the sentence up to the current p oint of pro cessing
h The interpretive mo dule serves as a lter o ccasionally discarding certain of these analyses whic
it deems less appropriate for the ongoing discourse than their comp etitors
This do cument is divided into three parts The rst is intro ductory and reviews a selection of
prop osals from the sentence pro cessing literature The second part explores a b o dy of data which
has b een adduced in supp ort of a theory of structural preferences one that is inconsistent
with the present claim I showhow the current prop osal can b e sp eci ed to accountforthe
available data and moreover to predict where structural preference theories will go wrong The
third part is a theoretical investigation of howwell the prop osed architecture can b e realized
using current conceptions of linguistic comp etence In it I present a parsing algorithm and a
meaning based ambiguity resolution metho d v
Contents
Acknowledgements iii
Abstract v
Intro duction
Previous Work
Memory Limitation
Representing an Analysis
Representing Comp eting Analyses
Delib eration b efore Ambiguity Resolution
Shieb er and Pereira
Syntactic Optimality
Lexical Asso ciation
Explicit Consideration of Syntactic Choices
The Weakly InteractiveModel
Discussion
The Central Claim
Accounting for Recency Phenomena
Right Asso ciation
Information Volume
A study
APotential Practical Application
olume and Sensibleness Information V
Conclusion vi
Other Constructions
Available Evidence
NP vs S complement
Late Closure Ambiguity
Degree of Disconnectedness
A Representation of Semantics
AFormal De nition
Consequences
Avoid New Sub jects
Given and New
Consequences
Summary
Parsing CCG
Goal
Steedman s Prop osal
Strict Comp etence and Asynchronous Computation
Synchronous and Asynchronous Computation
Evaluation
Identifying Ungrammaticality
Shift Reduce Con icts
Heavy Shift and Incremental Interpretation
Coping with Equivalent Derivations
Evaluation Criteria for a Parser
Previous Attempts
AProposal
Using the Recovered Constituent
Summary vii
A Computer Implementation
Desiderata
Data Structure
Control Structure
Bottom Up Reduce Algorithm
Bu er Admissibility Condition
Interpretation
Real World Implausibility
De nite Reference
Detecting the End of a Phrase
An Example
A Consistent Theory of Penalties
Desired Behavior
Fitting The Data
A Prediction
Summary
Conclusion
A Data from Avoid New Sub ject investigation
B A Rewrite System for Derivations
Bibliography viii
List of Figures
Main verb and reduced relative clause analyses of
An interactive sentence pro cessing architecture
A circuit for computing z xy y from Shieb er and Johnson
Recombining a recovered constituent with a rightward lo oking mo di er
System Diagram
De nite Reference Resolution Algorithm
B Schema for one redex in DRS
B Normal form computation of a quasi right chain
B When two redexes are not indep endent
B Why DRS is weakly Church Rosser
B One application of
ctr ix
Chapter
Intro duction
The question I address here is how p eople deal with the linguistic ambiguity which p ervades
natural language discourse I fo cus on syntactic ambiguity The task is to construct a detailed
theory of the sentence pro cessing mechanism its comp onents and the nature and dynamics of
their interaction
The data to b e accounted for are measurements of pro cessing di culty or lack thereof in
various sentence typ es b oth in and out of context Current metho ds of measuring pro cessing
di culty are often crude e g naive understandability judgements for a list of sentences or
at b est indirect e g spatially di use EEG resp onse patterns and chronometric measurement
of cross mo dal lexical priming self paced word by word reading eyemovement Nevertheless
observations of pro cessing di cultyvery often show remarkable and unmistakable regularity
This regularity is the data to b e explained
Many mo dels of human sentence pro cessing have b een put forth Most try to account for
pro cessing di cultyby p ositing some prop erties of the parsing comp onent of the linguistic
cognitive apparatus
Frazier and Fo dor and Marcus are well known examples which attempt to derive
a wide variety of phenomena from memory limitations in the pro cessor
Theories have also b een prop osed in which the parser emb o dies a preference for certain analyses
over certain others Frazier and her colleagues haveadvo cated preferences for certain structural
con gurations Pritchett has argued for preference for maximizing the degree to whichthe
principles of grammar are satis ed at every step of the parsing pro cess
Distinct from these parser based theories of pro cessing di culty is a theory advo cated by Crain
Steedman and Altmann CSA hereinafter which ascrib es the lo cus of ambiguity resolution pref
erences to higher level interpretive comp onents as opp osed to the lower level syntactic parsing
comp onent CSA describ e this architecture in broad terms and apply it in detail to a fairly
narrow class of phenomena essentially mo di er attachmentambiguity In this dissertation I
argue for a conception of the syntactic pro cessor which is a generalization of CSA s prop osal
In this work I do not address the strength of pro cessing di culty e ects nor the issue of howhumans cop e
with pro cessing di culty e g by rereading the o ending passage The aim is solely to account for those linguisti c
environments whichgive rise to pro cessing di culty
My claim is that the syntactic pro cessor is the simplest imaginable all it represents is syntactic
analyses of the input It is not resp onsible for resolution of ambiguity that task is p erformed
by the interpreter
This do cument is divided into three parts The rst is intro ductory and reviews a selection of
prop osals from the sentence pro cessing literature much of which implicitly assume a sp ecialized
syntactic pro cessor It concludes with a detailed statement of the central claim of the disser
tation The second part chapters and explores a b o dy of data which has b een adduced
in supp ort of a theory of structural preferences one that is inconsistent with the present
claim In these chapters I showhow the current prop osal can b e sp eci ed to accountforthe
available data and moreover to predict where structural preference theories will go wrong The
third part chapters and is a theoretical investigation of howwell the prop osed architec
ture can b e realized using current conceptions of linguistic comp etence Chapter addresses
issues of parsing it is an attempt to carry out Steedman s program of simplifying
the theory of the parser by adopting a comp etence grammar which de nes more incremental
analyses than other grammars Chapter is a synthesis of the parser develop ed in chapter and
the comp etence base ambiguity resolution criteria develop ed in previous chapters It describ es
an implemented computer mo del intended to demonstrate the viability of the central claim
Chapter provides a conclusion and suggests areas of further research
Chapter
Previous Work
In this chapter I review a selected sample of the sentence pro cessing literature Of the many
issues whichany prop osed mo del of human sentence pro cessing must address I fo cus on two
the role of memory limitations and the extent of delib eration which precedes ambiguity
resolution The reader is referred to Gibson for a general review and cogent critique of
the literature
Memory Limitation
Considering that the pro cess of sentence understanding is successfully implemented by the com
putational mechanism of the human brain one may ask ab out the nature of the architectural
features of this computational device what is the relation among the various sub comp onents
lexical syntactic and interpretive pro cesses and what sorts of limitations are imp osed on
computational and memory resources by the nite hardware dedicated to the task I b egin
with the latter question and fo cus on memory limitations
Representing an Analysis
The most familiar demonstration that the pro cessing system do es not nd all grammatically
p ossible analyses of a string with equal ease is the classic example from Chomsky and Miller
The rat that the cat that the dog bit chased died
Miller and Chomsky accounted for this in automaton theoretic terms the pro cessor cannot
be interrupted while pro cessing a constituentoftyp e X to pro cess another constituentoftyp e
X More recentwork Gibson Joshi Rambow and Joshi consider a variety
of constructions in English and German whichgive rise to center emb edding like e ects and
come to similar though not identical conclusions as it pro ceeds incrementally through the
input string the underlying automaton is incapable of maintaining a large numb er of separate
pieces of the input which are not integrated together I return to this issue in sections
and Di culty with sentences such as arise indep endently of syntactic ambiguity
they indicate an inherent limitation in the pro cessor in representing the linguistic structure
which they require The conclusion that this di culty results from memory constraints in the
pro cessor is unchallenged in the recent literature as far as I know
Representing Comp eting Analyses
The question of whether memory limitations are resp onsible for another form of pro cessing
di culty namely so called garden path sentences as in is much more controversial
The horse raced past the barn fell
With this sentence there is no question that the pro cessor is capable of representing the necessary
linguistic structure the grammatically identical sentence in causes no pro cessing di culty
The horse ridden past the barn collapsed
Authors suchasFrazier and Fo dor and Marcus see Mitchell Corley and Garnham
Weinb erg for more recent incarnations of the twoworks resp ectively have argued
that when the pro cessor encounters the lo cal temp orary ambiguity in the word raced in
it is incapable of keeping track of b oth available analyses of the input until the arrival of the
disambiguating information That is memory limitations force a commitment Other authors
Crain and Steedman Altmann and Steedman McClelland St John and Taraban
Gibson Pritchett Spivey Knowlton Trueswell and Tanenhaus inter alia
have argued that the pro cessor considers all grammatically available analyses and pic ks among
the alternatives according to certain preferences these authors di er widely ab out what the
preferences are I now consider a few of these pap ers in more detail
The Sausage Machine
Frazier and Fo dor prop osed an architecture for the syntactic pro cessor whose central
characteristic is a stage of pro cessing whose working memory is limited Their prop osal is that
the sentence pro cessing mechanism is comprised of mo dules
The Preliminary Phrase Packager PPP is a shortsighted device which p eers at
the incoming sentence through a narrow window which subtends only a few words
at a time It is also insensitive in some resp ects to the well formedness rules of the
language The Sentence Structure Sup ervisor SSS can survey the whole phrase
marker for the sentence as it is computed and it can keep track of dep endencies
between items that are widely separated in the sentence and of long term structural
commitments which are acquired as the analysis pro ceeds p
Interesting predictions of pro cessing di culty arise for situations where the PPP imp oses the
incorrect bracketing or chunking on a substring of the input Frazier and Fodor characterize
the PPP has having a memory size of roughly six words and attempting at any p ointto
group as may items as it can into a single phrasal package p Aside from predicting
di culties with center emb edded sentences e g in the PPP might try to chunk the rat that
the cat into one package their accountmakes interesting predictions with resp ect to mo di er
attachment Consider
Wewent to the lake to swim quickly
Their account predicts that the PPP will attempt to structure quickly with the material imme
diately to its left namely to swim rather than with went This prediction is not made when
the adverbial consists of more words e g
Wewent to the lake to swim but the weather was to o cold
In the adverbial clause but cannot t into the PPP together with to swim so the PPP
puts the two constituents into separate packages and the SSS has the opp ortunity to decide
how to attach the three packages
Wewent to the lake to swim but the weather was to o cold
The time pressure under which the pro cessor is op erating faced with quickly incoming words
leads Frazier and Fo dor to make another prediction ab out attachmentambiguity resolu
tactically simplest analyses will b e found rst thus preferred This was tion namely that syn
formalized byFrazier
Minimal Attachment Attach incoming material into the phrase marker
b eing constructed using the fewest no des consistent with the well formedness
rules of the language
Minimal Attachment predicts that the main verb analysis of raced in will b e initially pur
sued as can b e seen by the relativesyntactic complexity of the main verb and reduced relative
analyses in gure
Minimal Attachment similarly predicts that the sentences in eachgive rise to a garden path
a The cop shot the spy with the bino culars
b The do ctor told the patient that he was having trouble with to leave
In more recentwork Frazier and her colleagues Rayner Carlson and Frazier Frazier
prop ose a di erent mo dularization of the language pro cessing faculty the syntactic processor
constructs a single analysis of the incoming words according to structurally de ned criteria such
as Minimal Attachmentabove The thematic processor considers the phrasal constituents that
the syntactic pro cessor has found and considers in parallel all the p ossible thematic combina
tions of these constituents When it nds a b etter thematic combination than the one b eing
constructed by the syntactic pro cessor it interrupts the syntactic pro cessor telling it to reana
lyze the sentence
NP
l
l
l
DET N
e
e
the RC N
S horse S
Q
e
Q
Q
e
NP VP VP NP
DET N V trace V
the horse raced raced
Figure Main verb and reduced relative clause analyses of
PARSIFAL
Marcus seeks to reconcile the apparent sp eed and e ciency of the human sentence pro
cessing mechanism with traditional parsing techniques for ambiguous grammars whichare
signi cantly slower Standard parsing algorithm require time which is either p olynomial or ex
p onential in the length of the string but humans do not require words to arrive more slowly as
the input string the sentence b ecomes longer Marcus concludes that the pro cessor must
b e able to make all parsing decisions in a b ounded amount of time i e using a b ounded number
pro cessing steps He prop oses an automaton mo del which he calls Parsifal This mo del is a
pro duction system which has a data store and set of pattern action rules Toachieve a b ound
on the amount of time required by the pro cessor to make its move Marcus b ounds the p ortion
of the pro cessor s memory which is visible to the rules The store has two comp onents a parse
stack and a bu er of three cells each capable of storing one constituents The rules may only
mention the syntactic category of the contentofeach of the cells and roughly the top of the
parse stack The pro cessor pro ceeds deterministical ly in the sense that any structure it builds
by attaching constituents from the bu er into the stack may not b e destroyed When the
pro cessor reaches an ambiguity itmay either resolve it or it mayleave one or more constituents
uncombined in the bu er provided there is ro om If there is no ro om in the bu er for new
ed to make a commitment whichmay result in a garden path constituents the pro cessor is forc
An account of garden paths which is based strictly on the cell memory limit quickly runs into
empirical di culties Pritchett provides the following examples which can b e resolved
within a cell bu er but nevertheless app ear to b e garden paths see Gibson for a
detailed critique of Marcus s parser
a The b oat oated quickly sank
b Without her money would b e hard to nd
c While Tegan sang a song played on the radio
Minimal Commitment
Marcus Hindle and Fleck prop ose an architecture whichmaintains the cell bu er
of Marcus s earlier work but factors the pro cedural pattern action rules into a more elegant
collection of structural description rules and an engine which applies them While the rules of
grammar are ab out direct dominance of no des in the phrase marker the pro cessor maintains
partial sp eci cations by means of dominance statements and other devices Preserving the
determinism in Marcus s parser their pro cessor may not retract any assertions ab out the phrase
marker that it is constructing Weinb erg adopts Marcus et al s prop osal of partial
descriptions of phrase structure but jettisons altogether the idea of a b ounded bu er Instead
Weinb erg adopts an arguably less stipulative accountofgardenpathsentences
Principle of QuickInterpretation The parser attaches arguments using
the smallest numb er of dominance statements and features necessary to assign
grammatically relevant prop erties
This account predicts a garden path whenever the commitments necessitated by the Principle of
QuickInterpretation turns out to b e inconsistent with subsequent material No garden path is
predicted in cases where the commitment i e partial description constructed by the Principle
of QuickInterpretation is consistent with the rest of the string
For an illustration of Weinb erg s parser consider
a I knew Eric
b I knew Eric was a great guy
Weinb erg s accountentails that neither a nor b is a garden path This follows from the
description that the pro cessor builds after encountering the pre x I knew Eric
S
l
l
NP VP
NP I V
knew Eric
where the links are express dominance not direct dominance is compatible with either
the direct dominance interpretation of or with the analysis necessary for b where an S
no de intervenes b etween the VP no de and the Eric no de is in contrast with
np
Weinb erg s partial structural description include statements of dominance direct dominance linear prece
dence and partial category sp eci cation using features
After Mary ate the fo o d disapp eared from the table
When the pro cessor encounters the fo o d the Principle Of QuickInterpretation commits it
to the fact that the VP headed by ate dominates the NP the fo o d This commitmentis
inconsistent with the rest of the string so a garden path is correctly predicted
Weinb erg s prop osal is that the sentence pro cessor s working memory is limited to hold exactly
one structural representation UnlikeFrazier and Fo dor s and Marcus s prop osals this limita
tion is con ned to the representation of ambiguity Weinb erg s memory limitation makes no
predictions ab out di culty with center emb edding
The three prop osals ab ove all share the fundamental prop erty that the pro cessor pursues only
one analysis at a time This has b een called serial pro cessing as well as determinism Standing
in contrast to serial pro cessing are prop osals that the pro cessor constructs representations for
the various ambiguous analyses available at anypoint Of the many parallel prop osals in the
literature I shall review only two Gibson s prop osal of pro cessing load and breakdown
and the parallel weak interaction mo del of Crain Steedman and Altmann CSA Crain and
Steedman Altmann and Steedman
Gibson
Gibson prop oses that the human sentence pro cessing mechanism pursues all grammati
cally available analyses in parallel as it pro cesses the string discarding those analyses whichare
to o costly that is when the cost of one analysis A exceeds that of another analysis B by
more than P Pro cessing Load Units A is discarded necessitating conscious e ort to reconstruct
should it b e subsequently necessary The cost of an analysis is the sum of Pro cessing Loads which
it incurs byvirtueofhaving certain memory consuming prop erties Within Gibson s mo del a
theory of sentence pro cessing consists in a precisely de ned collection of memory consuming
prop erties and a numeric cost asso ciated with each Considering a variety of data mostly in
trosp ective judgements of pro cessing di cultysentences Gibson prop oses a collection of four
memory consuming prop erties three have to do with failures to identify the relations among
the various constituents in the string cf Chomsky s Principle of Full Interpretation
the fourth prop erty asso ciates a cost with the need to access a constituent which is not the
most recent Gibson concentrates on syntactic prop erties which he considers the most tractable
to investigate He acknowledges that a complete theory of sentence pro cessing would likely
require augmenting his set of prop erties with lexical semantic pragmatic and discourse level
prop erties which are asso ciated with signi cant pro cessing loads
Crain Steedman and Altmann
Crain and Steedman and Altmann and Steedman rep ort a collection of exp eri
ments which militate against a mo del in which the syntactic pro cessor op erates in a serial or
deterministic fashion Consider the lo cal ambiguity in a illustrated in b and c
One could argue that Frazier et al s mo del is a mix of serial syntactic and parallel thematic pro cessing
but what is relevant here is the question of whether the initial syntactic analysis is carried out in serial or parallel
a The psychologist told the wife that
b The psychologist told the wife that he was having trouble with her husband
c The psychologist told the wife that he was having trouble with to leave her
husband
A mo del where the syntactic pro cessor op erates serially would predict that the ambiguitywould
b e resolved on some structural grounds e g Minimal Attachment presumably toward the com
plementizer analysis of that as in b not the relativizer analysis in c This resolution
would o ccur indep endently of meaning of the constituents in question But Crain and Steedman
found that dep ending on compatibilitywiththediscourse context the pro cessor can b e made to
select either analysis When there were twowives in the discourse context b wasagarden
path re ecting a commitmenttoward a further restrictor on the set of candidate referents
When there was one wife in the discourse c was a garden path This basic nding was
replicated using a di erentambiguous structure and metho dologies by Altmann and Steedman
Sedivy and Spivey Knowlton and Altmann Garnham and Dennis Given
the sensitivity to the meaning of the various alternatives CSA argue that the pro cessor must b e
explicitly weighing the sensibleness of the alternatives It follows that the interpreter receives
representations in parallel from the syntactic pro cessor of all available syntactic analyses
Neither Gibson nor CSA discuss explicit b ounds on the numb er of analyses that are maintained
by the pro cessor at any time This just means that unlike Marcus s prop osal and the Sausage
machine it is only the preference criteria themselves not the memory b ounds that b ear the
y resolution b ehavior It must b e emphasized that neither paral explanatory role for ambiguit
lel mo del ab ove requires that the pro cessor b e able to represent the p otentially exp onentially
proliferating set of ambiguous analyses for a multiply ambiguous string whenever the pro ces
sor s preference reaches some threshold it discards the less preferred analyses thus keeping the
size of analysis set manageable Indeed most ambiguities are resolved very quickly making the
pro cessor app ear as if it op erates serially There is additional exp erimental evidence in supp ort
of a parallel mo del of the sentence pro cessor
Gorrell
Gorrell used a lexical decision task to show that b oth analysis of a temp orarily am
biguous sentence are maintained that is the ultimately dispreferred analysis exerts an e ect
of lexical decision facilitation With sentences such as Gorrell used target words is has
must which are consistent with the dispreferred complex analysis and found facilitation in
b oth the Ambiguous and Complex conditions but not for the unambiguous Simplex condition
Presentation of the sentences were interrupted at the p oints marked with for presentation of
the target word
CSA address their arguments sp eci cally against Minimal Attachment but it applies to other any structural
preference strategies such as those in the prop osals of Weinb erg ab ove Pritchett and others
Where in the middle of reading a sentence the sub ject is presented with a word and has to quickly resp ond
with whether it is a word of the English language It has b een argued Wright and Garrett that this task
is facilitated if the target word ts in at the p oint in the sentence that the sub ject is pro cessing
NP S Ambiguity
Simplex It s obvious that Holmes saved the son of the banker rightaway
Ambiguous It s obvious that Holmes susp ected the son of the
banker rightaway was guilty
Complex It s obvious that Holmes realized the son of the banker was guilty
Main Verb Participle ambiguity
Simplex The companywas loaned money at low rates to ensure high volume
Ambiguous The company loaned money at low rates to ensure high volume
decided to b egin expanding
Complex The company they loaned money at low rates decided to b egin
expanding
Hickok Pickering and Nicol
Additional exp eriments byHickok and Nicol and Pickering con rm Gorrell s nd
ings Working indep endently these researchers considered the lo cal ambiguity used byCSAin
Using the metho d of antecedent reactivation they found that the relative clause analy
sis which is strongly dispreferred to the complement clause analysis is still active and causes
reactivation of the WH trace at the p osition marked with
The girl swore to the dentist that a group of angry p eople called the o ce
ab out the incident
Hickok used visual computer paced presentation of the sentence while Nicol and Pickering
used cross mo dal priming the sentences were presented auditorily and the target word was
presented visually Results from the two exp eriments consistently show reactivation of the WH
antecedent This result is quite surprising given the remarkable extent to which sub jects are
garden pathed when faced with a string such as It suggests that dispreferred analyses are
not discarded outright they just fade away
The girl swore to the dentist that a group of angry p eople called that she was
going to quit
MacDonald Just and Carp enter
MacDonald Just and Carp enter argue that how quickly dispreferred analyses fade away
is sub ject to individual variations in short term memory MacDonald et al rated their sub jects
on their p erformance on the Reading Span Task a task in which the sub ject reads a list of
unrelated sentences keeping track of the the last word in eachsentence At the end of the list the
sub ject must recall the nal words Sub jects vary substantially on the length of the list for which
they can p erform the task accurately Score on this task is p ositively correlated with a variety
Where at the p osition of a WH trace the lexical decision times for words which are semantically related to
the antecedent of the trace are facilitated Swinney et al See Fo dor for a review
of language p erformance scores including SATverbal score The theory that MacDonald et al
prop ose is that high span sub jects maintain ambiguities for longer p erio ds of time This theory
makes the interesting and counter intuitive prediction that for lo cally ambiguous sentences which
are disambiguated consistently with the preferred analysis high span readers would havetowork
harder than low span readers since they would also b e maintaining the do omed non preferred
analysis This is indeed what they found They compared the lo cally ambiguous sentence in
a to an unambiguous control b and to the non garden pathing main verb analysis in
c
a The exp erienced soldiers warned ab out the dangers conducted the midnight raid
b The exp erienced soldiers who were told ab out the dangers conducted the
midnight raid
c The exp erienced soldiers warned ab out the dangers b efore the midnight raid
They found that high span readers could cop e b etter with the ambiguity in a On a reading
comprehension task high span readers p erformed b etter than low span readers correct
versus correct almost at chance on true false questions This con rms the relevance
of the reading span task to some asp ects of reading ability More interestingly MacDonald et
al found that for the main verb sentences as in c high span readers to ok signi cantly
more time to read the last word of the sentence For high span readers there was a very
slight elevation in the reading time of the ambiguous region warned ab out the dangers in
the ambiguous sentences a and c as compared to the lo cally unambiguous b This is
clear evidence of the additional burden whichmaintaining the p ossibility of a reduced relative
analysis imp oses on high span readers Slight though this e ect is it do es constitute an online
measure of the cost of maintaining multiple analyses in parallel
Summary
The existence of garden path sentences leads to the inescapable conclusion that not all syn tactic
analyses are maintained inde nitely The stronger conclusion that multiple syntactic analyses
are never retained from word to word is inconsistent with three sorts of psycholinguistic evidence
The meaning of the various comp eting analyses are compared hence computed requiring
the identi cation of syntactic relations Note that prop osals such as the thematic pro ces
sor of Frazier and her colleagues do not sp ecify howtheinterpretive mo dule can identify
which of the many p ossible relations among constituents are p otentially allowed by gram
matical analyses of the string which the pro cessor has not chosen Di erent languages
imp ose di erent restrictions on which constituents maybecombined so syntactic analysis
must precede interpretation
The discarded reading still manifests certain signs of life on su ciently sensitive tests
such as the lexical decision task
This e ect reached statistical signi cance only when data from many exp eriments with slightly di erent
conditions were p o oled together
For readers who show signs of coping b etter with ambiguity the dispreferred reading exacts
a measurable pro cessing cost
Delib eration b efore Ambiguity Resolution
Aside from memory limitations assumed by a mo del another dimension along which the various
prop osals vary is the nature and amount of computation that precedes ambiguity resolution The
two logically extreme p ositions haveeach b een advo cated that any pro cessing whatso ever
including arbitrarily complex inference can precede ambiguity resolution and that ambiguityis
not even identi ed by the pro cessor online let alone delib erated on Some pap ers advo cate inter
mediate p ositions In this section I present a few pap ers arranged in approximately increasing
order of amount of pre resolution delib eration
Shieb er and Pereira
Shieb er and Pereira prop ose a technique for constructing a deterministic au
tomaton given a p otentially nondeterministic grammar The automaton s memory consists of
a stack of symb ols grammatical categories and a register which stores the name of one of a
b ounded numberofstateswhich the automaton is in It is equipp ed with a pre compiled action
table which completely determines what move it should take next add remove items from the
stack change the state it is in based on the current state the next word in the input string and
the top of stack symb ol This action table is constructed from a grammar using a well known
grammar compilation technique LR parsing Aho and Johnson If the grammar is lo cally
ambiguous the compilation technique results in certain entries in the action table containing
sets of actions each corresp onding to a di erent analysis Shieb er and Pereira showhow struc
tural preference strategies such as Minimal Attachment and Lexical Preference see b elow
can b e used to resolvesuch indeterminacies in the action table at compile time The resulting
deterministic automaton will therefore follow the path of action consistent with the minimally
attached reading and not even detect the p ossibility of another analysis
Syntactic Optimality
Avariety of prop osals Frazier and Fo dor Rayner Carlson and Frazier Weinberg
Pritchett inter alia p osit structural preference criteria None of these prop osals
concretely sp ecify the algorithm by which the pro cessor nds the preferred parse Presumably
this involves some sort of searchover the space of analyses p ossible for the input so far For
example Frazier s Minimal Attachment principle could b e made to fall out of a pro cessor which
tries to integrate the next word into the current phrase marker by trying all combinations in
parallel and stopping as so on as it has found the rst grammatical solution In none of these
prop osals do es any non syntactic information enter into the pro cess of determining the rst pass
analysis
written at roughly the same time
Lexical Asso ciation
Ford Bresnan and Kaplan argue that aside from purely structural ambiguity resolution
criteria the pro cessor is also sensitive to the strength of asso ciation b etween certain words like
verbs and the nouns they take as arguments They conducted a questionnaire exp erimentin
which they presented participants with an ambiguous sentence such as and asked them to
identify which reading they got rst
The woman wanted the dress on the rack
They found that bychanging the main verb they could signi cantly alter the ambiguity res
olution preferences observed For example was resolved of the time with the PP
mo difying dress and of the time mo difying wanted however when wanted was replaced
with p ositioned the preferences reversed from vs to vs Ford et al incorp orate
such preferences into a serial pro cessing algorithm their pro cessor considers the set of p ossible
rules at any p oint applying b oth lexical preference and general structurally state rules to decide
which rule to apply next
Explicit Consideration of Syntactic Choices
The mo dels of Marcus and Gibson explicitly reason ab out the various syntactic
alternatives available at anypoint Marcus s system contains rules for di erential diagnosis
of lo cal structural ambiguity These rules consider the current collection of constituents and
decide howtocombine them Gibson s system explicitly constructs all grammatically available
structures and applies preference metrics to adjudicate among them While b oth systems adhere
y resolution their authors acknowledge the need for to solely syntactic criteria for ambiguit
certain meaning based preferences in more complete realistic versions of their work Gibson
chapter esp p Marcus chapter
The Weakly Interactive Mo del
CSA argue that the syntactic pro cessor constructs all grammatically available analyses and the
interpreter evaluates these analyses according to meaning based criteria While the criterion they
prop ose requires p otentially very elab orate inferences to apply their actual exp eriments
rely on relatively easy to compute asp ects of meaning
Principle of Parsimony Crain and Steedman
If there is a reading that carries fewer unsatis ed but consistent presupp ositions
or entailments than any other then other criteria of plausibili ty b eing equal that
reading will b e adopted as most plausible by the hearer and the presupp ositions
in question will b e incorp orated in his or her mental mo del of the discourse
In their exp eriments Crain and Steedman presented a lo cally ambiguous sentence suchas
in two di erent contexts as exempli ed in
The psychologist told the wife that he was having trouble with to leaveher
husband
a One couple context
Apsychologist was counseling a married couple One memb er of the pair was
ghting with him but the other was nice to him
b Two couple context
A psychologist was counseling two married couples One of the couple was ght
ing with him but the other was nice to him
The inference which their sub jects evidently were computing were rst going from a married
couple or two to a part of the couple namely a wife second determining whether the de nite
expression the wife referred uniquely presumably by determining whether the cardinalityof
the set of wives was greater than one In another exp eriment Crain and Steedman found
e ects of plausibili tyinhow often sub jects garden pathed on examples suchas
a The teachers taughtby the Berlitz metho d passed the test
b The children taughtby the Berlitz metho d passed the test
This is evidence that sub jects use online the knowledge that teachers typically teach and children
typically are taught Again one may argue that this sort of knowledge could conceivably b e fairly
directly represented and is very quick to access see Resnik Plausibility e ects on the
reduced relative main verb ambiguityin have since b een found by many researchers Pearl
ell mutter and MacDonald Trueswell Tanenhaus and Garnsey inter alia Truesw
and Tanenhaus have found that sub jects are sensitive to the temp oral coherence of the
discourse when parsing reduced relative clauses For example The student caughtcheating
is more likely to b e interpreted as a reduced relative when the discourse is in the future tense
than when it is in the past tense
Marslen Wilson and Young cited in Marslen Wilson and Tyler conducted an exp eriment
whichshows immediate e ects of a rather complex inference pro cess They placed ambiguous
phrases such as ying planes and visiting relatives in contexts which inferentially favor one of
their two readings
a If you wanta cheap holiday visiting relatives
b If you have a spare b edro om visiting relatives
Sub jects listened to an audio tap e of these materials and at end of the fragment they were
presented with a written word Their task was to read the word outloud the so called cross
mo dal naming task The words of interests were is and are consistent with the a and
b meanings resp ectively Marslen Wilson and Young found signi cant e ects of plausibili tyon
sub jects reaction times indicating that the relatively complex inference required is broughtto
b ear on the immediately fol lowing word It is not clear just howmuch inference is broughtto
b ear on a word by word basis this is due in part no doubt to our current inability to ob jectively
assess the complexity of inference
Discussion
There is a substantial and growing b o dy of evidence in supp ort of the claim that the human
sentence pro cessing mechanism consults a variety of information sources b efore it resolves am
biguities
prop erties of particular lexical items such as their preferred sub categorization frames Ford
Bresnan Kaplan Garnsey Lotocky and McConkie Trueswell Tanenhaus and
Kello Juliano and Tanenhaus
semantic prop erties asso ciated more or less directly with the words in the sentence e g
married couple wife teachers teach from Crain and Steedman cheap vacation
visiting relatives from Marslen Wilson and Young
t of the linguistic expression into the current discourse e g de nite reference CSA
coherence of tense Trueswell and Tanenhaus
There is not however a consensus that the language pro cessing architecture is indeed parallel
and highly delib erative Mitchell Corley and Garnham argue that there are separate
syntactic and thematic pro cessors see section while the thematic pro cessor do es con
sider the meanings of the various combinations of the words in the string so far the syntactic
pro cessor pursues only one analysis The thematic pro cessor can come to susp ect that the syn
tactic pro cessor may b e pursuing the wrong analysis and alert it very quickly to change course
This quick alert strategy whichMitchell et al refer to as stitch in time can sometimes trickthe
pro cessing system into a garden path The consequence of this argument is that if one trains
one s psycholinguistic measurement apparatus on the exact p oint in the pro cess one could catch
the syntactic pro cessor constructing the minimally attached analysis only to have this analysis
undred milliseconds later abandoned in favor of the contextually appropriate analysis a few h
This issue is currently b eing debated with researchers on b oth sides re ning their exp erimental
techniques see Altmann Garnham and Dennis
For information suchasverb sub categorizati on frame preferences it is very hard to tease apart whether the
information is asso ciated with the lexical entry for the verb or with the deep er representation of the concept
e g of the verb and how it is asso ciated to other concepts e g its arguments to which it is b eing related
by the sentence Current research on practical applications of natural language technology intryingtoavoid
the complexityofknowledge representation has b een quite successful in assuming rich relation among words
Collecting lexical co o ccurrence statistics from large text corp ora researchers e g Hindle and Ro oth are
able to construct ambiguity resolution algorithms which p erform signi cantly b etter than ones based hand co ded
domain knowledge In fact it is surprising to see just how far co o ccurrence based statistical approaches to
approximating natural language can go Church presents an algorithm for determining the form class of
words in text This algorithm is trained on hand tagged text it p erforms no syntactic analysis of its input it only
keeps track of the form class frequency for eachword and the frequency of consecutive form class tags in text
Using this remarkably imp overished approximation of the linguisti c phenomena of English Church s algorithm
was able to achieve form class determination p erformance of b etter than The success of these algorithms
can serve as a demonstration of how easy it is to cheat by attributing complex b ehavior using asso ciation based
strength of representations of surface observable ob jects suchaswords
The Central Claim
The claim that I argue in this dissertation is that the parsing mechanism a straightforward device
which blindly and faithfully applies the knowledge of language syntactic comp etence directly
to its input allowing the interpretation mo dule to imp ose its preferences in case of ambiguity
At each p oint in pro cessing the parser constructs all available syntactic analyses for the input
thus far The interpreter considers the set of available analyses and what eachwould mean and
selects a subset to discard The parser deletes these analyses and extends the remaining ones
with the next incoming word rep eating the pro cess until it is either exhausted the input string
or it is stuck none of the non discarded analyses has a grammatical continuation in the next
input word
The following are immediate consequences of this claim
There are no structural preferences e g Minimal Attachment enco ded or implemented
by the parser
All ambiguity resolution decisions among grammatically licensed analyses stem directly
from the linguistic comp etence in the broadest sense of the term
plausibili ty of the message carried by the analysis
quality of t of this message into the current discourse
felicity of the constructions used in the utterance to express the message
the relative frequency of use of a certain construction or lexical item
That is when resolving ambiguity the hearer answers the question which of these gram
matically p ossible analyses is the one that the sp eaker is most likely trying to communicate
to me
Each of the four criteria in ab ovecanbeinvestigated indep endently of syntactic ambi
guity
The parser uses a direct representation of the comp etence grammar as opp osed to some
sp ecially pro cessed enco ding intended solely for the task of parsing
y memory b ounds in the Certain parsing e ects whichhave b een heretofore explained b
parser have explanations elsewhere
Parsing do es not always pro ceed serially or deterministically
Bu er limitation based predictions of how long ambiguity can b e maintained and
when it must b e resolved e g Marcus s cell bu er the Sausage Machine s word
window will predict either to o long an ambiguity maintenance p erio d in case disam
biguating information is available early or to o short a p erio d in case disambiguating
information is not available
On the assumption that the knowledge of language sp eci es quantitative frequency information e g sub
categorization frame preference
True memory load e ects whichwould arise in arti cial situations where many lo
cally ambiguous readings are available for the input string but no disambiguating
information is applicable will result from a di use shortage in attentional resources
needed to keep track of the many analyses in parallel in analogy with an overloaded
multi user computer which exhibits gradual p erformance degradation
Chapter
Accounting for Recency Phenomena
In the previous chapter I reviewed evidence that the ambiguity resolution pro cess is sensitiveto
avariety of asp ects of sensibleness of the comp eting analyses real world plausibility felicityof
de nite reference and temp oral coherence In this chapter and the next I consider a collection
of ambiguities which seem at rst glance to b e resolved by criteria other than sensibleness
I will argue that when the notion of sensibleness is broadened to encompass the degree of t
to current discourse situation these ambiguities receive a straight forward sensibleness based
account
Right Asso ciation
Kimball prop oses the parsing strategy of Right Asso ciation RA RA resolves mo di ers
attachmentambiguities by attaching at the lowest syntactically p ermissible p osition along the
right frontier of the phrase marker Many authors among them Wilks Schub ert
Whittemore et al and Weischedel et al incorp orate RA into their parsing systems
yet none rely on it solely integrating it instead with ambiguity resolution preferences derived
from word constituent concept co o ccurrence based criteria On its own RA p erforms rather
well given its simplicity but it is far from adequate Whittemore et al evaluate RA s p erfor
mance on PP attachment using a corpus derived from computer mediated dialog They nd
that RA makes correct predictions of the time Weischedel et al using a corpus of news
stories rep ort a success rate on the general case of attachment using a strategy Closest
ttachment which is essentially RA In the works just cited RA plays a relatively minor role A
as compared with co o ccurrence based preferences
The status of RA is very puzzling Consider
a John said that Bill left yesterday
b John said that Bill will leaveyesterday
In China however there isn t likely to b e any silver lining because the economy
remains guided primarily by the state
from the Penn Treebank corpus of Wall Street Journal articles
John sold it to day John sold to day it
John sold the newspap ers to day John sold to day the newspap ers
John sold his rustysocket wrench set to day John sold to day his rustysocket wrench set
John sold his collection of RPM Elvis John sold to day his collection of RPM
records to day Elvis records
John sold his collection of old newspap ers John sold to day his collection of old news
from b efore the Civil War to day pap ers from b efore the Civil War
Table Illustration of heaviness and word order
On the one hand manynaive informants do not see the ambiguity of a and are often confused
by the semantically unambiguous b a strong RA e ect On the other hand violates
RA with impunity What is it that makes RA op erate so strongly in but disapp ear in
In the rest of this chapter I argue that the high attachment of the adverbial enco des
a commitment ab out the information structure of the sentence which is infelicitous with the
information carried in but not with that in This commitmentisaboutthevolume of
information enco ded in various constituents in the sentence and the feature which enco des this
commitmentisword constituent order
Information Volume
Quirk et al de ne end weight as the tendency to place material with more information
content after material with less information content This notion is closely related with end
focus which is stated in terms of imp ortance of the contribution of the constituent not merely
the quantity of lexical material These two principles op erate in an additive fashion Quirk et
al use them to accountforavariety of phenomena among them
genitiveNPs
the sho ck of his resignation
his resignation s sho ck
it extrap osition
It b othered me that she left quickly
That she left quickly b othered me
Information volume clearly plays a role in mo di er attachment as shown in table My claim
is that what is wrong with sentences such as is the violation in the high attachment of the
principle of end weight While violations of the principle of end weight in unambiguous sentences
e g those in table cause little grief as they are easily accommo dated by the hearer reader
the online decision pro cess of ambiguity resolution could well b e much more sensitivetosmall
di erences in the degree of violation In particular it would seem that in b the weight based
preference for low attachmenthasachance to in uence the parser b efore the temp oral inference
based preference for high attachment
Iamaware of no work which attempts to systematically tease apart the notion of amountof
linguistic material measured in words or morphemes from the notion of amount of information
communicated in the pragmatic sense In this do cument I use the term information volume
to refer to a vague combination of these two notions on the assumption that they are highly
correlated in actual sp eech and text To further simplify and op erationalize the de nition of
information volume I classify single word constituents and simple NPs as low information volume
and constituents which include a clause as high information volume In section I argue that
avery signi cant determinant of information volume is the pragmatic information carried by
the constituent not by length of its surface realization
A study
The consequence of my claim is that low information volume adverbials cannot b e placed after
high volume arguments while high volume adverbials are not sub ject to such a constraint
When the sp eaker wishes to convey the information in a high attachment there are other
word orders available namely
a Yesterday John said that Bill left
b John said yesterday that Bill left
If the claim is correct then when a single word adverbial mo di es a VP which contains a high
information volume argument the adverbial will tend to app ear either b efore the VP or b etween
the verb and the argument High volume adverbials should b e immune from this pressure
Toverify this prediction I conducted an investigation of the Penn Treebank corpus of ab out
million words of syntactically annotated text from the Wall Street Journal Unfortunately
the corpus do es not currently distinguish b etween arguments and adjuncts they are b oth
annotated as daughters of VP Since at this time I do not have a dictionary based metho d for
S when from VP left S when my search cannot include all distinguishing VP asked
adverbials only those which could never or rarely serve as arguments I therefore restricted
my search to subgroups of the adverbials
S s whose complementizers participate overwhelmingly in adjuncts after although as be
cause beforebesides but by despite even lest meanwhile onceprovided should since so though
unless until upon whereas while
single word adverbials now however then already heretoorecently instead often later once
yet previously especial ly again earlier soon ever rst indeed sharply largely usual ly together
quickly closely directly alone sometimes yesterday
The particular words were chosen solely on the basis of frequency in the corpus without p eeking
at their word order b ehavior
Each adverbial can app ear in at least one p osition b efore the argumenttotheverb sentence initial preverb
between verb and argument and at least one p ost verbal argument p osition end of VP end of S
For arguments I only considered NPs and Ss with complementizer that and the zero comple
mentizer
The results of this investigation app ear the following table
adverbial single word clausal
arg typ e pre arg p ost arg pre arg p ost arg
lowvolume
high volume
total
Of o ccurrences of single word adverbials app ear after the argument If we
consider only cases where the verb takes a high volume argument de ned as one which contains
an S of the o ccurrences only app ear after the argument This interaction with
the information volume of the argument is statistically signi cant p
Clausal adverbials tend to b e placed after the verbal argument only out of the o ccurrences
of clausal adverbials app ear at a p osition b efore the argument of the verb Even when the
argument is high in information volume clausal adverbials app ear on the right out of a total
of clausal adverbials
and are two examples of RA violating sentences whichIhave found
According to department p olicy prosecutors must make a strong showing that
lawyers fees came from assets tainted by illegal pro ts before any attempts at
seizurearemade
To summarize low information volume adverbials tend to app ear b efore a high volume argument
and high information volume adverbials tend to app ear after it The prediction is thus con rmed
RA is at a loss to explain this sensitivity to information volume Even a revision of RA
such as the one prop osed bySchub ert which is sensitive to the size of the mo di er and
of the mo di ed constituent would still require additional stipulation to explain the apparent
conspiracy b etween a parsing strategy and tendencies in generator to pro duce sentences with
the word order prop erties observed ab ove This also applies to Frazier and Fo dor s
Sausage Machine mo del which accounts for RA e ects using a narrow window in the parser see
section
APotential Practical Application
How can we exploit the ndings ab ove in our design of practical parsers Clearly RA seems to
work extremely well for single word adverbials but how ab out clausal adverbials Toinvestigate
this I conducted another search of the corpus this time considering only ambiguous attachment
sites I found all structures matching the following twolow attached schemata
By I mean match or more daughters By I mean constituentxcontains constituent y as a
x y
rightmost descendant By I mean constituent x contains constituent y as a descendant
x y
lowVPattached adv
vp s vp
low S attached adv
vp s
and the following two high attached schemata
high VP attached v adv
vp s
high S attached adv
s vp s
The results are summarized in the following table
adverb typ e low attached high att high
single word
clausal
As exp ected with single word adverbials RA is almost always right failing only of the
time However with clausal adverbials RA is incorrect almost one out of ve times
Information Volume and Sensibleness
Let us return to the question of whether the attachment preferences discussed ab ove are indeed
consistent with the thesis of sensibleness based ambiguity resolution If it turns out that infor
mation volume is simply a measure of surface complexity words morphemes phrase marker
tree depth etc then there is no role for interpretation and sensibleness to play it follows that
the comp etence grammar marks information volume as a feature on certain no des and assigns a
graded p enalty of some kind to certain sequences of volume markings While the idea of graded
p enalty against certain structural con gurations is not new cf sub jacency the requirement for
a High Volume feature is rather o dd Still there is nothing in this view which is inconsistent
with the main thesis
There is an interesting putative counterexample to the generalization that only low information volume
adverbials give rise to recency e ects shown in i I am grateful to Bill Wo o ds for bringing this example to my
attention
i The Smiths saw the Grand Canyon ying to California
Here there is a remarkably strong tendency to take the participial phrase ying to California as b elonging to an
argument to saw The more plausible reading treats the participle as mo difying the matrix sub ject or the matrix
predication I claim that this e ect is not residual RA but rather it stems from a subtle pragmatic infelicityin
the plausible construal of the participle My intuition is that when a participial adverbial is felicitousl y used the
relation b etween the adverbial and the matrix predication is not merely cotemp oraneity but rather the adverbial
must b e a relevant to matrix predication An informal survey of p ost head participles that do not app ear in
construction with their heads e g sp ent the weekend writing a pap er reveals that they most often app ear
delimited by a comma and are relevantly related to their heads serving such rhetorical purp oses as evidence
consequence elab oration and exception I did not nd examples of mere cotemp oraneity or scene setting In
fact for scene setting functions one tends to add the word while to the participle So the subtle infelicityin ii
can remedied as in iii or in iv
ii John collapsed ying to California
iii John collapsed while ying to California
iv John collapsed trying to run his third marathon in as manydays
In i the matrix attachment of the participial makes only the infelicitous scene setting cotemp oranei ty relation
available so the system is forced to the ECM analysis which for all it can determine online could have a felicitous
ending or a slow to compute metaphorical interpretation
The other p ossible domain over which to de ne information volume is pragmatics whatever
Grice s maxims of quantity and manner are ab out that is informativeness of the contri
bution and brevity prolixity resp ectively If this is the case then constituents are not marked
by the syntactic pro cessor with their information volume All that the syntax determines is
constituent order This constituent order can enco de the commitment that constituentXmust
carry less information than constituent Y The actual information volume is determined bythe
interpreter Such determinations may b e inconsistent with the order based commitments in
which case the analysis is deemed less sensible
Iwould like to suggest that the pragmatic sense is a strong if not an exclusive determinantof
information volume Here is one example
The acceptabilityofverb particle constructions clearly has to do with information volume
a Jo e called the friend who had crashed into his new car up
b Jo e called up the friend who had crashed into his new car
c Jo e called his friend up
d Jo e called up his friend
It has b een widely noted that pronouns are very awkward in p ost verb particle p ositions
a This pissed o him
b This pissed o Bob
The reason I claim for the relative acceptability of b is the accommo dation by the hearer
of the p ossibility of a context which places new information in the NP Bob e g
Mary passed John and Bob in the corridor without even saying hello
Surprisingly this only pissed o Bob John didn t seem to mind
a can b e made acceptable if the pronoun him is replaced by a deictic accompanies by
physical p ointing that is increasing the amount of information asso ciated with the word
When Returning to the central example of this chapter let us consider the dialog in
appropriately intoned this dialog shows that a constituent like that Bill will leave whichis
construed as b earing high information volume when it app ears out of context in b can indeed
b ear low information volume when it expresses a prop osition or concept which is already given
in the discourse
Ford Bresnan and Kaplan p oint out that RA e ects are sensitive to the syntactic category of the more
recent attachment site They contrast i with ii
i Martha noti ed us that Jo e died by express mail
ii Martha noti ed us of Jo e s death by express mail
It is quite clear that the absurd RA reading is more prominent in i than in ii This is rather surprising
b ecause on informational terms I can see no notion of information bywhich that Jo e died b ears any more
information than of Jo e s death
I am grateful to Ellen Prince for this example
A John said that Bill will leavenextweek and that
Mary will go on sabbatical in September
B Oh really When did he announce all this
A He said that Bill will leaveyesterday and he told
us ab out Mary s sabbatical this morning
Conclusion
Ihave argued that the apparentvariability in the applicabilityofRight Asso ciation can b e ex
plained if we consider the information volume of the constituents involved I have demonstrated
that in at least one written genre low information volume adverbials are rarely pro duced after
high volume arguments precisely the con guration which causes the strongest RA typ e e ects
Considering the signi cant in uence of pragmatic content on the degree of information volume
the interaction b etween information volume and constituent order provides a sensibleness based
account for the resolution of a class of mo di er attachmentambiguities
Chapter
Other Constructions
Two often discussed structural ambiguities have not b een mentioned so far
John has heard the jokeiso ensive
When the cannibals ate the missionaries drank
I will refer to the ambiguity in as NP vs S complement and to the garden path e ect in
as the Late Closure E ect the term whichFrazier and Fo dor intro duced
In this chapter I consider the psycholinguistic evidence available ab out these ambiguities and
consider two di erentways of accounting for these and other data The rst prop osal which I call
disconnectedness theory is a formalization of many accounts of pro cessing di culty that app ear
in the literature The second whichIcallAvoid New Subjects has not b een prop osed b efore in
relation to ambiguity resolution I then consider the evidence available for distinguishing these
two accounts ultimately trying to show that disconnectedness theory makes some incorrect
predictions
Available Evidence
NP vs S complement
Advo cates of structural ambiguity resolution strategies have argued that the ambiguityin
is initially resolved by Minimal Attachment
Tom heard the latest gossip ab out the new neighbors was false
and are intuitivel y garden paths One might argue that given the strong bias for jokes b eing heard
and cannibals eating missionaries the structures in and is irrelevant But it is equally plausible that
someone hears some fact and that cannibals engage in an intransitive eating activity so the question remains
of why these strings are resolved as they are
Frazier and Rayner used eyetracking to nd that for sentences such as p eople
slowed down when reading the disambiguation region was false Holmes Kennedy and Murray
used a sub ject paced word by word cumulative display exp erimenttoshow that the
slowdown whichFrazier and Rayner observed p ersists even when the ambiguity is removed by
the intro duction of an overt complementizer With exp erimental materials suchas
TR The maid disclosed the safe s lo cation within the house to the o cer
TC The maid disclosed that the safe s lo cation within the house had b een
changed
RC The maid disclosed the safe s lo cation within the house had b een changed
they found that in the disambiguation region either to the o cer or had been changed the
transitiveverb sentence TR was read substantially faster than the other twosentences The
that complement TC sentence was read slightly faster than the reduced complement RC
sentence
In resp onse Rayner and Frazier ran an an eye movement exp eriment which contradicted
the conclusions of Holmes et al Using materials whichwere similar but not identical
to those of Holmes et al they found that at the disambiguation region TC was read the
fastest followed by TR followed by the ambiguous RC consistent with the theory of Minimal
Attachment
In turn Kennedy Murray Jennings and Reid argued that Rayner and Frazier
intro duced serious artifacts into their eye tracking data bypresenting their material on multi
ple lines and not controlling for the resulting right to left eye movement Kennedy et al also
criticized other technical asp ects of Rayner and Frazier s exp eriment Kennedy et al ran an
eye tracking study using the materials from Holmes et al They found that TC and
RC sentences were read signi cantly slower in the disambiguation region than TR sentences
They found no reliable di erence b etween TC and RC In a further exp eriment to test the e ect
of line breaks they found statistically signi cant e ects whose nature was rather di cult to
interpret They to ok this as evidence that line breaks do indeed intro duce artifacts
In summary there is evidence that S complement sentences TC and RCabove takelonger
to comprehend than comparable NP complementsentences
Another debate is whether RC sentences take longer to read than TC sentences and under what
conditions Quite a few researchers haveinvestigated the question of whether RC sentences cause
a garden path e ect when the matrix verb prefers an NP rather than a clausal complement
Kennedy et al partitioned the materials for their rst exp eriment according to the bias
of the matrix verb NP versus clausal They found no e ects of verb bias on rst pass reading
time but found a statistically non signi cant e ect of verb bias on eye regressions initiated from
the disambiguating zone i e indications of backtracking confusion For b oth kinds of verbs
there were more regressions in the RC condition than the TC condition but the di erence was
greater for NP bias verbs However Kennedy et al did not demonstrate that they accurately
identi ed verb biases
Many groups of researchers rep ort exp eriments sp eci cally designed to investigate verb bias
e ects on the extent of garden path in RCsentences I rep ort the work of four Holmes Stowe
and Cupples Ferreira and Henderson Garnsey Loto cky and McConkie
and Trueswell Tanenhaus and Kello I refer to them as HSC FH GLM and TTK
resp ectively HSC GLM and TTK ran two phase exp eriments In the rst phase sub jects
were asked to complete sentences such as He susp ected Statistics from these data were used
to assess verbs biases Two groups of verbs were selected NP preference and S preference
HSC s criterion for counting a verb as having a bias was a or greater imbalance in sub jects
resp onses When assessing a verb s argument structure they lump ed TC and RC resp onses into
one category GLM and TTK kept separate tallies of these two kinds of complements This
di erence is signi cant b ecause the ambiguity in question is really b etween a TR and an RC
analysis FH did not use a questionnaire instead verbs were selected either on the basis of
normativedatacollectedby Connine Ferreira Jones Clifton and Frazier or according
to the intuitions of the exp erimenters The study of Connine et al asked sub jects to write a
sentence for each of a group of verbs They did not sp ecify that the verb must immediately follow
an agent sub ject and this mighthave certain e ects on the way their data can b e interpreted
for the purp ose at hand
HSC considered the e ects of two factors up on the degree of the garden path e ect verb bias
and plausibili ty of the p ost verb NP as a direct ob ject Their materials were of the form
NP bias verb
TC Plausible The rep orter saw that her friend was not succeeding
Implausible The rep orter saw that her metho d w as not succeeding
RC Plausible The rep orter saw her friend was not succeeding
Implausible The rep orter saw her metho d was not succeeding
Clausal bias verb
TC Plaus The candidate doubted that his sinceritywould b e appreciated
Implaus The candidate doubted that his champagne would b e appreciated
RC Plaus The candidate doubted his sinceritywould b e appreciated
Implaus The candidate doubted his champagne would b e appreciated
They tested the e cacy of the plausibility manipulation by asking sub jects to rate sentences
such as the rep orter saw her metho d This is an inadequate test of the online plausibility
of the NP analysis Just b ecause a sub ject rejects the string as a sentence it do es not mean
that the sub ject would online reject the NP analysis for her metho d doing so would
commit the sub ject dep ending on one s theory of grammar to rejecting strings such as The
rep orter saw her metho d fail miserably when interviewing athletes or The do ctor found the
fever discouraging This criticism applies to the ma jority of their implausible exp erimental
materials though unevenly for the NP and S bias verbs I therefore omit their ndings with
resp ect to this factor
They conducted three self paced exp eriments varying the metho d of presentation of the materi
als Their rst exp eriment used a self paced word byword cumulativedisplay After eachword
the sub ject had to decide whether the string is grammatical so far This resulted in remarkably
slow reading times three times slower than in eye movement exp eriments The RC condition
was slower at the disambiguation region than the TC condition this di erence was enhanced
in the NP bias condition consistent with their theory that lexical information is incorp orated
into the parsing pro cess Advo cates of Minimal Attachment often argue that slow presentation
metho ds may not b e sensitive to rst pass pro cessing and that it is not at all surprising that lex
ical information is incorp orated at the later stage tapp ed by this sort of exp eriment Addressing
this criticism HSC ran another word by word self paced exp eriment but this time sub jects were
required to rep eat the sentence when it was nished This resulted in somewhat faster reading
times still roughly twice as slowasineyemovement The ndings in the second exp eriment
were comparable to those of the rst although the garden path was detected roughly one word
later and the di erences in reading time were not quite as large
One problem with cumulative displays is that sub jects may b e employ a strategy of pressing
the self paced button faster than they are actually reading the words In a third exp eriment
HSC used a non cumulative display where the letters of eachword in the sentence except the
one b eing read were replaced with underscores Instead of manipulating the plausibili tyofthe
ambiguous NP they manipulated its length Two examples are
The lawyer heard that the story ab out the accident was not really true
The rep orter saw that the woman who had arrived was not very calm
For NP bias verbs at the rst word of disambiguating region was in and the RC
condition to ok ms longer than the TC ms versus ms p er word The di erence
between RC and TC was a slightly larger for short NPs but there was no statistically signi cant
interaction b etween NP length and overtness of complementizer
For clausal bias verbs RC sentences with long ambiguous NPs had a reading time of roughly
ms for the disambiguating word whereas the three other conditions i e the two TC conditions
and the short NP RC condition required roughly ms For short NPs the di erence b etween
TC and RCwas not signi cant whereas for long NPs it was the magnitudes were approximately
ms and ms resp ectively
In summary the third exp eriment con rmed the rst twoby showing more pro cessing di culty
for the NP bias verbs than the complement bias verbs In addition it showed that when the
ambiguous NP is long readers tend to interpretitasanargument to the matrix verb even for
complement bias verbs
Ferreira and Henderson attempted to dispute the claim that lexical bias is incorp orated
into the pro cessor s rst pass ambiguity resolution decisions They conducted three exp eriments
using three di erent exp erimental pro cedures eye movement non cumulative self paced read
ing and cumulative self paced reading They used the same materials for all three exp eriments
One example is
NP bias
TC Bill wrote that Jill arrived safely to day
RC Bill wrote Jill arrived safely to day
Clausal bias
TC Bill hop ed that Jill arrived safely to day
RC Bill hop ed Jill arrived safely to day
In their rst two exp eriments FH found no in uence of verb bias on the garden path e ect
between the RC and TC conditions They did nd a weak in uence in the third exp eriment
These results supp ort their claim that Minimal Attachment is relevant for rst pass pro cessing
and lexical prop erties such as argument preferences are considered only in subsequent pro cessing
But there are some serious aws with their exp eriment First they did not demonstrate that
their at least partially intuitively arrived at verb categories do indeed corresp ond to frequency
of use or any other measure of argument structure bias Second many of their examples give
rise to semantic implausibil iti es in the NP reading e g
Jan warned the re
Ed asserted eggs
Ed disputed eggs
These implausibili ty problems a ect the the NP bias materials and S bias materials in di erent
frequencies thus intro ducing a serious p otential source of di culty
Garnsey Lotocky and McConkie conducted one exp eriment where they tested whether
lexical bias information is quickly incorp orated by the pro cessor They used an eye movement
exp eriment with materials suchas
NP bias
TC The manager overheard that the employees were planning a surprise party
RC The manager overheard the employees were planning a surprise party
Clausal bias
TC The manager susp ected that the employees were planning a surprise party
RC The manager susp ected the employees were planning a surprise party
They found a statistically signi cantinteraction b etween complementizer presence and verb bias
This e ect app ears at the rst xation on the rst disambiguating word were
Trueswell Tanenhaus and Kello argued that sub categorization information is incorp o
rated into the analysis at the earliest p oint p ossible and furthermore that the e ect of verb
sub categorization is not categorical but graded re ecting preferences which show up in b oth
pro duction and parsing tasks They used a cross mo dal naming task with materials such as
The old man insisted accepted that
The visually presented targets were he and him TTK found that for TR bias verbs absence of
a complementizer commits the reader to an NP complement analysis as can b e seen by facilita
tion of him relative to he S complementbiasverbs ranged in the e ect of the complementizer
TC bias verbs tended to require the complementizer in order to activate the S complement anal
ysis whereas RC bias verbs showed activation of the S complement analysis even in the absence
of the complementizer TTK found converging evidence from two other exp eriments which used
non cumulativeword by word self paced reading and eye tracking resp ectively with materials
such as
The student forgot hop ed that the solution was in the back of the b o ok
They found garden path e ects as can measured by the slowdown in the disambiguating region
in RC sentences with TR bias verbs For S complementbiasverbs the extent of garden path
dep ended on how frequently the verb app ears with a that complementversus zero complement
TTK used two forms of statistical analysis They used ANOVA to argue for an e ect of TR bias
versus S bias verbs and a regression statistic to argue for a correlation b etween the strength of
complementizer preference and the degree of garden path for the S complementbiasverbs
Ignoring the older p otentially problematic self paced studies where FH s results con ict with
GLM s and TTK s the latter studies are more b elievable FH s failure to nd an e ect of verb
bias could b e due to a variety of factors as discussed at length in Trueswell et al
From the exp eriments listed ab ove I conclude the following
Absence of a complementizer in a RC string can lead to pro cessing di culty
The magnitude of this di culty is often less than that of standard garden path sentences
Some of this di culty mightwell p ersist when the complementizer is present
There is some evidence suggesting that the magnitude of the e ect b ecomes is higher when
the ambiguous NP is long
The magnitude of the di culty is sensitive to the sub categorization p ossibility preference
of the matrix verb
In short the evidence for the strong in uence of lexical factors is clear But there is some
evidence that some pro cessing di culty is residually asso ciated with sentential complements
indep endentofambiguity and lexical factors
Late Closure Ambiguity
Frazier and Rayner argue that Frazier s structural preference principle of Late
Closure is what is resp onsible for the garden path in a
Late Closure
When p ossible attach incoming lexical items into the clause or phrase currently
b eing pro cessed i e the lowest p ossible nonterminal no de dominating the last
item analyzed
a Since Jayalways jogs a mile seems like a short distance to him
b Since Jayalways jogs a mile this seems like a short distance to him
Late closure can similarly account for the pro cessing di culty in and
Without her contributions failed to come in From Pritchett
When they were on the verge of winning the war against Hitler Stalin Churchill
and Ro osevelt met in Yalta to divide up p ostwar Europ e From Ladd
Frazier and Rayner s garden path theory distinguishes two comp onents of the human language
pro cessor which I will followMitchell in calling the Assembler and the Monitor The
Assembler very quickly hyp othesizes a syntactic structure for the words encountered thus far
The Monitor evaluates this hyp othesis and sometimes initiates backtracking when it detects a
semantic problem The Assembler uses quickly computable strategies like Minimal Attachment
and Late Closure Mitchell investigated a prediction of the garden path theory that
the Assembler only pays attention to ma jor grammatical categories of the incoming lexical items
Finer distinctions suchasverb sub categorization frames are only considered in the Monitor
It follows that Late Closure e ects as in should p ersist even when the rst verb is purely
intransitive as in
After the child had sneezed the do ctor prescrib ed a course of injections
Mitchell s used sub ject paced reading time measurement Instead of a word by word pro cedure
eachkeypress would present the next segment of the sentence Segments were fairly large
eachtestsentence was divided into only two segments
As he predicted Mitchell found garden path e ects when the segment b oundary was after the
ambiguous NP the do ctor But this e ect could arise as an artifact of the way he segmented
hsegment as clause To address this criticism his materials leading the reader to construe eac
Mitchell and his colleagues Adams Clifton and Mitchell conducted an eye tracking study
Using materials as in they manipulated the availability of a transitive reading for the verb
and whether or not there was a disambiguating comma after the prep osed clause
After the dog struggled scratched the vet to ok o the muzzle
Their results suggest that when the comma was omitted sub jects attempted to construe the
ambiguous NP the vet as the ob ject of the preceding verb even when it was purely intransitive
In other words the lexical prop ertyofintransitivitywas not as e ectiveasthecommainavoiding
a transitive analysis
Stowe exp eriment provides evidence that directly contradicts Mitchell s claim that
verb sub categorization information is ignored by the rst phase of sentence pro cessing Stowe
exploited the phenomenon of causative ergative alternation exempli ed in to show that
readers are immediately sensitive to the sub categorization frames available for the verb
Just b ecause readers try to make sense of a constituentsuch as after the child had sneezed the do ctor it do es
not mean that they are ignoring sub categorizati on information It is not clear that putatively intransitive verbs
such as sneeze burp sleep are indeed ungrammatical when used transitively or merely lead to implausibi l iti es
It could b e that in a sentence like whatever it is that is resp onsible for the late closure e ect is driving the
interpreter to come up with a transitiveverb interpretation Transitive uses of many putatively intransitive verbs
are not imp ossible consider slept his fare share burp ed YankeeDoodle sneezed her way out of the o ce
sneezed his brains out
Causative John moved the p encil
Ergative The p encil moved
Using materials such as those in Stowe manipulated the plausibili ty of the sub ject as causal
agent e ectively changing sub categorization frame of the verb
Ambiguous
Animate Before the p olice stopp ed the driver was already getting nervous
Inanimate Before the truck stopp ed the driver was already getting nervous
Unambiguous
Animate Before the p olice stopp ed at the restaurant the driver was already
Inanimate Before the truck stopp ed at the restaurant the driver was already
Late Closure predicts that in the ambiguous conditions the disambiguating word was in
should not vary when the animacy of the sub ject is manipulated Stowe s account of early use
of lexically sp e ci ed information predicts a garden path e ect at was only in the ambiguous
animate condition And this is exactly what she found Using a sub ject paced word by word
cumulative display task where sub jects were required to monitor the grammaticalityofthe
string she found signi cantly elevated reading times and ungrammatical resp onses at the rst
disambiguating word in the ambiguous animate condition and nowhere else The exp erimental
technique used by Stowe has often b een criticized as to o slow for detecting rst pass pro cesses
But I am aware of no exp eriments whichchallenge Stowe s result
ween lexical preferences and In a followup exp eriment Stoweinvestigated the interaction b et
plausibility She used materials such as those in
Animate
Plausible When the p olice stopp ed the driver b ecame very frightened
Implausible When the p olice stopp ed the silence b ecame very frightening
Inanimate
Plausible When the truck stopp ed the driver b ecame very frightened
Implausible When the truck stopp ed the silence b ecame very frightening
She used the same pro cedure as in her rst exp eriment a sub ject paced word by word cumu
lative display task where sub jects were required to monitor the grammaticality of the string at
eachword Aside from the animacy e ect observed in her rst exp eriment Stowe found that
the implausibil ity of the sub ject NP silence in to serve as an ob ject of the preceding
verb is noted as so on as the word itself app ears p Stowe also observed The most
p erplexing p oint ab out the results of Exp eriment is that p eople apparently b ecome aware of
the unsuitability of the NP to b e an ob ject of the preceding verb even when there is evidence
that they exp ect an intransitiveverb structure i e in the Inanimate conditions p
In summary While the issue of whether verb sub categorization information comes to b ear imme
diately on resolving the late closure ambiguity is not de nitively settled the available evidence
suggests that it do es Nevertheless there is still evidence for some residual e ects Late Closure
and preference for NP complements over S complements that lexical prop erties alone cannot
account for Below is additional evidence for this claim
Consider
John nally realized just how wrong he had b een remained to b e seen
The main verb realize is biased toward a sentential complement yet there is still a p erceptible
garden path
Lexical bias alone also fails to account for all of the late closure e ect In
When Mary returned her new car was missing
the verb return o ccurs more frequently without an ob ject than with one Nor can lexical
bias account for the garden path sentences and rep eated here as and
Without her contributions failed to come in
When they were on the verge of winning the war against Hitler Stalin Churchill
and Ro osevelt met in Yalta to divide up p ostwar Europ e
In the rest of this chapter I investigate two theories to account for these preferences
Degree of Disconnectedness
One idea that has b een recently put to use by Pritchett and Gibson but go es
back at least as far as Eady and Fo dor and Marslen Wilson and Tyler is that the
Verb data from ve sources con rm this
NP RC TC RC TC units
Trueswell et al in completion task
Garnsey et al in completion task Garnsey p c
Connine et al frequency in questionnaire
Brown corpus raw frequency
Wall Street Journal corpus raw frequency
The verb return o ccurs in the Brown and Wall Street corp ora as follows
corpus transitive intransitive
Wall Street Journal
Brown
Note that while the verb return has b oth an intransitive and a transitive sub categorization frame it is
di erent from a verb like eat which is transitive but may drop its ob ject It may b e the case that ob ject drop
uses require a pro cess of accommo dating an implicit ob ject While di culties with this pro cess could p otentially
account for the garden path in they cannot account for a garden path in
pro cessor has di cultykeeping around many fragments for whichitdoesnotyet have semantic
connections and thus prefers b etter integrated analyses In this section I give one formalization
of this idea and showhow it can account for many di erent pieces of data including those just
discussed
The basic notion here is degree of disconnectedness Intuitively the disconnectedness measure
of an analysis of an initial segmentofasentence is howmany semantically unrelated pieces have
b een intro duced so far In standard syntactic theory disconnectedness has a straight forward
implementation
Theta Attachment Pritchett
The theta criterion attempts to b e satis ed at every p oint during pro cessing given
the maximal theta grid
The theta criterion is part of the comp etence theory which assigns every verb and other op en
class complement taking words such as adjectives and nouns a collection of thematic slots
called theta roles For a sentence to b e well formed every theta role must b e lled byan
argument and every argumentmust ll a particular slot It turns out that thematic roles
are not rich enough to capture the necessary semantic relations among words in a sentence
esp ecially when their semantic content e g agent instrument is ignored So Pritchett
broadens his heuristic to include every principle of syntax not just the theta criterion Gibson
eitwork with his parsing algorithm op erationalizes in a slightly di erentwaytomak
and data representation and notes that anysyntactic theory that mentions thematic relations
would give rise to a similar parsing heuristic In this pro ject I cast the notion of disconnectedness
minimization in purely semantic i e non syntactic terms I do not distinguish the semantic
relation of thematic role from any other semantic relation such as determiner noun or mo dal
verb etc This notion of disconnectedness will b e made more concrete presently But rst I
intro duce the semantic representation formalism which I will use in this dissertation
A Representation of Semantics
For the purp oses of the present pro ject the semantic representation whichIcho ose is b orrowed
from the work of Hobbs and his colleagues Hobbs Hobbs Stickel App elt and Martin
which is in turn an elab oration of work byDavidson Davidson argues that events
can b e talked ab out just likephysical ob jects so a logical form must include eventvariables
as well as the traditional thing variables Hobbs argues that predications e g states
must also b e a orded this treatment as rst class memb ers of the ontology The semantic
representation which he prop oses is not the usual term or logical formula but rather a set of
terms each comprising a predicate symb ol and one or more arguments which are either variables
or constants but crucially not terms themselves All variables are implicitly existentially
quanti ed For example the semantic representation for is
The b oywanted to build a b oat quickly
e e e x yPast e w ant e x e quick e e buil d e x y boy x
boat y
Which means something like
There is an event state e which o ccurred in the past in which the entity x wants
the event state e e is an event state in which the event state e o ccurs quickly
e is a building event in which x build y wherex is a b oyandy is a b oat
Hobbs motivates this at representation on the grounds that it is simpler and thus
enco des fewer commitments in the level of the semantics This is sup erior to hierarchical
recursively built representation he argues as semantic representation is di cult enough as it
is without additional requirements that it cleverly account for certain syntactic facts as well
e g count nouns vs mass nouns He defends the viability of this approachby showing that it
can cop e with traditional semantic challenges such as opaque adverbials de dicto de re b elief
rep orts and identity in b elief contexts This notation is used in the tacitus pro ject Hobbs
et al a substantial natural language understanding system demonstrating its
k exploits the simple structure of viability as a meaning representation Haddo c
each term to p erform e cientsearch of the representation of a prior discourse in order to resolve
de nite NPs
In the current pro ject a lexicalized grammar is used where eachword is asso ciated with a
combinatory p otential and a list of terms When words constituents are combined their term
lists are simply app ended to determine the term list of the combined constituent Details and
examples will b e given in section The semantic analysis then develops incrementally
word by word
AFormal De nition
To formally de ne the degree of disconnectedness of a semantic analysis S I rst construct an
undirected graph whose vertices are the variables b oth ordinary thing variables and event state
variables mentioned in S Twovertices are adjacent have an edge connecting them just in case
they b oth app ear as arguments of a term in S The disconnectedness measure of S is the number
of comp onents of the graph minus By number of components I mean the standard graph
theoretic de nition twovertices are in the same comp onent if and only if there is a path of
edges that connects them
For example when the initial segment in is encountered
When the cannibals ate the missionaries
there are two analyses corresp onding to the transitive and intransitive readings of ate resp ec
tively
when e e eat e e e de nite e cannibals e de nite e missionar ies e e1 e2
e3 e4
when e e eat e e de nite e cannibals e de nite e missionaries e e1 e2
e3 e4
Since the intransitive reading carries a higher disconnectedness measure than the transitive
reading vs the transitive reading is preferred A Late Closure E ect therefore results
when the next word is drank I use the capitalized term Disconnectedness to refer to the
theory that the pro cessor prefers to minimize the measure of disconnectedness
Disconnectedness similarly accounts for the garden path e ects in At the word contribu
tions there are two analyses corresp onding to the common noun and NP readings resp ectively
without e e feminine e of e e contributions e
without e e feminine e implicit quanti e r e contributions e
The common noun reading is thus preferred
Consequences
Disconnectedness predicts di culty with In fact since Disconnectedness is insensi
tive to lexical or conceptual preferences its input to the analysis selection pro cess could con ict
with the input from lexical preferences This con ict can account for the puzzling ndings of
Stowe s second exp erimentabove
The ndings of Holmes et al and Kennedy et al that in sentences suchas
ab ove rep eated here as the clausal conditions TC and RC are slower to read than TR are
also consistent with the additional disconnectedness asso ciated with the sub ject reading of the
safe s lo cation within the house
TR The maid disclosed the safe s lo cation within the house to the o cer
TC The maid disclosed that the safe s lo cation within the house had b een
changed
RC The maid disclosed the safe s lo cation within the house had b een changed
Additional evidence in supp ort of Disconnectedness comes from exp eriments with lling WH
gaps Boland Tanenhaus Carlson and Garnsey investigated whether plausibili ty a ects
gap lling They used materials such as those in and a sub ject paced word by word cumu
lative display metho d where sub jects were asked to detect when the sentence stopp ed making
sense They found that the word them caused di culty in a as compared to its control
b
a Whichchild did Mark remind them to watch this evening
b Whichmovie did Mark remind them to watch this evening
This is a placeholder for a semantic theory of bare plurals
Boland et al conclude from this and other exp eriments that inferential information suchasthe
argument structure of the verb are used as so on as logically p ossible Let us examine closely
what happ ens with these two sentences In b when the reader comes to the word remind
s he can check whether movies can b e reminded Since that is implausible the remindee sp ot
is not lled and the next word them causes no di culty In b a child is something that
can b e reminded so the gap lling analysis is pursued But the non lling alternatives is just as
plausible A p erson can remind someone of something having to do with children Plausibility
alone cannot fully explain why the lling analysis is preferred in this case Note that the non
lling analysis has a higher disconnectedness measure the relation b etween the WH element
and the rest of the material in the utterance is not established Without Disconnectedness
one need a partially structurally based theory such as rst plausible gap to account for this
gap lling b ehavior
The interpretation of the results of Holmes et al and Kennedy et al as dis
connectedness related pro cessing di culty in the unambiguous TC condition suggest that there
might b e other unambiguous highly disconnected structures which are hard to pro cess Indeed
center emb edding the classical example of an unambiguous structure that is hard to pro cess
reaches a disconnectedness measure of after the word dog
The rat that the cat that the dog de nite e rat e de nite e cat e de nite e dog e e1
e2 e3
The rat that the cat that the dog bit chased died
de nite e rat e de nite e cat e de nite e dog e
bite e e e past e chase e e e past e die e e past e e5e1 e6
e2 e3
e4
It seems quite likely that the computations of pro cessing load whichHawkins uses to
derive manyword order universals could b e recast in terms of disconnectedness score Rambow s
a b account of marginally grammatical scrambled sentences in German in terms of storage
requirements is also very likely to b e statable in terms of disconnectedness But this awaits
further research
Given that disconnectedness related pro cessing di culties such as the Late Closure E ect
are mitigated and often overridden by inferential preferences one would exp ect pro cessing
di culties with structures such as to b e ameliorated by b etter semantic coherence In
fact this prediction is b orne out Bever hyp othesized that might b e easier to pro cess
than
The dog that the destruction that the wild fox pro duced was scaring will run
away fast
Fo dor Bever and Garrett and Frazier and Fo dor mention and resp ec
tively which seem somewhat easier to understand than
The water the sh the man caughtswam in was p olluted
The snow that the match that the girl lit heated melted
Frank provides which seems to do away with pro cessing di culty altogether
A b o ok that some Italian I venever heard of wrote will b e published so on by
MIT press
Inferential and discourse factors are clearly involved in the degree of di culty of these sentences
For example note that having a deictic as the most deeply emb edded sub ject as in seems
to improve things somewhat and replacing the de nite sub jects in with inde nites
in seems to make a further improvement The interaction b etween these interpretive factors
and pro cessing di culty in the absence of ambiguity remain matters for further research as do
the subtle e ects of the choice of relativizer that vs who whom which vs zero
The connection b etween ambiguity resolution preferences for semantically b etter integrated
readings and pro cessing di culties with center emb edding has b een explored by Gibson
While Gibson s measure of semantic integration is formulated in terms of the Governmentand
Binding principles e g the Criterion see Chomsky and not graph theoretic notions
his prop osed underlying mechanisms are comparable to the account o ered here analyses
in which semantic relations among entities are established are preferred to analyses in which
they are not Gibson opts for a di erent explanation for the relativeimprovement of over
He assumes that each of through and presumably overwhelms the parser s
capacity and causes a breakdown of ordinary syntactic pro cessing In sentences like the
interpretive mo dule is still able to piece the uncombined fragments together using inferential
pro cesses such as determinations of plausibili ty This sort of inference cannot salvage a sentence
such as Gibson s account predicts that in deeply emb edded structure where the parser
breaks down if there is a choice b etween syntactic ill formedness and inferential implausibil i ty
the former will b e opted for by the inferential salvaging pro cess For example the string
Whatever manipulati ons one applies to to make it as go o d as can b e applied in reverse to cause
center emb edding typ e pro cessing di culty for structures which are usually considered unproblematical Consider
i which Gibson following Cowp er takes to b e unproblematical
i The p ossibili ty that the man who I hired is incomp etentworries me
Replacing the deictic pronouns with a de nite NPs renders the resulting sentence ii harder to understand
ii The p ossibil ity that the man who the executive hired is incomp etentworries the sto ckholders
Some Italian that a b o ok I venever heard of wrote will b e published so on by
MIT press
is predicted by his account to b e judged as acceptable or at least signi cantly b etter than
and construed as meaning the same thing as
There is no necessary connection b etween center emb edding and disconnectedness is just
as center emb edded as but do es not encounter disconnectedness at any p oint
John asked the woman that gave the b oy that kicked the dog some candy why
she s sp oiling him
Intuitively is slightly easier to read than a variant whose structure directly mirrors
that of
The woman that the b oy that the dog frightened is b othering rightnowisa
friend of John s
Eady and Fo dor rep ort an exp erimentinwhich they indep endently manipulated two
relative clauses one contained in the other for center emb edding versus right branching
They found that a and b were of comparable reading di culty c was substantially
harder to read than b and d was harder yet The largest di erence was b etween b
and c That is when the innermost relative clause is center emb edded the di cultyis
greatest Their results argue against an account of pro cessing di culty which is in their words
based on inherent prop erties of center emb edding Descriptively what matters most is whether
or not the ller gap dep endencies overlap Disconnectedness theory captures this nding the
maximum disconnectedness scores for a through d are and resp ectively
a Jack met the patient the nurse sent e to the do ctor the clinic had hired e
i i j j
to the do ctor the clinic had hired e met Jack b The patient the nurse sent e
j j i i
c Jack met the patient the nurse the clinic had hired e sent e to the do ctor
i j j i
d The patient the nurse the clinic had hired e sent e to the do ctor met Jack
i j j i
underlining depicts ller gap dep endencies
I assume that the string in is somehow derivable byacombination of scrambling op erations which op erate
in other languages but cannot b e ruled out for this English sentence b ecause the comp etence grammar is b eing
ignored
To corrob orate myintuitions I conducted a miniature survey of six colleagues I presented them with sentences
through
John asked the woman that gave the b oy that kicked the dog some candy why she s sp oiling him
John asked the woman that gave the b oy that the dog frightened some candy why she s sp oiling him
The woman that the b oy that the dog frightened is b othering rightnow is a friend of John s
The woman that the b oy that kicked the dog is b othering rightnow is a friend of John s
Their maximum disconnectedness measures are and resp ectively Everyone I asked initially rated all
sentences as equally bad After some b egging and coaxing on my part each informantprovided some partial
ranking All resp onses were consistent with the ranking from b est to worst and only this ranking
This is consistent with the predictions of Disconnectedness
A Disconnectedness based account of the di cultyof would predicts that should b e
completely free of anycenter emb edding typ e pro cessing di culty and that an even more deeply
nested structure should b e as easy to pro cess as its purely right branching control in
This do es not seem to b e the case more research is needed
John asked the woman that o ered the b oy that gave the dog that chased the
cat a big kick some candy why she s sp oiling him
John met the woman that rewarded the b oy that kicked the dog that chased the
cat
In summary the strategy of minimizing the measure of disconnectedness has a variety of evidence
to supp ort it
residual Late Closure E ects
residual NP preference for NP vs S ambiguities
gap lling
pro cessing di culty in unambiguous temp orarily disconnected sentences
But would adoption of Disconnectedness weaken the overall thesis After all Disconnectedness
is stated over the sense semantics of a string a level of representation which is on the interface
of syntax and interpretation It is quite conceivable that one could prop ose a notational variant
of disconnectedness theory which is stated solely in terms of structure After all its theoretical
predecessors Pritchett and Gibson s prop osals are based on thematic role assignment
in syntactic structure Nevertheless I claim that Disconnectedness is a viable candidate for a
comp onent of the thesis of ambiguity resolution from interpretation It is stated over the domain
of meaning not syntactic structure As is suggested by the susceptibility of disconnectedness to
discourse factors e g the lo cus of disconnectedness might not b e the sense semantics as I
de ned it but a level of meaning representation which is deep er more pragmatic
Another p otential ob jection is why should a temp orarily high disconnectedness measure matter
to the pro cessor Given that no complete grammatical sentence has any disconnectedness the
pro cessor can just patiently wait until the connecting words arrive There are two resp onses
to this ob jection First a pro cessor that waits for additional information b efore making its
decision might require large computational resources when faced with comp ounding ambiguity
i e waiting might b e to o exp ensive Second a pro cessor mightwell b e closely attuned to
disconnectedness since the very task of a sentence understanding system is to determine the
logical connection among the words in the sentence the b etter the connection the more
preferred the analysis It would follow that some connection is preferable to no connection
Inow turn to a drastically di erentaccount for most of the data in this section
Avoid New Sub jects
An examination of the syntactic structures that disconnectedness accounts for reveals that with
one exception they all involve a preference not to analyze an NP as a sub ject
a late closure e ects
When the cannibals ate the missionaries drank
Without her contributions failed to come in
When they were on the verge of winning the war against Hitler Stalin
Churchill and Ro osevelt met in Yalta to divide up p ostwar Europ e
b NP Preference for NP vs S complementambiguity
John has heard the joke is o ensive
c sub ject relative clause center emb edding
The rat that the cat that the dog bit chased died
d gap lling
Whichchild did Mark remind them to watch this evening
The one exception is gap lling The so called l led gap e ect Crain and Fo dor which
readers exp erience in d at the word them tends to b e less severe than the other garden
path e ects discussed in section
In a second set of exp eriments Boland et al presentintriguing evidence that the pro
cessing di culty in d is not of the same sort as the other garden path e ects in section
Using the same sub ject paced word by word cumulative display stop making sense task that
they used in their rst exp eriment describ ed on page ab ove they investigated the e ect of
the plausibili ty of the WH ller on reading time For materials as in
a Bob wondered whichbachelor Ann granted a maternityleave to this after
no on
b Bob wondered which secretary Ann granted a maternit y leave to this after
no on
they found that sub jects were able to detect the anomaly in a starting with the word leave
that is b efore the prep osition to could trigger the construction of the phrase whichcontains the
gap p osition This suggests that certain pragmatic integration pro cesses o ccur b efore b ottom up
syntactic evidence is available to tell the pro cessor that a gap is present
It follows from this nding that encountering the unexp ected NP them in d is o dd not
just syntactically but also pragmatically Indeed Boland p c rep orts varying strengths
of lled gap e ects for di erent lexical realizations of the surprising NP e g pronoun prop er
name inde nite NP de nite NP suggesting that inference and accommo dation might b e in
volved The di erence b etween a lled gap e ect and a garden path e ect is then in the pro
cessing comp onent in which they are detected a garden path is detected when the syntactic
pro cessor discovers that none of the analysis that it is currently maintaining can b e extended
with the currentword This condition results b ecause the necessary analysis was discarded
earlier A lled gap e ect on the other hand is initially detected in the interpreter not the syn
tactic pro cessor When the surprising NP app ears the interpreter has not yet told the syntactic
pro cessor to commit to the lled gap analysis
With lled gap e ects now eliminated from the collection of garden path data that Disconnect
edness is relevant for Disconnectedness is indistinguishable on the remaining examples from a
preference for avoiding treating an NP as a sub ject This is a very strange preference to have
in a pro cessor whose purp ose it is to understand sentences given that every sentence has a
sub ject Perhaps all of the sub jects in the examples in section are somehow sp ecial and the
prohibition is not on all sub jects only on this sp ecial sort of sub ject In this section I argue that
this is indeed the case All of the sentences were presented out of context and it is subjects
that are new to the discourse that the pro cessor seeks to avoid It must b e emphasized that
Avoid New Sub jects makes no primary distinctions b etween de nite and inde nite NPs Out of
context b oth are new to the discourse In context de nites tend to b e given more frequently
but de niteness is not a de ning characteristic of givenness
Given and New
Prince prop oses a classi cation of o ccurrences of NPs in terms of assumed familiarity
When a sp eaker refers to an entity which s he assumes salient familiar to the hearer s he tends
to use a brief form such as a de nite NP or a pronoun Otherwise the sp eaker is obliged to
provide the hearer with enough information to construct this entity in the hearer s mind Prince
classi es the forms of NPs and ranks them from given to new
evoked An expression used to refer to one of the conversation s participants or an entity which
is already under discussion usually a de nite NP or pronoun
unused A prop er name which refers to an entityknown to the sp eaker and hearer but not
already in the present discourse
tro duces an entity not already in the discourse but which is easily inferable A phrase whichin
inferred from another entity currently under discussion c f bridging inference of Haviland
and Clark
containing inferable An expression that intro duces a new entity and contains a reference to
the extant discourse entity from which the inference is to pro ceed e g One of the p eople
that work with me b oughtaToyota
brand new An expression that intro duces a new entity which cannot not b e inferentially related
or predicted from entities already in the discourse
Prince constructs this scale on the basis of scale based implicatures that can b e drawn if a
sp eaker uses a form which is either to o high or to o low such a sp eaker would b e sounding
unco op erative cryptic or needlessly verb ose resp ectively
By sub ject I refer solely to canonical pre verbal sub jects and not to the broader class of grammatical sub ject
whichmay include existential there sentences and V constructions such as Outside sto o d a little angel inter
alia
Using this classi cation Prince analyzed two texts the rst is an informal chat and the
second formal scholarly prose Her ndings are summarized in the following table
sp oken written
sub ject non sub ject sub ject non sub ject
Evoked
containing Inferable
New unused and brand new
In b oth genres there is a clear tendency to make sub jects more given If we construe this
tendency as resulting directly from a principle of the linguistic comp etence which calls for using
sub ject p osition to enco de given information wewould indeed exp ect a reader to prefer to treat
out of context NPs as something other than sub jects I refer to this principle as Avoid New
Subjects
Consequences
The theory of Avoid New Sub jects predicts that for ordinary text sp oken or written the Late
Closure E ect and the residual NP preference for NP vs S ambiguities should disapp ear I now
present corpus based investigations of these two predictions in turn
Late Closure and Avoid New Sub jects
To test the prediction that Late Closure E ects should disapp ear when the sub ject is given I
conducted a survey of the bracketed Brown and Wall Street Journal corp ora for the following
con guration a VP which ends with a verb and is immediately followed byanNP Crucially
no punctuation was allowed b etween the VP and the NP I then removed by hand all matches
where there was no ambiguity e g the clause was in the passive or the verb could not takethe
NP as argument for some reason Here are the remaining matches preceded by a bit of context
and followed by illustration discussion of the ambiguity
An article ab out a movie describ es how its comp oser approached one of the singers When
you approach a singer and tell her you don t want her to sing you always run the risk of
o ending
You don t w ant her to sing you a song
From the way she sang in those early sessions it seemed clear that Michelle Pfei er had
b een listening not to Ella but to Bob Dylan There was a pronunciation and approach that
seemed Dylan in uenced recalled Ms Stevens Vowels were swallowed word endings
were given short or no shrift When weworked it almost b ecame a joke with us that I
was constantly reminding her to say the consonants as well as the vowels
When weworked it out
After the crash and as a result of the recommendations of many studies circuit
breakers were devised to allowmarket participants to regroup and restore orderly market
conditions It s doubtful though whether circuit breakers do any real go o d In the
additional time they provide even more order imbalances might pile up as would b e sellers
nally get their broker on the phone
Even though this example involves gap lling the fact remains that the NP even more
order imbalances could b e initially construed as a dative as in In the additional time
they provide even the slowest of traders problems could
article is ab out the movie The Fabulous Baker Boys Preceding paragraphs describ e
the actors and movie in generalities When the movie op ens the Baker brothers are doing
what they ve done for years professionally and twice as long as that for themselves
They re playing pro cient piano face to face on twin pianos
The movie op ens the Baker brothers to criticism from
Jonathan Lloyd executive vice president and chief nancial o cer of Qintex Entertain
ment said Qintex Entertainmentwas forced to le for protection to avoid going into
default under its agreement with MCA The million paymentwas due Oct and the
deadline for default was Oct Mr Lloyd said if Qintex had defaulted it could have
b een required to repay million in debt under its loan agreements
Both Webster s and American Heritage Dictionary classify the verb default as b oth
transitiveandintransitive None of the o ccurrences of default in a larger corpus of
Wall Street Journal text take an NP complement
What s more the U S has susp ended million in military aid and million in economic
aid to Somalia But this is not enough Because the U S is still p erceived to b e tied to
Mr Barre when he go es the runway could go to o
There are many transitive uses of go in the corpus go a long way a step further a full
seven games gol ng town watching home nuts hand in hand
Butch McCarty who sells oil eld equipment for Davis To ol Co is also busy A nativeof
the area he is backnow after riding the oil eld b o om to the top then surviving the bust
running an Oklahoma City convenience store First year I came back there wasn t any
work he says I think it s on the waybacknow
First year I came back there I nearly
Story ab out the winning company in a comp etition for teenage run businesses its pres
ident Tim Larson and the organizing entity Junior Achievement For winning Larson
will receive a U S Savings Bond from the Junior Achievement national organization
winning Larson over to their camp
Why did the Belgians grant indep endence to the Congo a colony so manifestly unpre
pared to accept it Yet there were other motivations for which history may not nd
them guiltless paragraph break
As the time for indep endence approached there were in the Congo no fewer than
p olitical parties or approximately eight for each university graduate
As the time for indep endence approached there the p eople
Science has simply left us helpless and p owerless in this imp ortant sector of our lives
spirituality
paragraph break
The situation in whichwe nd ourselves is brought out with dramatic force in Arthur
Miller s play The Crucible which deals with the Salem witch trials As the play op ens the
audience is intro duced to the community of Salem in Puritan America at the end of the
eighteenth century
the play op ens the audience up to new
b o dybuildin g advice exp erimenting with a particular technique Oh you ll wobble and
weave quite a bit at rst But don t worry Before your rst training exp eriment has
ended there will b e a big improvement and almost b efore you knowityou ll b e raising and
lowering yourself just likeaveteran
Before your rst training exp eriment has ended there in the ro om you ll know
The givenness status of the ambiguous NPs is as follows
match NP givenness status
you evoked
it pleonastic
even more order imbalances brand new
Summary
The Baker brothers evoked
pleonastic
it evoked
evoked
the runway evoked
inferable
there pleonastic
brand new
Larson evoked
there pleonastic
the audience inferable
there pleonastic
Prince s givenness scale do es not include pleonastic NPs since they do not refer For the present
purp ose it su ces to note that Avoid New Sub jects do es not rule out pleonastics While the
numb ers here are to o small for statistical inference the data suggest that the prediction of
Avoid New Sub jects is maintained
If one had to guess the p erceived givenness status of pleonastic considering their tendency cross linguisti cal ly
to b e homophonous with pronouns and deictics one would guess that they are treated as given
Given the high frequency of given sub jects optionally transitiveverbs and fronted adverbials one might
exp ect more matches in a two million word corpus But examination of the Wall Street Journal corpus reveals
that most fronted adverbials are set o by comma regardless of p otential ambiguity Of sentence initial
adverbials only are not delimited by comma Of these adverbials have the category
SBAR of which only are not delimited by comma The great ma jority of fronted adverbials
have category PP of which are not delimited by comma The high frequency of the comma therefore
has the e ect of signi cantl y shrinking the available corpus of relevant examples
It must b e emphasized that these ndings are just suggestive Just b ecause a particular sentence app ears in a
newspap er it do es not mean that that it did not cause the pro ofreader to garden path This is esp ecially true of
sentence ab ove which causes some readers to garden path The only way to really test the currenthyp othesis
is using carefully constructed minimal pairs
Avoid New Sub jects also provides an account for the p erplexing results of the second exp eri
ment rep orted byStowe as discussed in the b eginning of section Recall that Stowe
used materials suchas
Animate
Plausible When the p olice stopp ed the driver b ecame very frightened
Implausible When the p olice stopp ed the silence b ecame very frightening
Inanimate
Plausible When the truck stopp ed the driver b ecame very frightened
Implausible When the truck stopp ed the silence b ecame very frightening
For the animate condition one exp ects an e ect of implausibili ty at the critical NP the silence
b ecause the reader is using the causative analysis of stopp ed Given the evidence from her
rst exp eriment using sentences like the inanimate plausible in that inanimate sub jects
cause readers to adopt the ergative analysis one would not exp ect the reader to consider the
ob ject analysis of the critical NP for inanimate conditions But this is exactly what Stowe
found implausibil ity e ects for the inanimate condition which mirrored those for the animate
condition
To resolve this paradoxical ndings one must maketwo observations First while the inan
imate sub ject truck indeed rules out a causative analysis for the verb stopp ed it do es
not necessarily rule out all other transitive analyses In particular stopp ed allows a third
sub categorization frame the so called instrumental
Causative John moved the p encil
Ergative The p encil moved
Instrumental The p encil moved the pap er
Unlike the ergative the sub ject of an instrumental is not the patient a ected ob ject The
name is somewhat of a misnomer b ecause in examples such as the instrumental sub ject
might not b e serving as an instrumentofan y causal agent
The sleet stopp ed the parade short
The second observation is that in the inanimate plausible condition the givenness status of the
critical NP the truck is inferable In the inanimate implausible condition it is brand new
more new in Prince s terms than an inferable
Both of these observation are true of the great ma jority of the exp erimental materials in Stowe
Given the availability of a transitiveverb analysis of the inanimate conditions and given
the tendency to avoid new sub ject a tendency whichislikely to b e sensitive to the degree of
newness it is no longer surprising that readers chose the ob ject analysis for the critical NP in
the inanimate implausible condition The presence of the instrumental analysis did not matter
Theological arguments to the contrary notwithstanding
for the rst exp eriment where the critical NPs were plausible and crucially inferable not
so new as to drive the pro cessor to the ob ject NP analysis
Disconnectedness theory is not conditioned on discourse status and cannot simultaneously ac
count for the plausibility e ects in the inanimate conditions of exp eriment and the lackof
garden path e ects in the ambiguous conditions of exp eriment Itwould have to b e restated
over representations which distinguish unrelated entities from those which can b e related by
means of bridging inferences
In order to decide b etween Disconnectedness and Avoid New Sub jects wemay b e able to
combine results from twovery di erent exp eriments using the following reasoning While Dis
connectedness theory makes predictions for gap lling Avoid New Sub jects do es not To falsify
Disconnectedness one could show that Disconnectedness acts irreconcilably di erently when
driving gap lling than it do es when driving late closure e ects One wayofcharacterizing a
preference is how strong it is compared to another one in this case plausibility Recall the
exp eriment of Boland Tanenhaus Carlson and Garnsey discussed on page Using
examples such as Boland et al argued that gaps are lled unless implausibili ty results
a Whichchild did Mark remind them to watch this evening
b Whichmovie did Mark remind them to watch this evening
It follows that for gap lling decisions Disconnectedness is not as strong a factor as Plausibil
ity Stowe s second exp eriment on the other hand suggests that Disconnectedness or Avoid
New Sub jects is su ciently strong so as to override Plausibility Of course to b e convincing
e Stowe s second exp erimentmust b e rep eated with materials which completely rule out transitiv
instrumental readings in the inanimate conditions For example
While the cakewas baking the oven caught re
As the plot unfolds the reader is ushered into a world
Complement Clauses
In order to b e relevant for the ambiguity in b Avoid New Sub jects must b e applicable
not just to sub jects of ro ot clauses but also to emb edded sub jects It is widely b elieved that
constituents in a sentence tend to b e ordered from given to new The statistical tendency to avoid
new sub jects may b e arising solely as a consequence of the tendency to place new information
toward the end of a sentence and the grammatically imp osed early placement of sub jects If
this were the case that is Avoid New Sub jects is a corollary of Given Before New then Avoid
New Sub jects would make no predictions ab out sub jects of complement clauses as these are
neither at the b eginning nor at the end of sentence utterance In this section I argue that it
is the grammatical function of sub jects not just their linear placement in the sentence that is
involved with the avoidance of new information
When a sp eaker writer wishes to express a prop osition whichinvolves reference to an entity not
already mentioned in the discourse s he must use a new NP S he is quite likely to avoid placing
this NP in sub ject p osition To this end s he may use constructions such as passivization there
insertion and clefts It is often observed that sp eakers tend to use structures like b in order
to avoid structures like a
a A friend of mine drives a Mercedes
b I have a friend who drives a Mercedes
The theory of Avoid New Sub jects predicts that this sort of e ort on b ehalf of writers should
b e evident in b oth ro ot clauses and complement clauses To test this prediction I conducted
another survey of the Penn Treebank I compared the informational status of NPs in sub ject and
non sub ject p ositions in b oth ro ot and emb edded clauses as follows I de ned sub ject p osition
as an NP immediately dominated by S and followed not necessarily immediately toallow for
auxiliaries punctuation etc byaVP I de ned non sub ject p osition as an an NP either
immediately dominated by VP or immediately dominated by S an not followed not necessarily
immediately byVP To determine givenness status I used a simple heuristic pro cedure
to classify an NP into one of the following categories empty category pronoun proper
name definite indefinite not classified The observed frequencies for the bracketed
Brown corpus are as follows
ro ot clause emb edded clause
sub j non sub j sub j non sub j
empty category
pronoun
proper name
definite
indefinite
not classified
total
All pronouns are either pleonastic or evoked they are thus fairly reliable indicators of given
at least non new NPs The category indefinite contains largely brand new or inferable NPs
thus b eing a go o d indicator of new information Considering pronouns and indefinites there
is a clear e ect on grammatical function for b oth ro ot clauses and emb edded clauses
ro ot clause emb edded clause
sub j non sub j sub j non sub j
pronoun
indefinite
p p
The prediction of Avoid New Sub jects is therefore veri ed
As remarked earlier in this section when a hearer reader is faced with an initial segmentsuch
as
John has heard the joke
the ambiguity is not exactly b etween an NP complementanalysisversus an S complement anal
ysis but rather b etween an TR transitiveverb analysis and an RC reduced S complement It
I am grateful to Rob ert Frank for helpful suggestions regarding this pro cedure
For clarity I only give results from the Brown corpus but all assertions I make also hold of the Wall Street
Journal corpus App endix A contains data for b oth corp ora
is therefore necessary to verify that Avoid New Sub jects is indeed op erating in this RC sub class
of sentential complements A further analysis reveals that this is indeed the case
TC RC
sub j non sub j sub j non sub j
empty category
pronoun
proper name
definite
indefinite
not classified
total
TC RC
sub j non sub j sub j non sub j
pronoun
indefinite
p p
If anything Avoid New Sub jects has a stronger e ect after a zero complementizer
Unambiguous Structures
The consequences of Avoid New Sub jects on unambiguous structures suchas
TR The maid disclosed the safe s lo cation within the house to the o cer
TC The maid disclosed that the safe s lo cation within the house had b een
changed
RC The maid disclosed the safe s lo cation within the house had b een changed
from Holmes et al and center emb edding are remarkably similar to those of Disconnect
edness theory When presented out of context TC requires the reader to accommo date a
This is in fact demonstrable when a writer must place a new NP in an emb edded sub ject p osition s he
tends not to omit the complementizer
emb edded sub ject TC RC
pronoun
indefinite
p
This observation provides a tantalizing suggestion that the that trace e ect exempli ed by may in fact
have a functional explanation the overt complementizer tends to signal new sub jects and a WH gap can b e
thought of as the most given NP p ossible
a Who did John say Mary likes
b Who did John say that Mary likes
c Who did John saylikes Mary
d Who did John say that likes Mary
As given here the mere tendency for new sub jects to b e asso ciated with an overt complementizer falls short of
completely accounting for the categorical that trace e ect This issue awaits further research
sub ject which is new to the discourse which the TR form do es not require The TC form is
thus predicted to present some di culty
Avoid New Sub jects also predicts a di erence b etween and
The rat that the cat that the dog bit chased died
A b o ok that some Italian I venever heard of wrote will b e published so on by
MIT press
requires the reader to accommo date three new sub jects simultaneously whereas re
quires only two since I is an evoked entity Substituting a new entity for I is predicted to
render the sentence harder to pro cess
A b o ok that some Italian the teacher has never heard of wrote will b e published
so on by MIT press
Also as with Disconnectedness changing the lo cus of the emb edding from sub ject to comple
ment predicts an amelioration of center emb edding di culty in rep eated here as as
compared with a mixed sub ject ob ject emb edding in and the doubly sub ject emb edded
John asked the woman that gave the b oy that kicked the dog some candy why
she s sp oiling him
John asked the woman that gavetheboy that the dog frightened some candy
why she s sp oiling him
The woman that the b oy that the dog frightened is b othering rightnowisa
friend of John s
Considering Eady and Fo dor s results see on page Avoid New Sub jects predicts di
culty when there are manysimultaneous new sub jects The maximum number of simultaneous
sub jects in a through d is and resp ectively This makes the incorrect predic
een b and c should b e the smaller tion that the di erence in pro cessing di cultybetw
than the other two di erences
Lastly b oth Avoid New Sub jects and Disconnectedness fail to account for the remaining center
emb edding e ects in non sub ject emb edding is emb edded one level deep er than The
additional level of emb edding exacts a cost in pro cessing di culty despite its ino ensiveness to
b oth Disconnectedness and Avoid New Sub jects
Rob ert Ladd p c hyp othesizes that di culties with center emb edded constructions stem from the
unavailabil i tyofwell formed proso dic structures Consider the following contrast
i The shirts that the maid Tom can t stand sent to the laundry came back in tatters
ii The shirts that the maid whom Tom can t stand sent to the laundry came back in tatters
Ladd argues that the vo cabulary of ma jor proso dic breaks i e the single item ma jor break or comma is
not su ciently rich to indicate nesting of brackets or even whether a break denotes a left or right bracket In i
one could get by with one break after the entire matrix sub ject and the sentence sounds ne except p erhaps
foranunusually long intonational phrase In ii three breaks are necessary one for each comma and one at
the end of the matrix sub ject so the nesting relations are not prop erly enco ded recovered
John asked the woman that o ered the b oy that gave the dog that chased the
cat a big kick some candy why she s sp oiling him
The residual di culty of center emb edding constructions is very likely explained by memory
limitations in the syntactic pro cessor Bach Brown and Marslen Wilson compared center
emb edding and crossed dep endency constructions in German and Dutch as in and found
that the center emb edded examples in German were harder to understand than their crossed
dep endency analogs in Dutch
German
Arnim hat Wolfgang der Lehrerin die Murmeln aufr aumen helfen lassen
Arnim has Wolfgang the teacher the marbles collect up help let
Arnim let Wolfgang help the teacher collect up the marbles
Dutch
Aad heeft Jantje de lerares de knikkers laten help en opruimen
Aad has Jantje the teacher the marbles let help collect up
Aad let Jantje help the teacher collect up the marbles
Rambow and Joshi prop ose a syntactic parsing automaton based on Tree Adjoining
Grammar Using the storage mechanism of their automaton they de ne a pro cessing complexity
metric based on howmany storage cells a particular parse needs and how long they are needed
for in analogy with paying rent for storage space This complexity metric is consistent with the
ndings of Bach et al This metric also provides an interesting predictor of pro cessing di culty
asso ciated with various word order variations of complex sentences in German Rambowand
Joshi show that a range of acceptability judgements can b e accounted for Applying Joshi
and Rambo w s automaton account to pre and p ost verbal center emb edding and
resp ectively yields no di erence Owen Rambow p c all that matters is that the
dep endencies are nested This suggests that center emb edding di culties really do originate
from memory limitations in the syntactic pro cessor We can conclude that the di culties with
the classic center emb edded sentences such as is the aggregate of di culties in two lo ci
memory requirements in the syntactic pro cessor and interpretive e ects resulting from sub ject
emb edding as discussed ab ove cf Eady and Fo dor
The rat that the cat that the dog bit chased died
Summary
Ihave presented two comp eting theories which account for human p erformance patterns on a va
rietyofsyntactic constructions Disconnectedness theory assigns a p enalty for each constituents
which has not b een semantically integrated with the rest of the constituents Avoid New Sub
jects theory assigns p enalty for noun phrases which app ear in sub ject p osition and intro duce
entities which are new to the discourse Avoid New Sub jects requires no assumptions ab out
the sentence pro cessing system b eyond what is already necessary for accounting for comp etence
phenomena namely that p eople use sub ject p osition to enco de given information Disconnect
edness theory requires the assumption that the pro cessor prefers to avoid disconnected analyses
even when the disconnectedness can b e eliminated by immediately forthcoming words in the
string
While these two theories are very di erent stated over di erent domains their predictions
coincide for much of the available data Disconnectedness theory as de ned here is inconsistent
with the p ost ho c analysis I have presented for Stowe s second exp eriment in section
i e it is insensitive to the degree of newness Of course a direct exp erimentwould b e necessary
to validate that analysis Another area of disagreementbetween the two theories is in gap
lling Disconnectedness theory predicts that other factors b eing equal the pro cessor would
prefer to ll a gap if a ller is available Avoid New Sub jects makes no predictions with regards
to gap lling Ihave argued at the end of section that putting together results from
di erent exp eriments could provide us with grounds for falsifying Disconnectedness theory The
necessary exp eriments remain for future research
In the next twochapters I present a computational framework for mo delling the various asp ects
of sentence pro cessing The aim is to ultimately provide a means of integrating exp erimental
results and theories regarding a variety of factors into one consistent picture
Chapter
Parsing CCG
In the preceding chapters I have argued for a view of the sentence pro cessing architecture where
the syntactic pro cessor the parser prop oses syntactic analyses for the incoming words and
the interpreter cho oses among them based on sensibleness This is depicted diagrammatically in gure
Syntactic Analyses
Semantic-Pragmatic Input Utterance Parser Interpreter
Analysis-suspension Competence Messages Grammar key:
declarative knowledge
computational process
data flow
Figure An interactivesentence pro cessing architecture
Having argued for an architecture of this general kind I now fo cus on the sp eci cs of eachof
the two constituent comp onents in turn In this chapter I consider the design of the parsing
comp onent and in the next I turn to the interpreter and the integrated system
Goal
In the preceding twochapters I have argued that syntactic ambiguity is resolved according to
the interpretations of the available readings Given the virtual immediacy in whichaword s
contribution to the meaning impinges on ambiguity resolution decisions it follows that the
parser whose task it is to identify for the interpreter the syntactic relations among the words
in the sentence must b e p erforming this task very quickly That is at every word the parser
identi es al l of the grammatical ly available alternative analyses and determines for each analysis
al l of the syntactic relations which arelogical ly entailedbythecompetencegrammar or at least
enough syntactic relations to draw the distinctions necessary for interpretation Crucially these
determinations must not b e delayed by the parser until the end of an utterance or even the end
of a clause
My aim in this chapter is to adhere to the central claim of the dissertation that the parser is
as simple as logically p ossible all that it enco des is analyses as de ned by the comp etence
grammar
Steedman has prop osed a pro cessor which he claims is able to construct sense semantics
in a timely fashion and in addition emb o dies a very transparent relation to the grammatical
comp etence the so called Strict Competence Hypothesis In the next section I present Steedman s
architecture In the following ve sections I consider ve di erentchallenges to the simplicity
and adequacy of Steedman s prop osal and advo cate certain extensions to Steedman s design
which promise to address shortcomings with the original
Steedman s Prop osal
How simple can the syntactic pro cessor b e Steedman argues as follows At the very
minimum the pro cessor needs three comp onents a transparent representation of the grammar
a metho d for constructing constituents by executing steps in the grammar and a metho d of
resolving ambiguity If the comp etence grammar is in its traditional form e g always dividing
a sentence into a sub ject and a predicate then it turns out that this minimal collection of three
comp onents is inadequate to provide the necessary sense semantics Consider the pair in
a The do ctor sentforthepatient arrived
b The owers sent for the patient arrived
While a is a garden path b is not This is b ecause the implausibil ity of the main verb
analysis of the owers sent is detected This detection takes place b efore the sentence is com
plete It follows that the sense semantics of the sub ject is combined the verb b efore the entire
VP is pro cessed If the grammar requires a VP no de however the straight forward interpre
tation of the minimal mo del ab ove wherein the pro cessor can only combine two constituents
when the syntax allows them to combine must wait until the VP is nished b efore the con
tent of the sub ject is integrated with the contentoftheVP Steedman argues that the obvious
ways of relaxing this strict rule to rule parsing hyp othesis which he calls the Strict Competence
Hypothesis such as adding Earley style dotted rules Earley or a top down parse stack
complicate the design of the parser and shifts additional explanatory burden to the theory of
evolution Steedman argues that if the grammar do es not require an explicit VP constituent
i e if it is able to treat The owers sent where sent is the main verb as a constituent strict
comp etence can b e restored to the pro cessor
Details of Steedman s grammatical theory Combinatory Categorial Grammar CCG provide
an illustration of his claim In CCG every constituent has a grammatical category drawn from
a universe of categories as follows There is a nite set of basic categories such as s np n
etc It is given either as a list of symb ols or a space of nite feature structures There are two
binary typ e forming connectives and n such that if X and Y are categories then X Y and
XnY are also categories The set of categories is the set of basic categories closed under the
connectives and n Byconvention slashes asso ciate to the left so snnp np is usually written
snnp np Intuitively a constituent with category X Y or XnY is an X which is missing a Y
to its right left CCG is a lexicalized grammar formalism which means that the collection
of constituent combination rules is rather minimal and most of the complexity of the grammar
resides in the way individual words are assigned a category or a set of categories in case of
lexical ambiguity From the description of the meaning of the slash connectives one exp ects
the combinatory rules in
Forward Functional Application BackwardFunctional Application
X Y Y X Y XnY X
By convention the arrow is in the direction of parsing not generation These are actually rule
schemata where X and Y are variables which range over categories A rule combines twoadja
cent constituents whose categories match its left hand side and creates a new constituent with the
category on its right hand side A particular CCG can stipulate restrictions over the categories
that the variables may takeasvalues In addition to the two so called functional application
rules ab ove CCGs also includes functional composition rules suchasX Y Y Z X Z In
the rest of this do cument I use the following uni ed notation for application and generalized
functional comp osition
Forward Combination rule name Backward Combination rule name
X Y Y X Y XnY X
X Y YjZ XjZ YjZ XnY XjZ
X Y YjZ jZ XjZ jZ YjZ jZ XnY XjZ jZ
X Y YjZ jZ XjZ jZ n YjZ jZ XnY XjZ jZ n
n n n n
In the table ab ove jZ stands for either Z or nZ Underlined regions in a rule must match
Aside from the combination rules ab ove CCG systems often include two other kinds of rules
Typ e raising and Substitution Typ e raising schematized as
Forward Typ e Raising Backward Typ e Raising
X Y YnX T X Yn Y X T
is assumed to apply in the lexicon and is therefore not included as a rule in the grammar The
Substitution rule p osited in order to handle parasitic gaps Steedman Szab olsci is
Substitution
Y Z XnY Z X Z S
and is also included in the universe of rules
In addition to the ab ove rules There is a sp ecial rule for co ordination whichcombines three
sub constituents
X co ord X X
A derivation is a tree whose leaves are categories and whose internal no des are valid rule
applications A string is grammatical just in case there is a derivation whose frontier is a
sequence of categories which are each in the lexical entry of the corresp onding word in the
string Aside from determining the syntactic category of a string a derivation can also assign it
semantics One wayofachieving this is using combinators Curry and Feys Quine
Steedman A combinatory semantics consists of augmenting each lexical entry with a
semantic ob ject and eachcombinatory rule with a semantic combination recip e The lexicon
then maps a word to a set of pairs h syntactic category semantic ob ject i The semantic
combinations recip es are as follows
i
X a Y b Z B ab i
i
Y b X a Z B ab i
X a Y YnX T a T
X a Yn Y X T a T
Y Z b XnY Z a X Z S ab S
Juxtap osition denotes term application By con
vention terms asso ciate to the left so xy z is
written as xy z
i
The semantic terms B T and S are sp ecial symb ols called combinators They do not carry
any semantic content themselves rather they enco de combinatorial recip es of their arguments
according to the following equations
i
B xy y x y y
i i
T xy yx
S xyz xz yz
By way of an illustration consider the following unambiguous lexicon
John s snnp T j notice the application of T in the lexicon
has snnp snnp has
met snnp np met
Susan np s
The string John has met Susan is grammatical since it is p ossible to derive a single constituent
from it as follows
John has Susan met
s snnp Tj snnp snnp h snnp np m np s
s snnp B Tj h
s np B B Tj h m
s B B Tj h ms
B Tj h ms
Tj h ms
h ms j
Notice however that there are other derivations for this string which yield the same semantic
result For example
Susan has met John
s snnp Tj snnp snnp h snnp np m np s
snnp np B hm
s np B Tj B hm
s B Tj B hm s
Tj B hms
Tj h ms
h ms j
These analyses makes use of the functional comp osition rule to construct the non traditional
constituent John has met It has b een argued e g Dowty Steedman
that such constituents are necessary for a prop er treatment of the syntax of co ordination WH
dep endencies and sentence level proso dic structure The reader is referred to these pap ers for
details of the theory of comp etence
Steedman s p oint then is that a pro cessor for CCG uses the slash mechanism of the competence
grammar the same mechanism which is resp onsible for constructing the material b etween a
WH ller and its gap and non standard constituents for co ordination in order to pro duces
a grammatical constituent for the owers sent in which rep eated here as whereas
a pro cessor for a traditional phrase structure grammar would have to use grammar external
devices such as dotted rules to achieve the same e ect
a The do ctor sent for the patient arrived
b The owers sent for the patient arrived z
+ phase 2
uv
x - phase 1
x y
Figure A circuit for computing z xy y from Shieb er and Johnson
Strict Comp etence and Asynchronous Computation
Shieb er and Johnson claim that Steedman s argument that a standard right branching
grammar requires a more complicated parser rests on an incorrect assumption They distinguish
two sorts of computational architectures synchronous and asynchronous and argue that the
assumption of a synchronous architecture necessary for Steedman s argument is no more likely
a priori than that of an asynchronous architecture and may in fact b e less likely In this section
I present Shieb er and Johnson s argument and assess its force
Synchronous and Asynchronous Computation
Supp ose one had to construct a machine to compute the following function of twonumeric
arguments x and y
f x y xy y
out of comp onents which p erform primitive arithmetic op erations
One way to do this is to use a two phase circuit as in gure The rst phase computes the
intermediate results u xy and v y and the second phase computes the sum of u and v
The multiplication unit could come in one of twovarieties synchronous or asynchronous The
synchronous variety requires that b oth of its inputs b e sp eci ed b efore the output is computed
The asynchronous v ariety emits an output as so on as it can whenever one of its inputs is
zero it do es not wait for data on the other input b efore emitting the answer zero on its output
If the circuit is indeed built from asynchronous comp onents then a y input of zero would cause
it to emit a z answer of zero without waiting for the value of x If one were using asynchronous
comp onents but wanted phase level synchronization i e all of a phase s inputs must b e sp eci ed
b efore its output is emitted one would have to build in additional restraints into the circuit
Shieb er and Johnson argue that Steedman s strict comp etence hyp othesis precisely imp oses
phase level synchronization at the level of the mo dule Steedman requires that the syntactic
mo dule makeavailable to the interpretive mo dule analyses of only complete constituents That
is the interpreter maynotsee the results of combinations of incomplete syntactic constituents
They argue that this phase level synchronization is not necessary and could in fact makethe
design of the pro cessor more complicated as is the case with the design of phase synchronous
digital electronic devices
To illustrate the viability of asynchronous computation they prop ose a grammatical formalism
which pairs partial parse trees with partial LF like representations May The appa
ratus uses the same structure combining op eration for b oth the construction of grammatical
constituents including the residue of WH movement as well as the construction of partial
constituents such as
S
H
H
H
H
VP NP
Q Q
Q Q
Q
Q
the owers sent unspeci ed
Such a tree is paired by the formalism with an undersp eci ed logical form representation similar
to
unspeci ed op unspeci ed op send hthe owersi unsp eci ed ob ject
This representation anticipates zero or more sentence level op erators whichmay app ear syn
tactically adjoined to the VP no de but move to S at the logical form level The sub ject of
the sending is sp eci ed but the ob ject is not An interpreter may lo ok at this structure and
opp ortunistically draw whatever conclusions it can from the parts that are sp eci ed
The formalism that Shieb er and Johnson use is that of Synchronous Tree Adjoining Grammars
Shieb er and Schab es I nowsketch the idea brie y The reader is referred to the original
pap ers for details
Lexicalized Tree Adjoining Grammar TAG Joshi Joshi and Schab es is a gram
matical formalism where a grammar asso ciates a nite set of trees with each lexical item and
trees can combine using one of two op erations substitution and adjunction One tree is sub
stituted into another by simply replacing one non terminal symb ol at the frontier of with
by excising a subtree of Adjunction is slightly more complex a tree is adjoined into
that is ro oted at some nonterminal X substituting the subtree into at some o ccurrence of X
and then substituting the new into the original X site in Synchronous TAG no relation to
synchronous computation is a grammatical formalism for transduction the idea is that two
TAGs are synchronized or coupled such that op erations in one memb er are re ected in the
other Given twoTAGs a synchronous TAG can b e de ned as a set of ordered pairs of trees
from the resp ective grammars Within an ordered pair no des in one tree can b e paired with
corresp onding no des in the other Whenever an op eration substitution or adjunction happ ens
to one no de in a tree a corresp onding op eration must happ en to the no de it is linked to
Evaluation
Steedman claimed that a pro cessor for a right branching grammar needs to have a grammar
external op eration for partial combination Given that the interpreter must b e able to take
advantage of applications of this op erations it must follow that it to o is able to see the op eration
as well Shieb er and Johnson have shown that using an asynchronous computational paradigm
it is not necessary to augment the parser or interpreter with any op erations b eyond those allowed
by the comp etence grammar
The ultimate question of whose system is simpler Steedman s CCG or Shieb er and Johnson s
asynchronously computed partial structure paradigm can only b e resolved when they are b oth
extended to provide wide coverage of the linguistic phenomena and their precise implementation
details are given
One imp ortant asp ect which Shieb er and Johnson do not address in their pap er is co ordina
tion CCG provides a uniform mechanism for incremental interpretation and the constituency
necessary for co ordination For example CCG assigns the string John loves the grammatical
category S NP This category can b e co ordinated with another of the same typ e giving rise to
constructions such as Right No de Raising
John loves and Bill hates London
Such an analysis of Right No de Raising is not readily available in Shieb er and Johnson s mech
anism which assigns John loves the grammatical category S which the grammar cannot dis
arious approaches p ossible tinguish from the category for John loves London While there are v
for extending Shieb er and Johnson s account e g by adopting a prop osal by Joshi or
elab orating on the work of Sag et al more research is needed to determine whether the
elegance and simplicity of their accountwould remain once it is extended to cover co ordination
One p otential source of di culty for CCG is the need for interpretation of constituents in which
not all combinators have b een rewritten For example the interpretation for the main verb
analysis the owers sent is
B T the owers send
One might supp ose that interpreter contains sp ecial strategies to cop e with such expressions
but suchamoveintro duced serious complexity
Another p ossibility is to rede ne the notion of combinator Instead of treating it as a primitive
symb ol one could treat is as standing for a term Given their de nitions ab ove this is
straightforward
i
B xy y x y y
i i
T xy yx
S xyz x z y z
Given this interpretation rewrites and reduces to
x send x the owers
whichisvery similar to Shieb er and Johnson s representation in
I conclude that Shieb er and Johnson have made a comp elling case against Steedman s claim
that CCG clearly gives rise to a simpler pro cessor than any system based on a right branching
grammar This debate therefore is far from settled For now either approach is viable so I use
one CCG or the remainder of this do cument
A note is in order ab out the choice of a formalism for semantic representation The obvious
formalism is that of an applicative system suchascombinators or terms describ ed ab ove
Such formalisms require the application of zero or more reduction rules after each syntactic
combination It is p ossible to eliminate the necessity for reduction rules using pre compilation
The idea describ ed in Pereira and Shieb er is to replace the simple category symbol with
a Prolog term which in addition to enco ding the usual syntactic features suchasnumber and
gender enco des the predicate argument structure as well See Pareschi and Steedman
and Park for discussions of applications of this approach to CCG Mo ore argues
that reduction rules cannot b e eliminated altogether the problem is that uni cation based
approximations of the calculus do not treat separate bindings of the same v ariable as distinct
A clear illustration of this problem arises in sub ject co ordination as in
John and Bill walk
If the predicate walk is treated as in
X walk X
and the co ordinate sub ject John and Bill is given a generalized quanti er treatment i e typ e
raised as in
S P john P bill P
Pereira and Shieb er intro duce the in x op erator as a notation to enco de terms So a term suchas
x y f xy
would b e enco ded in Prolog as
X Y f X Y
then the two copies of the predicate b ound by the variable P will fail to serve as indep endent
binders of X The uni cation step necessary to give the result
john walk john bill walk bill
will b e blo cked Park considered a collection of co ordinate structures and argued that
for each it is p ossible to construct a co ordination rule in his case a separate CCG lexical entry
for the word and whichprovides the correct logical form using only uni cation But in some
cases Park s resulting logical form is not the intuitively obvious simplest one but rather a
more complicated form which is truth conditionally equivalent For example the logical form
assigned to a is b not c
a A farmer and every senator talk
b x Farmer x y y x Talk y
z Senator z y y z Talk y
c x Farmer x Talk x z Senator z Talk z
Park suggests that a p ost semantic pro cess within the interpreter could massage forms like
b into forms like c But these manipulations of logical form while they do preserve
the entailments of the meaning might b e to o heavy handed for other more pragmatic asp ects of
meaning I conclude from Park s results that Mo ore s observations are quite accurate attempt
ingtosimulate reduction using term uni cation results in rather contorted and unnatural
semantic representations Phenomena such as co ordination indeed do require interleaving ap
plications of the combinatory rules of grammar with applications of semantic reduction rules
The Davidsonian approach to semantic representation is somewhat similar to a uni cation based
approach to semantics in CCG It to o is incapable of an elegant treatmentofmany co ordinate
structures e g a But it is quite straight forward to extend it to allow one The idea is
to move to a representation which separately enumerates each argument to a predicate Thus
a would have c as its representation instead of b
a Most students prefer denim
b most X student X tns E present prefer E X Y denim Y
c most X student X tns E present sub j E X ob j E Y denim Y
A co ordinate structure such as a would have the semantic analysis in b
a Most students and some professors prefer denim
b most U student U some V professor V and U V X tns E present
sub j E X ob j E Y denim Y
In the current implementation I use the traditional Davidsonian approach mostly for ease of
readability If co ordination were to b ecome imp ortanttothework it would b e straight forward
to map the system to the newer representation
Identifying Ungrammaticality
The minimum conceivable sentence pro cessor contains a representation of the grammar a non
deterministic algorithm for applying the rules of the grammar and an ambiguity resolution
mechanism which it is claimed here is solely based on sensibleness of the available analyses
Consider how such a pro cessor would cop e with the unremarkable sentence in
The insults the new students shouted at the teacher were appalling
The word insults has two categories a plural noun and a nite transitiveverb The noun
analysis can combine with the determiner to its left The verb analysis cannot Should the
pro cessor abandon the verb reading As external observers we can examine the grammar of
English and conclude that the verb analysis is do omed weknow that salvation cannot arrive
later in the string in the form of a category suchassn np n n snnp np But the pro cessor
cannot know this since it do es not contain pre compiled knowledge ab out the grammar The
pro cessor cannot automatically prefer a combined analysis to an uncombined analysis as this
would constitute a structural preference strategy whichwould have wide ranging and bizarre
predictions For example in when the word Chris is encountered the pro cessor would
prefer to co ordinate the two NPs Sandy and Chris b ecause that would yield a single constituent
for the whole string
Kim likes Sandy and Chris likes Dana
The only available recourse given the minimal pro cessor is an accountby which the interpreter
is able to discard the verb analysis of insults But this is rather unlikely There is no a priori
reason to exp ect that an uncombined determiner should present a problem for an interpreter
thus imp osing a p enalty on the verb analysis In fact there are languages where determiners
e g deictics quanti ers are routinely kept uncombined for manywords until their head nouns
are pro cessed For example in Korean the structure of a noun phrase is
Determiner Relative Clause Noun
The noun analysis of insults on the other hand do es incur p enalties when the subsequent
words arrive and require a restrictive relative clause analysis whichentails a complex pro cess
accommo dating the resulting noun phrase out of context These same words p ose no problems
for the verb analysis the verb phrase insults the new students is constructed Given the
preference for avoiding complex accommo dation pro cesses one would exp ect the interpreter to
discard the noun analysis leading to a garden path e ect at the disambiguating word shouted
This is clearly wrong The sentence causes no conscious pro cessing di culty
Whatever solution is provided for eliminating inappropriate analyses e g the verb analysis
for insults in it must op erate rather quickly and ruthlessly otherwise the number of
surviving ungrammatical analyses b ecomes unmanageable
Another problem with preferring a combined to an uncombined analysis incorrectly predicts di culties with
the string Which house did John paint a picture of This will b e discussed in section
The problem here is by no means a new one The minimal design is a classic b ottom up
parser which online can determine which analyses are p ossible but not which are imp ossible
Ungrammaticality information is only available at the end of the string when no analysis contains
one category which spans entire input Many parsing techniques have b een prop osed that
address this problem LR tables see section are pre compiled guides to a parse stack
which identify viable sequences of stack elements and implicitly enco de how these elements will
b e ultimately combined The Earley parser Earley constructs this sort of information
online using annotations on the rules of the grammar to enco de which constituents have b een
seen and which are exp ected Marcus s parser Marcus see section contains rules
which explicitly diagnose whichavailable syntactic analysis should b e followed
To address this problem I prop ose to augment the syntactic parsing mo dule and interpretation
mo dule with a third mo dule an unviable state lter There are three issues p ertaining to the
design of this lter
Should it op erate as a categorical lter ruling out most or all ungrammatical analyses
or should it have graded judgement rating certain analyses b etter than others
Should it b e conceived of as innate of biological standing equal to the other two mo dules
or should it b e conceived of as a skill which an exp erienced language user acquires for
discriminating grammatically viable analyses from ones that are do omed
Should this mo dule b e placed b efore the syntactic pro cessor mediating lexical access by
p erforming a rst cut disambiguation pro cess over the available grammatical categories or
should this mo dule op erate on the output of the syntactic pro cessor discarding unviable
category bu er con gurations
Implementing the lter as rating among available analyses can b e thoughtofasawayofim
p orting structurally lexically based ambiguity resolution preferences For example supp ose the
pro cessor rates a complementizer analysis of the word that when it follows a noun higher than
it rates the relativizer analysis The exp ectation then is indistinguishabl e from that of Minimal
ttachment in examples like A
The psychologist told the wife that he was having trouble with
Similarly for ambiguous words like raced in
The horse raced past the barn
As has b een argued in the preceding chapters there is little evidence for structurally based
preferences so a categorical lter that evaluates each analysis on its own without regard to its
comp etitors is preferable
As for the second choice it is clear that an innate account of this lter is evolutionarily unpar
simonious if this element is necessary for language communication then a grammar could not
haveevolved without it nor could the lter haveevolved without the grammar An empiricist
account of the lter as a skill is rather plausible When a child b egins acquiring language the
lter is totally p ermissive allowing all analyses even those of a determiner followed byaverb
At this stage the proliferating candidate analyses usually quickly overwhelm the pro cessor s
abilitytokeep track of them Consequently only short utterances are prop erly understo o d The
child observes that a bu er such as the DET insults VERB never gives rise to valid utterances
and learns to lter it out Gradually this lter is re ned to its observed sophistication in adult
listeners
The third choice concerns the placement of the lter in the rest of the system Placing the lter
as prop osed by Steedman b etween the lexicon and the syntactic pro cessor allows one
to exploit muchrecentwork in automatic part of sp eech lab eling of words Church has
shown that is very easy to train a part of sp eech tagger on a tagged corpus to achieve accuracy
b etter than on unseen text There has b een much recentwork on improving the accuracy
of such taggers and or reducing the volume of training materials necessary see Brill and
references therein Such taggers are sensitive to only a small p ortion of the syntactic context in
whichaword app ears usually a windowofafewwords to either side of it In many taggers
it is p ossible to adjust a parameter called the precision recall tradeo When precision is high
the tagger is likely to nd few incorrect categories for a word When recal l is high the tagger
is likely to miss few correct categories but it may increase the numb er of incorrect parts of
sp eech it guesses for eachword It is quite plausible that an excellent recall mo derate precision
h tagger mediates lexical access I am aware of no comparable existing work for part of sp eec
automatically training a lter which discriminates viable b ottom up bu er states But given
the fact that an unviable bu er state never results in a grammatical analysis for a string or
at least an analysis which do es not require correction on the part of the hearer whereas every
viable bu er state do es eventually give rise to a grammatical sentence and given the fact that
the space of viable bu er states is quite small and regularly structured it is plausible that such
a lter can b e trained by observation of successful and unsuccessful bu er states
Either placement of the lter is therefore viable In the next two sections I consider a two further
problems to the lterless minimal architecture These problem are resolved using a lter placed
between the syntactic pro cessor and the interpreter
Here is a sketch of an algorithm which could b e used to carry out the acquisition of this skill
of identifying viable bu ers After eachword for each parser state record the sequence of
categories in that state s bu er At the end of a grammatical string for each state in the correct
analysis go back and add a mark to that state s bu er For each state in each analysis
which did not turn out to b e the correct one add a mark After some training the resulting
collection of marks can b e used to implement the viable bu er criterion as follows
If a particular bu er con guration contains at least one mark then it is viable
If the bu er con guration contains no marks and more marks than some threshold
then it is unviable
If the bu er con guration contains no marks and fewer marks than the threshold
then not enough information is available In the absence of de nitive information accept
the bu er thus trading e ciency for completeness
Note that this algorithm considers each bu er state individually the viability of a bu er is
indep endent of other comp eting analyses As given ab ove the algorithm is ine cient mayb e
even impractical in that it requires p otentially unb oundedly long bu er con gurations b e stored
and retrieved But as will b e seen in the next three sections the parser in practice will construct
very short bu ers rarely exceeding three constituents Furthermore it maywell turn out I
susp ect that it is su cient to consider only the right most two or three constituents in a long
bu er for the purp oses of bu er viability Finally there is the issue of howmany distinct
categories must b e kept track of Considering current CCG analyses of English the collection of
relevant categories is likely to turn out to b e quite small For Dutch whichallows verb clusters
to form constituents see Steedman some additional bit of cleverness may b e necessary
to make the theoretically in nite space of categories manageable Empirical investigation of
particular induction strategies for the viable bu er criterion await a broad coverage CCG for
English
Shift Reduce Con icts
A b ottom up parser for CCG encounters three kinds of nondeterminism
categorial ambiguity Aword mayhave more than one part of sp eech e g rose is either n
or snnp or even for the same part of sp eech a word mayhave more than one combinatory
potential e g raced is either snnp or nnn pp In LR parsing parlance this is can b e
thought of as a shift shift con ict
ts in the bu er maycombine in more than one way howtocombine constituents constituen
One example is PP attachment Chris tickled the dog with the feather This is a reduce
reduce con ict
whether to combine constituents Consider the string
Which house did you paint a picture of
After the word paint the relevant bu er state is
Which house did you paint
q s np s np
inv inv
Combining the two constituents is a valid move It yields an analysis wherein it was a
house that was painted not something else The combined analysis cannot b e continued
grammatically by another NP a picture This is obviously not the appropriate move
here The two constituents must remain uncombined until the end of the string as in
Which house did you paint a picture of
q s np s np np n n pp pp np
inv inv
s n
inv
s pp
inv
s np
inv
q
This is a shift reduce con ict
The rst twoambiguities were the topic of the preceding chapters Shift reduce ambiguityisa
sub ject of serious concern since it p otentially applies to every combination
One may opt to treat this latter ambiguityasany other pursue b oth analyses in parallel and let
the interpreter work it out While this sort of solution mightwork for ordinary phrase structure
grammars it is impractical for CCG b ecause of CCG s asso ciativityofderivation Recall that
CCG s rule of functional comp osition cangiverisetomultiple equivalent analyses as in
and on page This derivational ambiguity proliferates very quickly For example
the string in has truth conditionally equivalent CCG analyses
John was thinking that Bill had left
s snnp snnp snnp snnp s s s s snnp snnp snnp snnp
In general for sequences of functional comp ositions the degree of this ambiguity grows as
the Catalan series that is roughly exp onentially
Catalan
X
Catalan n Catalan i Catalan n i
i n
In section I describ e a way for the parser to cop e with this proliferation of equivalent analyses
bykeeping track of one representative from each truth conditional equivalence class of anal
yses The pro cessor can therefore pursue only the maximally left branching analysis ignoring
the p ossibility that two constituents may remain uncombined But the lo cal ambiguity in
a ects truth conditions it is either a house that was painted or a picture So that example
requires sp ecial treatment The pro cessor must know to distinguish the uncombined analysis
and pursue it in this case One may argue that the question of whether to leave which house
and did you paint uncombined can b e easily resolved bywaiting for the very next word for dis
ambiguating information But this is not always p ossible sometimes syntactic disambiguating
information is delayed for manywords as in
This has b een called spurious ambiguity Wittenburg Although it has b een p ointed out that this
ambiguity of CCGs is necessary on linguisti c grounds see section
a Here is the cathedral that John drew and Bill b ought three b eautiful
charcoal sketches of
b Which of his daughters was Percival planning to donate to the university
an extravagantportraitof
In these examples it is clear that interpretation determines how the lo cal ambiguity is resolved
The parser therefore must present the interpreter with b oth analyses On what basis then
can the parser know to make the interpreter aware of the uncombined analysis in this case but
not to b other the interpreter with the many other truth conditionally irrelevant uncombined
analyses The viable bu er lter discussed in section o ers a solution Placing this lter
between the parser and the interpreter allows the p ossibility for distinguishing relevant non
reductions which in the case of picture nouns e g are identi able by a sequence of
categories of the form X s np s np Placing the lter b etween the lexicon and the parser
do es not immediately prop ose such a solution
Notte that allowing the WH ller and the gap containing constituent not to combine precisely
implements the idea argued for in section that the lo cus of lled gap e ects is in the
interpreter not the parser
Heavy Shift and Incremental Interpretation
Another challenge to the lterless architecture arises from the interaction of heavy NP shift
and referential pro cesses The Strict Comp etence Hyp othesis section taken together with
the usual assumption of Comp ositionality that combinations in the syntactic domain are
are mapp ed to combination in the semantic domain predicts that the interpreter may not
b ecome aware of combination of semantic constituents b efore parser p erforms the corresp onding
syntactic combination Steedman uses this reasoning section to argue against a grammar
which requires a VP no de But do es CCG provide su ciently incremental analyses so as to
overcome every instance of this problem The places to lo ok for an answer is where CCG do es
not provide a word by word left branching analysis One such place is around the canonical
p osition of heavy shifted arguments
a exempli es heavy NP shift Once one of a verb s arguments is heavy shifted it is ungram
matical to move its other arguments asshown by the ungrammaticality of WH movement
b and right no de raising in c Note that multiple right no de raising is not imp ossible
in general as d shows the latter is from Abb ott
a The bird found in its nest a nice juicy worm
b What did the bird nd in a nice juicy worm
c The bird found in and its mate found near the nest some nice juicy worms
d I promised but you actually gave a pink Cadillac to Billy Schwartz
In order to rule out b and c CCG must delay the combination of found and in until the
entire PP in its nest is constructed If wecanshow that b efore the PP is fully pro cessed the
pro cessor is nevertheless aware of the combination of found and in then wehave shown that
even CCG fails to provide su ciently incremental analyses
Evidence for the detection of the heavy NP shift b efore the PP is fully pro cessed is provided by
the lack of garden path in a as compared to b
a The bird found in its nest died
b The horse raced past the barn fell
In a when the pro cessor encounters found it has two analyses reduced relative clause
or main verb Out of context there is no mutually established background so the reduced
relative analysis requires the accommo dation of a fairly complex set of presupp ositions as dis
cussed in section The main verb analysis has no problems so far But the next word
a prep osition do es present a problem for the main verb analysis the verb nd is obligatorily
transitive so heavy NP shift must b e assumed The construction of heavy NP shift is felic
itous when the material which mediates the verb and the shifted argumentisbackgrounded
given information Out of context heavy shift is therefore not felicitous so the main verb
reading also carries a p enalty Faced with two imp erfect analyses the pro cessor has no basis
for preferring one over the other so it keeps them b oth leading to the acceptability of either
continuation a and a When the ambiguous verb is p otentially intransitive e g
raced in b encountering a prep osition do es not presentany di culties for the main verb
analysis so the pro cessor decides to discard the reduced relative clause analysis in favor of the
main verb analysis leading to the garden path in b
Crucially the pro cessor is able to detect the inevitability of heavy NP shift for the main verb
analysis of a b efore the PP is fully pro cessed Were the pro cessor to wait until the end of
ould surely b e able to avoid the garden path in b the PP to resolve the ambiguity itw
Placing a viable bu er lter b etween the parser and the interpreter can provide the necessary
mechanism for identifying the unavoidable heavy NP shift in a In the same way that
such a device would learn the inevitable failure of certain bu er con gurations it could also
learn the inevitabili ty of the heavy shift construction which the parser will nd The current
implementation of this mechanism is presented in section
One p ossible attempt to salvage the minimal account is to argue that the pro cessor do es not actually determine
that heavy shift is unavoidabl e in the main verb analysis of the bird found in but rather that the pro cessor
merely notices that there are two constituents which it cannot yet combine In such cases the pro cessor pro ceeds
cautiously not discarding comp eting analyses i e the reduced relative clause analysis
To counter this argument one could make the following observation While b oth analyses of found are main
tained when the sentence is presented out of context there are contexts which can cause the pro cessor to makea
commitment b efore the end of the PP The relevant case here is a context which makes heavy shift felicitous as
in the question i
i What did the bird nd in the nest
The resp onse in ii is a garden path
ii The bird found in the nest died
A theory in which the pro cessor do es not discard infelicitous analyses e g an unnecessary restrictive relative
clause would fail to predict the garden path in ii
Coping with Equivalent Derivations
In this section I address the problem of proliferating equivalent analyses stemming from CCG s
asso ciativityofcombination as intro duced in section I rst examine existing prop osals for
coping with this sort of ambiguity Combining ingredients from two of the prop osals Pareschi
and Steedman s idea of lazy parsing with Hepple s normal form construction
I then intro duce a new parsing system which addresses shortcomings in its predecessors
Evaluation Criteria for a Parser
In light of the discussion earlier in this chapter any algorithm whichistoserve as an adequate
parser must satisfy the following desiderata
soundness All parser outputs must b e consistent with the grammar and the input string
completeness Given a grammar and a string every grammatical analysis for the string should
b e constructible by the parser That is the parser is free of structural ambiguity resolution
tendencies
incrementality Given an initial segment of a sentence the parser must b e able to identify
all the semantic relations which necessarily hold among all of the constituents seen thus
far For example having encountered a sub ject NP followed by a transitive main verb
the parser must identify or merely narrowdown dep ending on one s theory of thematic
relations the semantic role which the sub ject NP plays in the main sentence
feasibility The computational resources needed to run the algorithm must plausibly b e pro
vided by the human brain Given our current understanding of the brain this criterion
is unavoidably fuzzy Clearly algorithms which are exp onential in the length of the string
e brand infeasible any algorithm which do es not b ound the are infeasible but should w
pro cessing time of eachword to a constant The answer is less clear issues of implemen
tation of parallelism and the brevity of most utterances complicate matters In the case
of parsing CCG the asso ciativityofderivations must not impact the parser s p erformance
adversely
transparency The parser uses the comp etence grammar directly not a sp ecially transformed
or compiled form
Previous Attempts
There has b een a variety of prop osals for parsing CCG Wittenburg Wall and Wittenburg
prop ose that the grammar b e compiled into a di erent one in whicheachsemantically
distinct parse has a unique derivation or in some cases a few but muchfewer than the Catalan
series Their prop osal addresses only the rules and It do es not seem to generalize
obviously to higher order combinations esp ecially when so called mixed comp osition in which
the slashes are not all of the same direction This compilation pro cess comes at the cost
of substantially changing the constituency structure de ned by the linguist s original source
grammar hence compromising transparency Furthermore the complexity of the op erations
required to p erform this compilation renders suchascheme a rather unlikely accountofhuman s
representation of grammar
Following up on the work of Lamb ek who prop osed that the pro cess of deriving the
grammaticality of a string of categories b e viewed as a pro of there have b een quite a few
prop osals put forth for computing only normal forms of derivations or pro ofs Mo ortgat
K onig Hepple and Morrill Hepple The basic idea with all of these works is
to de ne normal forms distinguished memb ers of each equivalence class of derivations and
to require the parser to search this smaller space of p ossible derivations These prop osals enjoy
the advantage of transparency Unfortunately most of them cannot result in parsing systems
which pro ceed incrementally through the string This results either from an intrinsically non
string based Gentzen like pro of system Mo ortgat K onig or from a right branching
normal form Hepple and Morrill A p ossible exception to this criticism is the work of
Hepple Hepple considers Meta Categorial Grammars a close relative of CCG prop osed
by Morrill Hepple s normal form derivations are as left branching as the grammar
allows just the sort of incrementality necessary for our parser But Hepple do es not provide
a computational implementation for the elegant normal form construction which he presents
Unfortunately Hepple s claims that his system can b e parsed su ciently incrementally are not
tenable The problem is with the timing moving left to right through the input the parser
cannot know what is ahead b efore it must commit to a normal form parse for the input so far
For example in
John loves Mary
s vp vp np np
Of the two p ossible derivations the left branching one the one which treats John loves as a
constituentoftyp e s np is the normal form However in
John loves Mary madly
s vp vp np np vpnvp
There is only one derivation This derivation treats loves Mary as a constituentoftyp e vp
So what is a parser to do after having encountered John loves Mary It is not allowed to
construct the non normal form derivation If it commits to the normal form derivation for these
three words then it would b e stuckifanadverb were to come next It is also not allowed
to simply wait and not decide b ecause that would violate incrementality Stated di erently
For example
an b d s a e b c c d e
b d e
crossing
a e
s
the problem is the inability to extend a left branching normal form by adding the next word
When the next word in the input is encountered the pro cessor computes distinct normal forms
representative for each distinct analysis it has no general way of excluding those analyses which
are extensions of analyses whichhave already b een discarded
Karttunen prop oses a very simple solution to the problem of asso ciativity of derivation
He uses a b ottom up chart parser and simply avoids adding duplicate arcs into the chart Since
he uses a uni cation based system he checks for subsumption rather than simple equality
or uni ability of terms It follows that for a string of n applications of the rule of forward
comp osition O n arcs are added to the chart instead of O Catalan n Karttunen s
parser is clearly sound complete and transparent But it do esn t construct derivations or
analyses Instead it constructs arcs The di erence may app ear insigni cant at the end of
the parse those arcs that span the whole string are exactly the analyses The di cultyarises
in the interaction with the interpreter The interpreter cannot simply checkeach arc against
every other arc a constituentmust b e evaluated in the context of its preceding constituents
in the analysis The syntactic context indep endence assumption which dynamic programming
algorithms such as Karttunen s chart parser rely up on is not compatible with the context
necessary for interpretation The pro cess of computing all valid constituent sequences which
span the input so far is quite complex esp ecially if one wishes to consider only maximally long
void truly spurious ambiguity The cost of constituents and not their sub constituents i e a
integrating this chart parser with the rest of the current system thus renders it infeasible
Pareschi and Steedman have made a third sort of prop osal construct only maximally left
branching derivations but allow a limited form of backtracking when a lo cally non maximally
left branching analysis turns out to have b een necessary For example when parsing
Pareschi and Steedman s algorithm constructs the left branching analysis for John loves Mary
When it encounters madly it applies in reverse to solve for the hidden constituent loves
Mary by subtracting the s vp category John from the s category John loves Mary
madly John Mary loves
s vp vp np np vpnvp
s np
s
reveal
vp
vp
s
The idea with this revealing op eration is to exploit the fact that the rules n and n when
viewed as three place relations are functional in all three arguments That is to say knowing
anytwoof fleft constituent right constituent resultg uniquely determines the third There are
some problems with Pareschi and Steedman s prop osal
The rst class of problems is the incompleteness of the parsing algorithm which they give a
chart parser Hepple The essence of these problems is that in a chart parser common
sub pieces are shared across di erent analyses In Pareschi and Steedman s lazy chart parser
the presence in one analysis of a certain arc can lead to the omission in another analysis of a
crucial arc Pareschi and Steedman use a scheme right generator marking wherein if an arc
has b een combined with another arc to its left then it is prevented from combining with anyarcs
on its right In the vp np arc for loves is such an arc and is therefore prevented from
combining with the np arc Mary to yield a vp arc In the presence of ambiguity this could lead
to incompleteness For example in from Hepple one category of the word that
comp oses with he This renders he unable to combine with liked It follows that the parser
cannot nd an analysis for the whole string which is of course grammatical
he liked he told the woman that that it was late
s s n n nnn s np s vp vp np
s s
s s
s s
s vp
s np
stuck
This problem of separate analyses contaminating one another through shared chart cells can b e
eliminated if one replaces the chart parsing framework with one that do es not factor sub results
as I do b elow
A second class of problems with Pareschi and Steedman s revealing computation is the unsound
ness which results from the assumption that the combinatory rules are invertible In the
category a b is subtracted from a to reveal the category b as the result of combining b c and
an a c This is an unsound inference regardless of the control algorithm in whichitisembed
ded The consequence is that the parser nds an analysis for which is not licensed bythe
grammar
an a c bnb a b b c
a c
a
reveal
b
b
a
This form of unsoundness is not a problem if the grammar happ ens to b e such that whenever a
constituent has a typ e raised category of the form Xn X Z for some categories X and Z then
it also has the category Yn Y Z for any other category Y While the class of such grammars
maybeofpotential interest e g it would include any reasonable CCG for English additional
arguments on language universal grounds would b e necessary b efore one accepts this theoretical
unsoundness as having no practical imp ort
Hepple provides another illustration of the unsoundness of the revaling pro cedure For
heavy NP shift CCG allows a rule of backward crossing comp osition as in
loves madly the crazy Scottish p o et
vp np vpnvp np
x
vp np
vp
Also the grammar allows co ordination of non traditional constituents as in
loves Mary madly and Susan passionately
vp np vpn vp np vpnvp conj vpn vp np vpnvp
vpn vp np vpn vp np
co ord
vpn vp np
vp
Using revealing the pro cessor construct the following parse for
Mary and Susan passionately loves madly
vp np np vpnvp conj vpn vp np vpnvp
vp vpn vp np
vp
reveal
vpn vp np
co ord
vpn vp np
vp
The revealing step in cannot help but also reveal a vpn vp np constituent for the string
madly the crazy Scottish p o et thus allowing the pro cessor to admit which is ruled out
by the grammar
loves and the crazy Scottish p o et madly Susan passionately
vp np vpnvp np conj vpn vp np vpnvp
x
vp np vpn vp np
vp
reveal
vpn vp np
co ord
vpn vp np
vp
Again this unsoundness is not a problem if one assumes as I do in this pro ject that the
semantic analysis of heavy shifted constructions suchas loves madly bare markings which
distinguish them from unshifted constructions The discrepancy in this marking will prevent
the co ordination rule from treating the two constituents in as like categories But unless
one has strong cross linguistic evidence that the unsoundness ab ove will never present a problem
for any reasonable grammar it is b est to have parser whichworks correctly for every grammar
in the formalism
Aside from intro ducing unsoundness the revealing pro cedure is also incomplete In the
category bnc cannot b e revealed after it had participated in twocombinations of mixed direction
and
bncn bnc a b c bnc
b
a
stuck
A Prop osal
Pareschi and Steedman s prop osal emb o dies an app ealing idea construct the maximally left
branching analysis revising this commitment only when it b ecomes necessary The chart parser
implementation of this lazy parsing idea is clearly unacceptable Given the di culty of in
crementally computing partial derivations from intermediate chart parser states and given the
interactive nature of the current system the traditional advantages of a chart parser i e its
reuse of analyzed substring across divergent analyses are eclipsed by its disadvantages Replac
ing the chart parser with a shift reduce parser whichsimulates nondeterminism using explicit
parallelism eliminates the problems asso ciated with the system of right generator marking
But there are still problems with the subtraction style op eration of revealing The unsoundness
and incompleteness in and resp ectively still remain One way out of these problems
is to reparse the constituent instead of revealing it Reparsing the substring need not b e
p erformed from scratch if the parser s data structure maintains links from a each constituent
to its sub constituents then this derivation history can b e reused when constructing the revealed
constituent For example to reparse the vp saw three birds the derivation history tells us to
use the rule to combine saw and three and then use the rule to combine saw three
with birds
three yesterday birds Fred saw
s vp vp np np n n vpnvp
s np
s n
s
reparse
vp
vp
s
This prop osal is sketched in Hepple
Unfortunately the presence of categorial applicabili ty conditions on combinatory rules presents
the following problem to this recip e reparsing approach Supp ose one wanted to rule out the
following derivation from the comp etence grammar
p otato Dan b ought a and ate the
s vp vp np np n co ord vp np np n n
vp n vp n
co ord
vp n
s n
s
One could stipulate the following restriction on the rule
X Y Y X unless X Y vp n
Granted this is not the only way of capturing this fact nor is it a particularly app ealing one
But this is a substantive grammatical question and should not b e resolved arbitrarily bythe
parsing algorithm Could recip e reparsing handle the following example
p otato ate Dan the quickly
s vp vp np np n n vpnvp
s np
s n
s
Recip e reparsing done the obvious way i e mirroring the derivation would rst combine ate
the to make vp n and then attempt to combine that with p otato This derivation is ruled
out But there do es exist a derivation for the ab ove string
quickly Dan ate the p otato
s vp vp np np n n vpnvp
np
vp
vp
s
Recip e reparsing therefore results in incompleteness Note that if one were to change recip e
reparsing so as to work the other way around i e build the right branching structure then it
would b e p ossible to construct a counter example which required the left branching analysis
The move from revealing by subtraction to recip e reparsing is one which trades e ciency for
accuracy Given that recip e reparsing is not su ciently accurate is it necessary to giveup
Pareschi and Steedman s intuition of lazy parsing altogether and use full edged parsing on the
substring to b e recovered I now argue that the answer is No
The way I prop ose to exploit the information implicit in the derivation history is by rewriting
the derivation into another derivation which preserves all the semantic relations enco ded in the
original derivation but makes p ossible syntactic combinations which the original did not For
example the derivation
Mary loves John
s vp Tj vp np l np m
s np B Tj l
s B Tj lm
can b e rewritten using one step to the equivalentright branching derivation
John loves Mary
s vp Tj vp np l np m
vp lm
s Tj lm
which has the vp constituent necessary for combining with madly whose category is vpnvp I
use the technique of term rewrite systems and normal forms Hepple and Morrill Hepple
Intuitively semantics preserving derivation rewrite rules such as the one mapping
to can b e applied rep eatedly to correctly compute the right branching equivalentofany
derivation This computation can b e p erformed quite e ciently in time prop ortional to the
size of the derivation In App endix B I provide a formal de nition of the rewrite op eration
an e ective pro cedure for applying this op eration to compute right branching derivations and a
pro of of the correctness and e ciency of this pro cedure
Using the Recovered Constituent
Given the rightmost sub constituentrecovered using the normal form technique ab ove how
should parsing pro ceed Obviously if the leftward lo oking category which precipitated the
normal form computation is a mo di er i e of the form XnX then it oughttobecombined with
the recovered constituent in a form analogous to Chomsky adjunction as in gure As an
illustration shows a state of the parser when it encounters a backward lo oking category
Normal form computation results in the state shown in From here two states are p ossible
corresp onding to the twoways of Chomsky adjoining the mo di er low and high attachment
resp ectively These are given in and x\x x\x x (recovered) x (recovered)
x
Figure Recombining a recovered constituent with a rightward lo oking mo di er
said yesterday Mary John Bill that saw
s vp vp s s s s vp vp np np vpnvp
s vp
s s
s vp
s np
s
yesterday John said that Bill saw Mary
s vp vp s s s s vp vp np np vpnvp
vp
s
s
vp
s
yesterday John said that Bill saw Mary
s vp vp s s s s vp vp np np vpnvp
vp
vp
s
s
vp
s
yesterday said that Bill saw Mary John
s vp vp s s s s vp vp np np vpnvp
vp
s
s
vp
vp
s
But what if this category is not of the form XnX Should the parser compute the reanalysis in
a b b c c d sn a b n b d a b b c c d sn a b n b d
a c b d
a d sn a b
s
Suchamovewould constitute a very o dd form of cost free backtracking Before reanalysis the
derivation enco ded the commitment that the b of the rst category is satis ed bythebofthe
b c in the second category This commitment is undone in the reanalysis This is an undersirable
prop ertytohave in a computational mo del of parsing commitment as it renders certain revisions
of commitments easier than others without any empirical justi cation Furthermore given
the p ossibility that the parser change its mind ab out what serves as argument to what the
interpreter must b e able to cop e with such non monotonic up dates to what it knows ab out the
derivation so far this would surely complicate the design of the interpreter
Summary
This chapter b egan by reviewing a very b old prop osal of Steedman s The internal representation
used by the human syntactic parser consists only of grammatical analyses The prop osal is b old
on two counts
This pro cessing mo del is unusually imp overished
On the basis of the parsimony of the grammar parser package Steedman attempted to
argue for a certain theory of comp etence
The primary thrust of the argument point that in principle a pro cessor for CCG avoids
design complexity which is necessary for other grammatical frameworks was challenged by
Shieb er and Johnson s argument that asynchronous computation could capture the same com
putational simplicity for rather traditional lo oking phrase structure grammars Resolution of
this issue awaits re nement and elab orations of each of these theories to allow their evaluation
as adequate characterization of how the brain actually represents and pro cesses grammars
e I considered whether an imp overished pure b ottom up CCG parser Returning to p oint ab ov
can serve as an adequate parsing mo dule for the language pro cessing system I considered three
problems whichwould traditionally have received some sort of precompilation of the grammar
or top down prediction in the parsing sense of top down
I am indebted to Henry Thompson for a discussion of this issue of monotonicity
Timely detection of ungrammaticality e g the ability to quickly detect that an adjacent
pair of categories e g determiner verb has no chance of ever leading to a grammatical
analysis
Shift reduce con icts identifying the rare set of cases where a CCG rule should b e allowed
not to apply e g picture noun extractions
Timely detection of crossing comp osition detecting the inevitability of certain rule ap
plications b efore they actually happ en e g detecting heavy shift when an obligatorily
transitiveverb is immediately followed by a prep osition
Not surprisingly the pure b ottom up pro cessor cannot handle these cases correctly More
interestingly however I have argued that one s theory of the innate pro cessor can remain as
parsimonious as Steedman s if one makes the rather plausible assumption that while the ability
to parse is innate the ability to parse e ciently is not The skill which the language learner
acquires by attending to intermediate parser con gurations and their eventual outcomes can
serve to p erform the predictive functions necessary for the three cases ab ove The acquisition
pro cess is similar in some ways to the training of n gram mo dels for part of sp eech taggers
In the last section of this chapter I discussed a problem which is quite sp eci c to CCG CCG
distinguishes left branching and right branching analyses which are often truth conditionally
equivalent To cop e with the additional ambiguity brought ab out by CCG s asso ciativityof
derivation I prop osed that only the maximally left branching analysis as allowed by the gram
mar b e maintained and whenever this analysis turns out not b e the correct one the necessary
right branching analysis is computed from the derivation history
Steedman s prop osal of a parser which only represents grammatical analyses has therefore sur
hallenges which it had b een put to In the next chapter I showhow the resulting vived the c
parsing algorithm is used in the broader sentence pro cessing system
This prop erty has b een called spurious ambiguity Wittenburg Steedman has argued that this
ambiguity is not spurious rather di erent constituencies corresp ond to di erentways of breaking the string into
a theme and a rheme proso dic constituents which are used to enco de information status But CCG provides
more ambiguity than what is necessary for proso dic constituency the theme and the rheme may in turn receive
many truth conditiona ll y equivalent derivations
Chapter
A Computer Implementation
In this chapter I instantiate the parsing mechanism describ ed in chapter and the meaning
based ambiguity resolution mechanisms presented in chapters through I do so by presenting
a computer program which simulates human sentence pro cessing p erformance The aim of this
chapter and the implementation it describ es is to show the consistency of the collection of
subtheories develop ed thus far to account for the limited data that has b een collected and to
test whether these ingredients can indeed b e combined in a straight forward and non ad hoc
way
The program accepts words as input one at a time developing a set of partial analyses as it
progresses through the sentence If at any time this set b ecomes empty the pro cessor is said
to have failed the analog of a garden path In this pro ject I do not address recovery from a
garden path This mo del is successful just in case two goals are achieved
It correctly predicts garden path e ects in the range of examples discussed in the earlier
chapters
The implementation is straight forward that is it is a simple pro cedure which applies
linguistic comp etence to the input representation without having to resort to sp ecialized
algorithms
Desiderata
Let us b egin by stating the desiderata for the computational mo del in detail The system is
divided into the mo dules shown in gure The b ottom up syntactic rule applier i e the
parser constructs in parallel all p ossible analyses for the initial segment seen so far The bu er
viability lter detects unviable analyses and immediately signals the parser to discard these The
semantic pragmatic interpreter examines only the sense semantics which the parser constructs
and not other more sup er cial asp ects of the syntactic analyses The parser in turn may
not lo ok inside the interpreter the only information owing from the interpreter to the
t analyses The actual program do es not literally parser is whether to maintain or discard curren Syntactic Analyses Discourse model Lexicon
Semantic-Pragmatic Bottom up Viable Buffer Input Utterance Interpreter Rule Applier Filter
Combinatory Analysis-suspension Messages World Knowledge Rules key:
declarative knowledge
computational process
data flow
Figure System Diagram
separate the di erent pro cedural mo dules into informationally encapsulated mo dules e g using
asynchronous communicating pro cesses but nevertheless ob eys these restrictions on data ow
Toavoid the inferential complexities asso ciated with accommo dation the rep ositories of knowl
edge ab out the world and knowledge of the preceding discourse are not up dated by the inter
preter i e they are treated as read only storage
The following phenomena are covered
Referential Felicity Crain and Steedman show context sensitivity in pairs such
as see section Altmann et al
a The psychologist told the wife that he was having trouble with to leave
her husband
b The psychologist told the wife that he was having trouble with her
husband
Inacontext with just one wife a is a garden path whereas b is not The
opp osite is true if the context mentions twowives
Complexity of Accommo dation Crain and Steedman s Principle of Parsimony
see section entails that out of context the simplex NP reading of the wife
compatible with b would b e preferred to the restrictively mo di ed NP reading of
a
Principle of Parsimony Crain and Steedman
If there is a reading that carries fewer unsatis ed but consistent pre
supp ositions or entailments than any other then other criteria of
plausibility b eing equal that reading will b e adopted as most plau
sible by the hearer and the presupp ositions in question will b e in
corp orated in his or her mental mo del of the discourse
In the current pro ject the complex pro cess of determining the numb er and plausibilityof
presupp ositions carried by an NP will b e approximated byavery simple and crude metho d
accommo dating a simple NP incurs no cost while accommo dating an NP which is restric
tively mo di ed carries some xed cost It must b e emphasized that this approximation is
not based on the syntactic complexity of complex NPs but on the presupp osition enco ded
by the use of a restrictive relative clause there is more than just one entity matching
the description of the head noun so a restrictive mo di er is necessary to individuate the
referentintended by the sp eaker
Plausibility and Garden Paths Bever noticed that garden path e ects as in
The horse raced past the barn fell
can b e minimized when the plausibili ty of the main verb analysis of the rst verb is
decreased see section Bever s example is
I do not consider non restrictive relative clauses in this pro ject
The light airplane pushed past the barn crashed into the p ost
In this pro ject I use a slightvariant
a The p o et read in the garden stank
b The p o em read in the garden stank
Heavy Shift and Garden Paths Pritchett p oints out the garden path e ect
in is absent in
The bird found in the store died
Clearly the fact that nd is an obligatorily transitiveverb plays a role here Given that
The bird found in the store a corner in whichtonest
is also not a garden path sentence it follows that b oth reduced relative and main verb
analyses of found are pursued in parallel It is p ossible to force one or the other reading
using an appropriate context
Q What did the bird nd in the store
A The bird found in the store died
A The bird found in the store a corner in which to nest
In the p et store two exotic birds escap ed from their cages One was lo cated in
a nearby tree and the other was found hiding inside the store
The bird found in the store died
The bird found in the store a corner in whichtonest
Adverbial Attachment In chapter it was argued that considerations of information
volume were resp onsible for the low attachment preference of the adverbial in
The p o et said that the psychologist fell yesterday
But that no such considerations apply to the attachment of the adverbial in
The p o et said that the psychologist fell b ecause he disliked us
Since the inference required for determining correct attachment decisions in is op en
ended and non linguistic the current program leaves this ambiguity unresolved and rep orts
b oth readings
So called Late Closure E ects Out of context the examples in are garden
paths
a When the cannibals ate the missionaries drank
b Without her contributions failed to come in
c When they were on the verge of winning the war against Hitler Stalin
Churchill and Ro osevelt met in Yalta to divide up p ostwar Europ e
Ihave implemented b oth a new sub ject detector and a disconnectedness determining pro
cedure in order to exp eriment with the two theories presented in chapter
Center Emb edding E ects While do es not give rise to garden path e ects the
system do es represents the fact that it is harder than other sentences
The worm that the bird that the p o et watched found died
This measure of di cultyislower when some of the sub jects are given in the discourse
Syntax
The comp etence grammar in this system is an instantiation of Steedman s Combinatory
Categorial Grammar which is capable of constructing left branching analyses as discussed in
section A prop er linguistic investigation of grammatical comp etence b eing outside the scop e
of this work the aim of the grammar here is to provide at least one analysis for each reading
relevant for the examples in section To this end the following grammar will do
A Basic category is represented as an ordered pair a Prolog term and a semantic variable
separated by a colon The Prolog term is a ma jor category symb ol with zero or more arguments
its features The basic categories are
Basic categories
n NUM common noun
np PERS NUM noun phrase
s TNS FIN COMP sentence or SBAR
part PART particle
pp PREP prep ositional phrase
eop end of phrase marker zero morpheme
A feature may b e unsp eci ed or haveavalue from the following domains
I use Prolog notation throughout symb ols b eginning with a lowercase letter are constants symb ols b eginning
with an upp ercase letter are variables an underscore denotes an anonymous variable di erent o ccurrences
of denote di erent anonymous variables See Pereira and Shieb er for an intro duction to the Prolog
programming language
There is an exception to this naming scheme however Prolog is usually unable to keep track of names of
variable names after uni cation takes place When it must printavariable it prints something tedious suchas
Tomake terms easier to read I use a printing pro cedure which gives semantic indices names such as e
e syntactic variables names suchass s and category variables names suchasc c
Feature Possible values
NUM sg pl
PERS
TNS to en ing plup ed s fut ed s fut
FIN dep ends on TNS plup ed s are ed s fut to en ing are
PART away down up over
COMP that q q is sp ecial it means that the s is a WH question
PREP in to without
For example the basic category s ed that X stands for a sentence in the past tense whose
complementizer is that e g that Mary loves John In the accompanying semantic term list
the variable X represents the main situation in the sentence
The lexicon is stored as a collection of words and their asso ciated part of sp eech lab el When
the system is started a pro cess generates lexical entries from these part of sp eech lab els A
lexical entry is a triple of a word a syntactic category and a semantic term list Examples of
the di erent parts of sp eech lab els are as follows a S a S of X john closed X prs X pl X closed X prs Y pl Y prs X pl X closed X ax X tns S s y S sw y S X tns S s er X Y er X Y a tic term list w t S X Z Y tns S s a closed Y the X of X Y y S X Y tns S s ax X alk S X esterda go try S X Y tns S s tell S X Z Y tns S s gran in S Y in S Y passionately S sw w sa bird X w the X name the X st the X in X Y npmo d X whenev whenev seman call S X Y tns S s exist X w the X st the Y st juicy X y Z remind S X Z Y tns S s X X s s X Z np Z Y Y pl X np np np X Y n n n n Z Y eop X Y eop X S S np S np S N X F N X np P Y Y np n y a s T F np P s T F s T F Y n w n n n y n N X np a n w tactic category np part a eop S s to Y eop S s pp to Y np eop S s to Y eop S s s T F S VP VP VP VP VP VP n sg X n sg X np sg X np pl X np N X eop X n N X pp in X np VP syn np sg X s T F S s T np N X eop X n N X part a n N X n N X s T F S n N X s T F S s T F S s T F VP s T F S er X s T F Y y y er X Y s T F Y s T F a w t y a y alk ax e esterda w call w go a reminds tell gran bird w John us the in whenev try sa w our juicy Y whenev example a passionately y t pl X np n al S eV e pron e pron eV e transitiv in transitiv V part V in nitiv V Ob j V Sinf S Scomp complemen V Obj PP V Obj common noun mass noun det mass np prop er name ob ject pronoun determiner particle adjectiv prep osition N p ostmo di er S p ostmo di er S premo di er sub ordinating conj nominativ p ossessiv adv adv stands for s s S pro pro vp s pro VP O S lab el o oi c oc op
P v v vpr vi v v v v cn mn pn nom ob j p oss det part adj p ost p ost prep sconj
word category term list comment
that s T that E s T E complementizer
that n N Enn N E eop S s Snnp N E npmo d E sub ject relativizer
that n N Enn N E eop S s S np N E npmo d E ob ject relativizer
which n N Enn N E eop S s np N E npmo d E ob ject relativizer
which s T q E s T q np N E n N E wh E question
did s ed q E s ed E for sub j aux inversion
to s to Ennp P N X s to Ennp P N X in nitiveto
will s fut Ennp P N X s fut Ennp P N X future E aux will
was s ed Ennp P N X s ing Ennp P N X past progressive
is s s Ennp P N X s ing Ennp P N X pres progressive
had s plup Ennp P N S s en Ennp P N S plup erfect E
b een s en Ennp P N S s ing Ennp P N S past p erf progressive
eop X closed X end of phrase marker
Table Lexical entries for closed class items
In addition to the ab ove lexical entry generators idiosyncratic i e closed class words have
the lexical entries listed in table
The annotation swa X is asso ciated with all single word adverbials The annotation npmo d X
is asso ciated with all nominal mo di ers These annotations allowinterpreter to approximate
the detection of a low information volume adverbial preceded by a high information volume
argument as discussed in chapter
of X of X Y and phrase The zero morpheme eop X and the semantic terms the X name
closed X are part of the reference resolution system They will b e describ ed in section
closed X is abbreviated in the table ab ove as simply closed X The latter annotation phrase
for reasons of space
There are lexical entries for eachverb in all of its in ected forms as follows
form category semantics
walked s ed Ennp X walk E X tns E ed
walked s en Ennp X walk E X tns E en
walking s ing Ennp X walk E X tns E ing
walks s s Ennp sg X walk E X tns E s
walk s T Ennp X walk E X tns E T untensed
walk s s Ennp pl X walk E X tns E s plural present
walk s s Ennp X walk E X tns E s st p ers present
walk s s Ennp X walk E X tns E s nd p ers present
The tense system implemented in the current grammar is rather crude but it su ces to construct
the analyses necessary for the examples
Sub ject Aux inversion is handled as follows
nd did Mary
s ed q e s ed e s X Y e s X Y e nnp sg e s T e nnp sg e np e
s ed q e s ed e nnp sg e
s ed q e np sg e
of e mary nd e e e tns e ed name
Notice that the identity of the tense ed in this case is passed from did through Mary where
the tense variable X is uni ed with ed to nd whose lexical semantics include a variable T
which is uni ed with ed
The s ed q np constituent did Mary nd can then combine to form a WH question
whichbird did Mary nd
s T q e s T q e np e s ed q e np sg e
s ed q e
wh e bird e name of e mary nd e e e tns e ed
A sub ject typ e raising rule applies to all NPs with the exception of ob jective case pronoun
np P N X sem S s T F S s T F nnp P N X sem sub j X S
Avariant of this rule applies to all determiners
np Pers Num X eop X n Num X sem S
s T F S s T F nnp Pers Num X eop X n Num X sem sub j X S
Words that create non sub ject WH dep endencies relativizers wh question words eachhave
in addition to the categories listed in table two additional categories which re ect one and
two applications of a non direction preserving version of Geach s division rule Geach
X Y X Z Y Z
For example the relativizing pronoun which has in addition to the category
n N Enn N E eop S s np N E
listed in table the following two categories
X np N E n N Enn N E X eop S s
n N Enn N E X Y eop S s X Y np N E
The latter two categories are included in order to allow for non p eripheral extraction for
example
Recall fo otnote that symbols like e stand for semantic variables
The Prolog notation H T stands for a list whose rst element is H and the rest of whose elements are T
The sub ject typ e raising rule therefore adds the notation that the NP app ears as sub ject to the semantic term
list asso ciated with the NP
See Steedman for a di erentway to capture non p eripheral extraction interaction b etween crossing
comp osition and ob ject typ e raising
Mark reminded the babysitter to watch the movie
the babysitter that Mark reminded to watch the movie
The combinatory rules are as follows
left child right child result rule name
A B B A
A B B C A C
A B B C D A C D
A B B C D E A C D E
B AnB A
A B CnA C B
The capital letters in the rules are Prolog variables and these rules op erate by uni cation
Along lines suggested by Aone and Wittenburg there is a rule for p ositing a zero mor
pheme adjacent to a category which exp ect it The pro cessor blo cks excessive applications of
this rule For example given a determiner and a noun the rule applies to combines them
and yield the category np eop
bird the
np eop n n
np eop
The zero morpheme eop end of phrase is then p osited to the right of the noun and immediately
combined to yield an np
the bird
np eop n n eop
np eop
np
When a rule is applied to combine two constituents the semantic term list of the result is
simply the concatenation of the term lists of the two constituents with one exception the
last combinatory rule so called backward crossing comp osition intro duces an additional term
h shifted X Y which designates that argumentX ofY was heavy shifted For example in
a nice car found yesterday john
s e s e nnp e s e nnp e np e s e ns e np e eop e eop e
s e np e
s e np e
s e eop e
s e
Inessential details inside the categories are omitted for clarity
the semantic term lists asso ciated with the marked derivation step are as follows
John found
the e name of e john closed e nd e e e tns e ed
yesterday
yesterday e swa e
john found yesterday
the e name of e john closed e nd e e e tns e ed yesterday e swa e h shifted e e
Data Structure
The pro cessor maintains one or more analyses in parallel Each analysis has data comp onents
on twolevels Syntax Semantics and Interpretation Evaluation There are four comp onents
altogether
Bu er
Syntax Semantics
Semantic Term List
Interpreter Annotations
Interpretation Evaluation
Penalties
The Bu er is a sequence of constituents Adjacent constituents maybecombined using the
combinatory rules or revealing see chapter I use the term revealing for the pro cess of
recovering the implicit constituent using derivation rewriting A constituent is a tuple
hCategory Rule LeftChild RightChildi
where LeftChild and RightChild are normally constituents and Rule is the name of a combina
tory as listed in section When a constituent is a single word Rule is lex LeftChild holds
the actual word and RightChild holds the place holder There is a sp ecial rule init whichis
used in the initial state of the parser It is discussed in section The Semantic Term List
holds the list of semantic terms asso ciated with a constituent In case the Bu er contains more
than one constituent the Semantic Term List is the concatenation of the term lists of those
constituents
The interpreter may read the Semantic Term List but not mo dify it It records its results
e g pronoun resolution in the Interpreter Annotations comp onent The interpreter records its
assessment of the sensibleness of the analysis in the Penalties comp onent This comp onent has
two parts the p enalty list whichenumerates the particular p enalties asso ciated with the state
and a score which is determined from the p enalty list and is used for comparing the current
analyses
Given this representational system it is logically p ossible that there b e two terms in the term list which
originate from di erent constituents thus having no semantic indices in common Subsequently when the two
constituents are combined uni cation could cause two such distinct indices to b ecome identical Curiously
such a phenomenon do es not arise in the grammar and semantics of the current system That is whenever
two constituents do not combine it is never the case that they b oth intro duce semantic terms over semantic
indices which will subsequently b e uni ed If this prop erty remains in more comprehensive grammars it provides
opp ortunities for certain monotonicity related inferences whose consequences require further research
Control Structure
When the system encounters a string the following top level control algorithm is executed
Start with one initial state S where
init
S s bu er is the single constituent htls T C X eop X s T C X init i
init
S s semantic term list interpretations and p enalties are all empty
init
For eachword W in the input
For each lexical entry hW Cat Semi
For each current state S
Make a copy S of S
Add the constituent hCat lex W i to the Bu er of S
App end Sem to the Semantic Term List of S
For eachway S of nondeterministicall y applying the rules of grammar to S section
If the resulting bu er is an admissible one section then
For eachwayofinterpreting S section
Compute the p enalty of the interpretation
unless subsumed byanextantstate Save S
Remove S
Perform discarding pro cedure on the current set of states section
Continue with the next word
Of the states whose bu er has the singleton constituent whose category is a tls
display the most sensible state or states i e the one s with the least p enalty
The category in the initial state has as its result the sp ecial symbol tls top level sentence whichis
not mentioned elsewhere in the grammar This symbol is intro duced mostly for convenience and
should b e thought of as identical to the symb ol s The di erence will b e ignored in the exp osition
X which whenever p ossible The category has as its rst argument the basic category s
creates the exp ectation for a tensed sentence
Bottom Up Reduce Algorithm
The nondeterministic reduce computation is as follows
reduce state S
either
Sasis
or
if there is a reduce step that can b e applied to the bu er of S
then p erform this step and recursively call reduce on the resulting state
or
let RC b e the rightmost constituent of the bu er of S
if the category of RC is of the form Z
where Z matches a zero morpheme e g eop X
then
app end the constituent hZ lex i to the right of the bu er
app end the semantic term list asso ciated with that zero morpheme to S s semantics
recursively call reduce on the resulting state
end if
end
There are twoways of p erforming a single reduce step as discussed in chapter
letX Ybethetwo rightmost constituents in the bu er of S
let XC and YC b e the syntactic categories of X and Y resp ectively
metho d
if there is a combinatory rule R of the form XC YC Z
then
replace X and Y in the bu er with the constituent hZ R X Yi
if rule R has any semantic terms
then app end these terms to the semantic term list of S
metho d
If YC is of the form WnW
then
let XNF b e the right normal form of X
if there exists a right sub constituent RS of XNF such that
the syntactic category of RS is RC and
there is a combinatory rule R of the form RC YC Z
then
replace RS by hZ R RS Yi
if rule R has a nonempty semantic term list
then app end this list to the semantic term list of S
Bu er Admissibility Condition
As discussed in chapter esp ecially sections the adult listener reader has access to a
pro cedure which identi es and discards unviable bu ers such as the DET insults VERB For
y stipulating the the purp oses of this pro ject I circumvent the step of acquiring this pro cedure b
condition in which is adequate for the grammar I use
Bu er Admissibili ty Condition
For every pair of adjacent constituents whose categories are X and Y
No obligatory combinatory rule exists whichcancombine X and Y and
the categories of X and Y are ultimately combinable
All combinatory rules are obligatory except those forward rules n where the left category is
np i e those rules which determine ller gap relations as discussed in section
X and Y are ultimately combinable in case either or or holds
Xisoftheform Z and
m
YisoftheformZ
n
for some m n and some category Z
and YisoftheformAnB
n
there is a combinatory rule which can combine X and AnB
YisoftheformAnB and
n
the right normal form of X has a right sub constituentRS
such that there is a combinatory rule which which can combine RS with AnB
Conditions and anticipate applications of certain backward combinatory rules In
section I argued that semantic terms in particular a term for marking crossing comp osition
a signal for heavy NP shift whichwould b e intro duced by the anticipated rule application
should b e detected immediately and not delayed until the rule is actually applied This is realized
in the implementation
Interpretation
The interpretive comp onent in the current system p erforms only two of the manyinterpretive
functions of its human counterpart It p erforms a simplistic database lo okup op eration for resolv
ing de nite noun phrases against the prior discourse without any so called bridging inferences
see Haviland and Clark It also implements a trivial form of plausibility impl ausibi l ity
inference relying on a hand co ded database of implausible scenarios These inferences are
of interest of course only insofar as their contribution to the evaluation of comp eting analyses
Real World Implausibility
Minimal pairs suchas
a The p o et read in the garden stank
b The p o em read in the garden stank
where a is a garden path but not b demonstrate the reliance of the pro cessor on world
knowledge inferences see section It do es not follow of course that al l ambiguities which
can b e resolved by inference are indeed thus resolved online One could set up arbitrarily complex
puzzles the solutions of which are crucial for resolving a particular ambiguity An accountof
which inferences are su ciently fast so as to direct online ambiguity resolution is far outside
t pro ject I assume that suchan the scop e of the currentwork For the purp oses of the curren
inferential device exists and is able to quickly notice certain obvious semantic incongruities and
alert the interpreter One could think of the N signal in electro encephalograms Garnsey et
See Shastri and Ajjanagadde for one view of fast inference
al as a correlate of the human analog of this incongruity alert I simulate the b ehavior of
such an anomaly detector byanticipating each anomalous situation which will b e encountered by
the system and enco ding that situation by hand A partial list of these situations is as follows
S is the semantic variable of the implausible scenario
scenario description explanation
read S X p o em X Po ems can t read
p o em X Po ems can t read anything read S X
X poem X One can t warn p o ems warn S
stop S X p o em X Po ems can t stop anything
future S yesterday S Anything that happ ened yesterday is not in the future
De nite Reference
In the current system all de nite NPs pronouns names and NPs with de nite determiners
have uniform semantic representations A segment of the semantic term list which b egins
with the term the X ends with the term phrase closed X Between these markers lie semantic
terms Here are some examples
phrase semantic term list
the p o em the X p o em X phrase closed X
she the Y third p ers Y feminine Y singular Y
phrase closed Y
john the Y name of Y john phrase closed Y
his p o em the X third p ers X masculine X singular X
phrase closed X the Y of Y X p o em Y phrase closed Y
the p o em that the Y p o em Y npmo d Y the Z name of Z john
john likes phrase closed Z like W Z Y tns W s phrase closed W
phrase closed Y
Terms suchasname of X Y and p o em X are called restrictive Others such as phrase
closed X and the X are non restrictive as they do not servetonarrowdown the set of p ossible
referents It is assumed that all mo di ers are restrictive i e non attributive
The algorithm for resolving de nite reference is in gure
Some illustrations will make this algorithm s op erations clear
Supp ose that is encountered out of context
The horse shown to the p o et fell
When the rst word the is pro cessed the state has the semantics sub j e the e Since
there are no restrictivesemantic atoms the algorithm do es nothing The next word horse
intro duces a syntactic ambiguity is the phrase the horse closed or not
Recall that sub j X is intro duced by the sub ject typ e raising rule It is not a restrictive semantic atom
Given a database D representing the entities of the prior discourse and relations among them
and given a state S
with semantic term list SEM
interpreter annotation list IA and
p enalty list P
Scan SEM from right to left SEM s atoms re ect the order of the input string
For each o ccurrence O of the X
if accom X or resolved X is in IA then do nothing Already processed
else
let SEM b e the nal segment of SEM which b egins with O
let Q b e the query derived by conjoining all the restrictive atoms of SEM
if the Q is empty
then do nothing Don t look for a referent of a phrase stil l missing its lexical head
else
let C b e the set of values for X for which Q succeeds on D
if C is empty then
closed X app ears in SEM if the term phrase
then add accom X Q to IA
else add accom complex description X to P
end if
else if kCk
add resolved X C to IA where C is the elementofC
closed X do es not app ear in SEM if the term phrase
then add oversp eci ed ref X to P
end if
else if kCk then
closed X app ears in SEM if the term phrase
then
let C b e an arbitrary member of C
add resolved X C toIA
add undersp eci ed ref X to P
end if
end if
end if
end if
end for
Figure De nite Reference Resolution Algorithm
state i
the horse
s e s e nnp e eop e n e n e eop e
s e s e nnp e eop e
s e s e nnp e
sub j e the e horse e phrase closed e
state ii
the horse
s e s e nnp e eop e n e n e
s e s e nnp e eop e
sub j e the e horse e
In state i the parser nondeterministically chose to close the NP The discourse representation
is queried to nd all things X whichmatch the query horse X Since the discourse representation
is empty the result of this query is the empty set The following annotation is therefore added to
the state s Interpreter Annotations List accom e horse e No p enalties apply In state ii
the parser chose not to close the NP The p enalty accom complex description e is added to
the state s p enalty list since the state s bu er enco des a commitment to restrictive p ostmo di ers
for the NP
The next word shown resolves the closure nonclosure ambiguity as it triggers a restrictive
reduced relative clause When the reduced relative clause is nished again there is a closure
ambiguity asfollows
state iii
horse shown to the p o et the
s e s e nnp e eop e n e n e n e nn e eop e
n e
s e s e nnp e eop e
s e s e nnp e
sub j e the e horse e show e e e tns e en npmo d e to e e the e p o et e
phrase closed e phrase closed e
Inumb er the states solely for ease of reference
state iv
the horse shown to the p o et
s e s e nnp e eop e n e n e n e nn e
n e
s e s e nnp e eop e
sub j e the e horse e show e e e tns e en npmo d e to e e the e p o et e
closed e phrase
State iii gets the interpreter annotation
accom e horse e show e e e tns e en to e e p o et e
ignoring the indep endent pro cesses of resolving the NP the p o et State iv is not yet closed
so it do es not get this accommo dation annotation Instead it gets another accom complex
description e p enalty which is subsequently removed by a duplicate removal pro cedure
The presence of the main verb fell disambiguates the closure question this time by selecting
the closed state iii
Supp ose the prior discourse contains two horses intro duced for example by the passage
There were two horses b eing shown to a prosp ective buyer One was raced in the
meadow and the other was raced past the barn
In this context the interpretation of pro ceeds di erently After encountering the rst two
words the horse the parser constructs states i and ii ab ove The query of horse X now
returns two p ossible candidates call them horse and horse State i in which the NP is
marked as closed is incapable of acquiring additional information to identify a unique referent
for the horse The interpreter then cho oses one of these arbitrarily say horse and adds the
annotation resolved e horse to the interpreter annotations of state i Noting this premature
choice it adds the p enalty undersp eci ed ref e to the state s p enalty list State ii is not
closed so the algorithm decides to wait for additional individuating information
The next few words which the pro cessor encounters are shown to the p o et When interpreting
state ii the interpreter decided to wait for information to distinguish horse from horse But
this restrictive clause is infelicitous It refers to a p o et which is not in the current discourse and
must b e accommo dated When the algorithm applies the query
horse X show Y Z X tns Y en to Y P poet P
it nds no matching candidates for the variable X As in ab ove the interpreter adds an
annotation accommo dating the de nite description Also it applies the p enaltyaccom complex
description e
Had the restrictiverelative clause b een appropriate e g had the sentence b een
The horse raced past the barn fell
The set of discourse elements satisfying the query
horse X race Y Z X tns Y en past Y P barn P
would have b een the singleton set horse The algorithm would add the annotation resolved e
horse to the interpreter annotations list and apply no p enalties
Detecting the End of a Phrase
In this section I provide the rationale for the end of phrase mechanism used in the current
implementation
The de nite reference resolution algorithm relies on the accurate signaling of the end of an NP
Without the ability to identify the b oundaries of a noun phrase the pro cessor would b e unable
to distinguish from the various assertions made of a semantic variable those which are within
the scop e of the determiner from those which are not
Since this algorithm ts squarely within the interpretation mo dule it do es not have direct
access to the syntactic representation so identi cation of the end of the phrase cannot b e simply
p erformed bychecking that a particular no de or constituent is no longer on the right frontier of
the emerging analysis The detection of an end of a noun phrase must therefore b e identi ed by
the syntactic pro cessor and passed to the interpreter using the only available data path namely
the sense semantics Given the tremendous variation of NP structure in the world s languages it
is natural to place the burden of end of phrase detection with the language particular grammar
not with the pro cessor in general
How can can phrase b oundary b e implemented in a CCG In semantics of the usual sort where
a constituent is assigned a meaning term or a logical form the mechanism of quanti er
scop e is available and nothing sp ecial is required However in the semantic term list approach
whichIhave adopted here see section scop e is rather di cult to express in the sense
semantics One way to implement phrase closure detection in CCG is to disallow recursive
p ostmo di cation of NPs and simply state in advance in the lexical entry for a determiner or a
noun exactly what the constituents of the NP are This is rather awkward and maywell b e
missing the generalization that p ost head adjectival apply recursively The other wayusto
use the narrowly constrained zero morpheme scheme as I have presented ab ove I use the same
zero morpheme eop for clauses as well This move is not forced byanything and is adopted
mostly for uniformity It happ ens to play a convenientroleinavoiding certain shortcomings
whichwould otherwise arise from the wayrevealing is implemented in Prolog
An Example
The pro cessor consists of the comp onents discussed ab ove comp etence grammar control
structure parsing algorithm and interpreter as well as state adjudication algorithm Before
It is not clear to me whether restrictive adjectivals really can recurse but they are commonly assumed to do
so
turning to the details of this nal comp onent it would b e b est to illustrate the op eration of the
pro cessor so far with an example In this example state adjudication should b e thoughtofas
working out by magic In section I present a decision pro cedure for it
Let us b egin with the string
The p o et read in the garden stank
encountered out of context
Before anywords are pro cessed the parser starts with one initial state whose bu er has one
constituent
htls T C E eop E s T C E lex init i
The rst word the is encountered It has two lexical entries corresp onding to the original
determiner category and the sub ject typ e raised determiner resp ectively
np s e eop e n N e
s s e s s e nnp s e eop e n s e
The nondeterministic reduce algorithm results in three states
state the initial category and the non typ e raised category for the determiner
the init
tls T C E eop E s T C E np s e eop e n N e
state the initial category and the typ e raised category for the determiner
init the
tls T C E eop E s T C E s s e s s e nnp s e eop e n s e
state initial category and typ e raised determiner combined
init the
tls T C E eop E s T C E s s e s s e nnp s e eop e n s e
tls s e eop e s s e nnp s e eop e n s e
State is ruled out by the second clause of the Bu er Admissibili ty Condition which
requires adjacent constituents to b e ultimately combinable State is ruled out by the rst
clause of which requires that adjacent constituents not b e immediately combinable State
is therefore the only one which the parser outputs Since it do es not have its head noun yet
the interpreter do es not add anyinterpretations or p enalties to this state For the rest of this
example I ignore the initial state and pretend that the current state has the category
s s e s s e nnp s e eop e n s e
I also omit parser states which are ruled out by the Bu er Admissibili ty Condition
The next word p o et is encountered It gives rise to
state
Bu er s s e s s e nnp sg e eop e
Semantics sub j e the e p o et e
The parser also nondeterministically p osits a zero morpheme following p o et yielding
state
Bu er s s e s s e nnp sg e
closed e Semantics sub j e the e p o et e phrase
In state Because the de nite phrase e is closed the interpreter accommo dates a p o et In
state the interpreter anticipates further restrictive mo di ers so it p enalizes the state for hav
ing to accommo date a complex NP The results are
state
Bu er s s e s s e nnp sg e eop e
Semantics sub j e the e p o et e
Interpretation
Penalties accom complex description e
state
Bu er s s e s s e nnp sg e
closed e Semantics sub j e the e p o et e phrase
Interpretation accom e p o et e
Penalties
Despite the p enalty in state b oth states are maintained for now Also b oth states and
incur a p enalty for having a new NP the p o et in sub ject p osition Because this p enaltywill
apply to every state in the rest of this example it will turn out to b e irrelevant so I omit it
The next word read is many ways ambiguous The untensed verb reading and the present
tense non rd p erson singular reading are ruled out b ecause their features con ict with the
nnp sg requirement of the sub ject NP category Three readings remain past tense s
intransitive past tense transitive and past participle acting as head of a reduced relative clause
The rst two combine with state to yield states and resp ectively The third is added to
state to yield state
state
B s ed e
S sub j e the e p o et e phrase closed e read e e e tns e ed
I accom e p o et e
P
state
B s ed e np s s e
S sub j e the e p o et e phrase closed e read e e e tns e ed
I accom e p o et e
P
state
B s s e s s e nnp sg e eop e
n sg e nn sg e s s s e ns s s e
S sub j e the e p o et e read e e e tns e en npmo d e
I
complex description e P accom
Recall that the category of this state is actually tls ed e eop e So in addition to state the pro cessor
constructs state where it p osits an end of phrase morpheme signaling the end of the main clause This state
has no continuation and it disapp ears when the next word is encountered
Notice that state satis es clause of the Bu er Admissibility Condition That is the
category n is revealed from the right hand edge of the rst constituent the p o et and this
category may b e mo di ed by the second constituent read when the latter has received all of
its arguments namely the adverbial in the garden
The next word in is four wayambiguous as shown in the table in section Of these only
one category sentential p ost mo di er is not ruled out by the bu er admissibility condition
States and then b ecome states and resp ectively
state
B s ed e s ed e ns ed e np s s e
closed e read e e e tns e ed S sub j e the e p o et e phrase
in e e
I accom e p o et e
P
state
B s ed e np s s e s ed e ns ed e np s s e
closed e read e e e tns e ed S sub j e the e p o et e phrase
shifted e e in e e h
I accom e p o et e
P shifted past non given e
state
B s s e s s e nnp sg e eop e n sg e nn sg e np s s e
S sub j e the e p o et e read e e e tns e en npmo d e in e e
I
P accom complex description e
State incurs a p enalty for heavy NP shift past material whichisnotgiven in the discourse
see section States and are discarded b ecause while they each carry a p enalty state
do es not By discarded state the pro cessor has resolved the main verb reduced relative
ambiguity of read selecting the main verb analysis By discarding state it has further
committed to the intransitive use of this verb The consequence of the latter commitmentwill
b e discussed in section
The word the yields state from state
state
B s ed e s ed e ns ed e eop e n s e
closed e read e e e tns e ed S sub j e the e p o et e phrase
in e e the e
I accom e p o et e
P
The word garden leads to the familiar closure ambiguity in states and
state
B s ed e s ed e ns ed e eop e
S sub j e the e p o et e phrase closed e read e e e tns e ed
in e e the e garden e
I accom e p o et e
complex description e P accom
state
B s ed e
closed e read e e e tns e ed S sub j e the e p o et e phrase
in e e the e garden e phrase closed e
I accom e garden e accom e p o et e
P
But neither state is compatible with the next word stank The lack of surviving states indicates
the garden path e ect of the sentence
A Consistent Theory of Penalties
Inow construct an adjudication algorithm a pro cedure to evaluate the set of pro cessor states
and based on the kind and numb er of p enalties which they have decide which if any to discard
I b egin by considering the examples listed in section the p enalties that the pro cessor assigns
each analysis and the appropriate action at eachmoment I then present one of many p ossible
algorithms that t these data
Desired Behavior
For convenience I refer to p enalties by their numb er as follows
implausibil i ty
ref undersp eci ed
oversp eci ed ref
accom complex description
new sub ject
arg light mo di er heavy
shifted past non given
One wife context
The psychologist told the wife that he disliked Florida
that relativizer
that complementizer ok
The psychologist told the wife that he disliked that he liked Florida
that relativizer gp
that complementizer
two wife context
The psychologist told the wife that he disliked that he liked Florida
that relativizer ok
that complementizer
two wife context
The psychologist told the wife that he disliked that he liked Florida
that relativizer
that complementizer gp
out of context
The p o et read in the garden stank
main verb
reduced relative gp
The p o em read in the garden stank
main verb
reduced relative ok
The bird found in the nest a nice juicy worm
reduced relative
main verb ok
The bird found in the nest died
reduced relative ok
main verb
context what did the bird nd in the nest
The bird found in the nest a nice juicy worm
reduced relative
main verb ok
The bird found in the nest died
reduced relative gp
main verb
context Fred found a bird in a nest and Bill found one in the garden
The bird found in the nest a nice juicy worm
reduced relative
main verb gp
The bird found in the nest died
reduced relative ok
main verb
out of context
Without her contributions dwindled
determiner
ob ject pronoun gp
Without her contributions the charityfailed
determiner ok
ob ject pronoun
The p o et said that the psychologist fell yesterday
low attachment ok
high attachment
The p o et said that the psychologist will fall yesterday
low attachment
high attachment gp awkwardness
Fitting The Data
The simplest conceivable state discarding algorithm is
If at least one state has no p enalties
then discard every state that has one or more p enalties
There are two problems with The rst has to do with the fact that not all p enalties are
of equal strength scenario shows that p enalty is stronger than p enalty Using similar
reasoning one can deduce the following constraints among p enalty strengths s stands for the
strength of p enalty
scenario constraint provided
s s
s s
s s
These constraints underdetermine the ranking of the p enalties with resp ect to strength The
following is one of manyschemes which are consistent with the constraints It uses two strength
levels the minimum p ossible
p enalty strength in numb er of p oints
The second problem with is that of timing Scenarios and show that sometimes a state
which has a p enalty is not discarded even when it is comp eting with one that has none In
these scenarios information whicharrives one or twowords after the rst detection of a p enalty
p enalty is broughttobearandprevents discarding This is in contrast with scenario where
as so on as a p enalty p enalty is detected the o ending state is discarded I let each p enalty
typ e carry a graceperiod aninterval of time When the p enalty is detected a countdown
clo ck asso ciated with it is started The p enalty is ignored until its clo ckreaches zero
The scenarios ab oveprovide the following constraints on the assignment of grace p erio ds where
g stands for the grace p erio d of p enalty measured in words
scenario constraint provided
g
g
g
g g
g g
g g
g
No timing constraintisprovided by scenario b ecause the interaction b etween p enalty and
o ccurs at the end of the sentence
These constraints again underdetermine the grace p erio ds Here is one solution which minimizes
the grace p erio d values
In fact I am making a great simpli cation by treating all instances of a p enaltyashaving the same strength
For example implausibi li ty is surely a graded judgement as is the degree of complexity of accommo dation
Using the word as a measure of time is intended to b e an approximation Clearly the time course of pro cessing
function words is very di erent from that of pro cessing long or novel contentwords Given the currently available
psycholingui sti c evidence only a crude timing analysis is p ossible at this time But see Trueswell and Tanenhaus
for some preliminary work at trying to understand the time course of the interaction of comp eting p enalties
constraints in their terms
p enalty name strength grace p erio d
implausibili ty
undersp eci ed ref
oversp eci ed ref
accom complex description
new sub ject
heavy arg light mo di er
shifted past non given
The revised algorithm then is
For each state let its p enalty score b e the sum of the strengths of all
p enalties whose grace p erio ds have passed
Find the minimum score
Discard each state whose score is ab ove the minimum
It must b e emphasized that the algorithm and parameters serve merely to demonstrate the
consistency of the set of p enalties listed in the b eginning of this section so the particular
numbers oreven exactly what they measure should not b e construed as a prop osed theory
A Prediction
Despite the preliminary nature of the sp eci cs of the state discarding algorithm it is nevertheless
p ossible to deriveaninteresting prediction from the system as develop ed so far in particular
from the interaction of the choice of the theory of syntax and the state discarding pro cedure
The account presented here assumes p enalties for heavy shift that is infelicitous in context and
for accommo dating a complex NP Recall the example in section The verb read has three
categories a reduced relative clause and two main verb categories transitiveandintransitive
Consider what the account do es when faced with each sentence in out of context
a The p o et read in the garden stank
b The p o et read in the garden a lengthy article ab out Canadian earthworms
In a the complex NP accommo dation p enalty correctly excludes the reduced relative anal
ysis resulting in a garden path What is the predicted status of b Given that the
reduced relative analysis is discarded one would exp ect the main verb analysis to p ersist But
CCG has two completely separate main verb analyses The transitive analysis requires heavy
NP shift which is deemed infelicitous out of context The surviving analysis is of the intransitive
verb and cannot cop e with the shifted NP So the account predicts a garden path in b
This prediction arises of course b ecause of the lexicalized nature of CCG every combinatory
p otential of a word is treated as a separate lexical entry Inotherwords CCG do es not distin
een categories e g sub categorization from ma jor di erences e g guish small di erences b etw
main verb versus reduced relative clause
So while CCG predicts a garden path for b a more traditional phrase structure theory
of grammar might not dep ending on whether it distinguishes analyses on the basis of lexical
sub category The garden path status of b is an empirical one and necessitates teasing
apart any pro cessing di culty asso ciated with the infelicity of the heavy NP shift from truly
syntactic parsing e ects indicative of a garden path It remains for future research
Summary
Using the meaning based criteria develop ed in chapters through referential felicity felicity
with resp ect to givenness plausibili ty and the parsing algorithm presented in chapter I
have presented a simple mo del of the pro cess of sentence comprehension The p oint of this
demonstration is to show that it is p ossible to construct a simple sentence pro cessor whichcan
account for signi cant subset of the data available ab out when garden paths arise in English
This enterprise is largely successful the data structures and algorithms needed by top level of the
architecture are obvious and straightforward Complexity arises from the sp eci c requirements
necessitated by the grammar formalism CCG and by the scop e of the state discarding criterion
The latter is severely underdetermined by the available data
The long term goals of this work is to provide a detailed mo del of sentence pro cessing one which
makes clear and testable prediction While this is still a long way o I havenevertheless shown
that already it is p ossible to make some sort of predictions from the interaction of the various
ingredients
The program as describ ed in this chapter is written in Quintus Prolog and is called arfi
Ambiguity Resolution From Interpretation It is accompanied by a graphical user interface
which provides an easy to use facility for insp ecting the execution trace of the pro cessor on a
particular input string The insp ector program called the viewer is written for the X window
system and requires Common LISP and the software package CLIM arfi and viewer are have
b een available on the Internet by anonymous FTP They are on the host ftp cis up enn edu in
the directory pub niv thesis
Chapter
Conclusion
Of the class of computational functions p erformed by the human language faculty ranging from
phonetics to the so cial activityofcommunication I have considered two
parsing the application of the rules of syntax to identify the relations among the words in the
input string
interpretation the up dating of the hearer s mental mo del of the ongoing discourse based on
the sense semantic relations recovered by the parsing pro cess
Ihave argued for a particular view of the interaction b etween these two functions First I
adopted the uncontroversial assumption almost uncontroversial see Marslen Wilson and Tyler
that parsing and interpretation o ccur in separate mo dules and that these mo dules interact
through well sp eci ed channels the parser sends nothing but sense semantic representations to
the interpreter and the interpreter sends the parser nothing but feedback ab out sensibleness of
the various analyses
Second I adopted the more controversial assumption that the parser computes all of the gram
matically licensed analyses of the string so far and sends them all to the interpreter for evalua
tion in parallel I claimed that the parser do es not provide its own ranking or evaluation of the
analyses it constructs by applying structurally stated preference criteria that all observable
preferences among ambiguous readings stem from principles of the linguistic comp etence princi
ples such as Grice s maxims of quantity and manner for evaluating the felicity of de nite referring
expressions the comp etence principle in English to place high information volume constituents
after lowvolume ones to use sub ject p osition for enco ding given information etc I did not
explore in detail the question of whether the parsing comp onent applies some evaluation of the
various analyses based on strengths of various alternatives in the comp etence grammar While
ord Bresnan and Kaplan Trueswell Tanenhaus verb sub categorization preferences e g F
and Kello can b e ascrib ed either to the lexical entries part of grammatical comp etence
or a deep er representation of the concepts attached to them there are some preference phe
nomena which seem to necessitate grammatically sp eci ed strength of preference Kro ch
This issue remains for future research
Third I investigated the design of the parser My aim was to identify the simplest design p ossi
ble I investigated Steedman s thesis that conceiving of syntactic knowledge of language
as a Combinatory Categorial Grammar CCG allows one to construct a parser that is signif
icantly simpler than what would b e needed for traditional grammars while still maintaining
the input output b ehavior necessary to function as the syntactic parsing mo dule in the overall
sentence pro cessing system design It turned out that designing a parser for CCG runs into its
own collection of complexities Certain of these complexities detection of inevitable ungram
maticality detection of inevitable word order non canonicality in the form crossing comp osition
identifying optional combinations e g picture noun extraction can b e elegantly handled by
assuming that the ability to parse in adults is comp osed of an innate ability to put grammatical
constituents together and an acquired skill of quickly anticipating the consequences of certain
combinations One nal complexity the interaction of CCG s derivational equivalence with
the incremental analysis necessary for timely interpretation necessitated assuming that the
history of the derivation is prop erly a comp onent of a grammatical analysis and augmenting the
rep ertory of the parser with an op eration which explicates the interchangeability of derivational
equivalent analyses by manipulating the history of the derivation
Fourth I have constructed a computational simulation of the sentence comprehension pro cess
whichallows one to evaluate the viability of the central claim of the dissertation that the
syntactic pro cessor blindly and transparently computes all grammatical analyses and ambiguity
resolution is based on interpretation This simulation serves as a computational platform for
exp erimenting with various analysis pruning strategies in the interpreter It has shown that at
the moment the available psycholinguistic data greatly underdetermines the precise strategy
but some empirical predictions do emerge
the dissertation gives rise to three sp eci c empirical questions
Given the example dialog in on page whichshows that discourse status can a ect
p erceived information volume just howmuch of the information volume as op erationaliz
able by observing attachment preferences can b e accounted for by discourse factors and
howmuch of it is irreducible to the form of the constituent the amount of linguistic
material and other syntactic attributes such as grammatical category
Do es Disconnectedness theory playanyroleatallinambiguity resolution To what
extentisAvoid New Sub jects really resp onsible for the data cited in chapter As
discussed in detail in section if one were to re run the second exp eriment rep orted
by Stowe ruling out the instrumental reading and still get implausibil ity e ects in
the inanimate condition one would have an empirical basis to rule in Avoid New Sub jects
and rule out Disconnectedness theory
Is CCG correct in its equal treatment of ma jor category and sub category distinctions
That is can the pro cessor b e made to commit to an intransitive analysis for a verb thus
garden pathing on its direct ob ject as predicted in on page
Note that it is logically p ossible to de ne a more extreme condition on a parsimonious parser This condition
would disallow op erations and representations which are not strictly de ned bythewell formedness rules of the
grammar Since CCG do es not strictly sp eaking de ne well formed analyses only well formed constituents
and since it do es not explicitl y related equivalent derivations this view of grammar is not compatible with the
derivation rewrite algorithm I have presented
App endix A
Data from Avoid New Sub ject
investigation
Brown Corpus
Sub jects Non Sub jects
givenness status tc rc tc rc matrix tc rc tc rc matrix
empty category
pronoun
proper name
definite
indefinite
not classified
total
Wall Street Journal Corpus
Sub jects Non Sub jects
givenness status tc rc tc rc matrix tc rc tc rc matrix
empty category
pronoun
proper name
definite
indefinite
not classified
total
Non zero cells of empty categories in non p ost ZERO complementizer sub jects are due to an
notation errors in the corpus
App endix B
A Rewrite System for Derivations
In this app endix I de ne a formal system DRS a rewrite system for CCG derivations as
sketched in section I then prove that DRS preserves the semantics of a derivation and show
that it can form the basis of a correct and e cient algorithm for computing the right frontier
of a derivation
De nition Two derivations are equivalent just in case the category of their resp ective ro ots are
equal
Inow give the de nition of DRS DRS allows one to describ e equivalence classes of derivations
and provide the means of picking out one representative from each equivalence class
Given the set D of valid derivations de ne the relation D D to hold b etween a pair of
derivations d d just in case exactly one application of one of the derivation rewrite rules in
and to some no de in d yields d
Any subtree of a derivation whichmatches the left hand side of either or is called
a redex The result of replacing a redex by the corresp onding right hand side of a rule is called
the contractum A derivation is in normal form NF if it contains no redex
Y jZ jZ c jY Y b W X a X jY
m n m m
m
m
W jY jY Y B ab
m m
n
n m
W jY jY jZ jZ B B ab c
m n
W X a X jY jY Y b Y jZ jZ c
m m m n
n
n
X jY jY jZ jZ B bc
m n
m n
m n n
W jY jY jZ jZ B a B bc
m n
For an overview of rewrite systems the reader is referred to Le Chenadec esp ecially section
Y jZ jZ c X jY jY nY b WnX a
m n m m
n
n
X jY jY jZ jZ B bc
m n
m n
m n n
W jY jY jZ jZ B a B bc
m n
Y jZ jZ c X jY jY nY b WnX a
m n m m
m
m
W jY jY nY B ab
m m
n
n m
W jY jY jZ jZ B B ab c
m n
Lemma preserves equivalence of derivations
pro of When the semantic terms from the ro ots of the left hand derivation and right hand
derivation are compared by applying each of them to su ciently many arguments so that all
reductions take place the results are identical
n m
B B ab cd d
n m
m
B ab cd d d d
n n n m
a b cd d d d
n n n m
m n n
B a B bc d d
n m
n
a B bcd d
n m
a b cd d d d
n n n m
Let b e the converse of Let be Let b e the re exive transitive closure
of and similarly the re exive transitive closure of and the re exive transitive
closure of
Note that is an equivalence relation and that d d d d are equivalent but
the converse do es not hold b ecause two categories could b e accidentally equivalent an o dd
prop erty for a linguistic analysis to have but a p ossible one nonetheless The reader mayverify
that no other combinatory rules may b e substituted for in and in to yield a
semantics preserving derivation rewrite rule In particular the twocombinatory rules must b e
of the same directionality
Theorem For a derivation with n internal no des every sequence of applications of is
nite and is of length at most n n
pro of Every derivation with n internal no des is assigned a p ositiveinteger score whichis
b ounded by n n An application of is guaranteed to yield a derivation with a lower
ation as score This is done by de ning the functions weight and score for each no de of the deriv
follows a b bc d c a f
e e’
Figure B Schema for one redex in DRS
if x is a leaf no de
weight x
weight left child x weight right child x otherwise
if x is a leaf no de
score x
score left child x score right child x weight left child x otherwise
Each application of decreases the score of the derivation This follows from monotonic
dep endency of the score of the ro ot of the derivation up on the scores of each sub derivation and
from the fact that lo cally the score of a redex decreases when is applied In gure B a
derivation is schematically depicted with a redex whose sub constituents are named a b and c
Applying reduces the score e hence the score of the whole derivation
in redex
weight d weight a weight b
score b weight a score d score a
score e score d score c weight d
score a score b weight a score c weight a weight b
score a score b score c weight b weight a
in contractum
score f score b score c weight b
score e score a score f weight a
score a score b score c weight b weight a
score a score b score c weight b weight a
n is also the lower b ound on sequences of application of Inow show that n
A left chain is either a leaf no de or a derivation whose left child is a left chain and whose
right child is a leaf no de A right chain is either a leaf no de or a derivation whose right child
is a right chain and whose left child is a leaf no de A quasi right chain is a derivation whose
right child is a leaf and whose left child is a right chain n-2 n n-1
n n+1
Figure B Normal form computation of a quasi right chain
Lemma A quasi right chain of n internal no des can b e rewritten using n application of
to a right chain
pro of Atevery p oint in the rewriting op eration there is only one redex This is depicted in
gure B
Lemma A left chain C of n internal no des can b e rewritten to a right chain using a sequence
of exactly n n applications of
pro of By induction on n
n C is already a right chain steps are required
n C is a redex One application of rewrites it into a right chain
n Supp ose this is true for all m n Apply as follows Rewrite the left child of C to a
right chain in n n steps The result of this is a quasi right chain of n internal
no des which can b e rewritten to a right chain using n applications of The total
numb er of applications of is
n n n n n n n n n
n
A rewrite system is strongly normalizing SN i every sequence of applications of is nite
Corollary DRS is SN
pro of Immediate corollary of theorem
So far I have shown that nondeterministic computation of the right branching NF of a derivation
is quite tractable quadratic in the size of the derivation I will nowshow that this is even so
on a deterministic machine
A rewrite system is Church Rosser CR just in case
x y x y z x z y z
A rewrite system is Weakly Church Rosser WCR just in case
x y w w x w y z x z y z
Lemma DRS is WCR
ation with two distinct redexes x and y yielding the two distinct pro of Let w b e a deriv
derivations w and w resp ectively There are a few p ossibilitie s
case x and y have no no des in common There are three sub cases x could dominate y
include y as a sub constituent x could b e dominated by y orx and y could incomparable
with resp ect to dominance Either way it is clear that the order of application of makes
no di erence
case x and y share no des Assuming that x and y are distinct and without loss of generality
that y do es not dominate x wehave the situation depicted in gure B Note that all three
internal no des in gure B are of the same combinatory rule either or In this case
there do es exist a derivation z such that w z w z This is depicted in gure B
Lemma Newman W CR SN CR
Theorem DRS is CR
pro of Follows from lemmas and corollary
Corollary CR x y x y x y are NFs x y Y
X
Figure B When two redexes are not indep endent
d c b a
cb d d cb a a
cb The arrows are annotated a by the sub-structure to ba which they are applied. d c
d
Figure B Why DRS is weakly Church Rosser
pro of By contradiction supp ose CR x y x y x y are NFs x y Then
z x z y z Given that x is a NF x z entails that x z Similarly y z This
contradicts the assumption that x y
Therefore every DRS equivalence class contains exactly one NF derivation It follows that any
deterministic computational path of applying will lead to the normal form
As for the e ciency of computing the rightbranching NF for a derivation of n internal no des
theorem shows that for a derivation of n internal no des every sequence of applications of is
at most n n steps long This is the worst case which arises from applying as far away
from the ro ot as p ossible An insp ection of the de nition of score suggests that applying as
close to the ro ot as p ossible yields the largest decrease in score since weight a is maximized
In fact in case it is always grammatically p ossible to apply to the closest redex to the ro ot
every derivation has a CTR closest to ro ot rewrite sequence of length O n The pro of requires
de ning a function which measures the numb er of CTR rewrite steps that a derivation requires
to reach NF Let us rst de ne the function which applies once to the closest redex
ctr
to the ro ot of its argument
x if x is a leaf no de
combine left child x right child x if left child x is a leaf
ctr
x combine left child left chil d x otherwise
ctr ctr
bine right child left chil d x com
right child x
Let cost x b e the numb er of times that must b e iterated on x so as to yield an NF
ctr
de nes cost x by the following recurrence equations
ctr
if x is a leaf no de
cost right child x if left child x isaleaf
cost combine left child left chi ld x otherwise cost x
combine right child left child x
right child x
Observe that in the third case subsequent applications of will pro cess all of left child left
ctr
c hild x then pro ceed to pro cess right child left chil d x and nally pro cess right child x
This is illustrated in gure B
The cost of doing these three steps can b e accounted for separately
cost x
cost combine left child left chi l d x l
cost combine right child left chil d x l
cost right child x
where l is a dummy leaf no de
It is now p ossible to proveby induction on the derivation tree that
Figure B One application of
ctr
cost x x minus the depth of the rightmost leaf in x
where D is the number of internal no des in the derivation D
Base cases
x is a leaf cost x
x has one internal no de cost x
Induction
Supp ose true for all trees of fewer internal no des than x Let d b e the depth of the rightmost
leaf of x
Case left child x is a leaf cost x cost right child x n d n d
Case left child x is not a leaf
Let a b c b e left child left chil d x right child left child x right child x resp ectively Note
that x a b c
cost x cost combine a l cost combine b l cost c
a b c depth of rightmost leaf in c
x d
x d
So while the worst case sequence of applications of is quadratic in the size of derivation the
b est case p ossible as long as the grammar allows it is linear
Bibliography
Abb ott Barbara Right no de raising as a test for constituentho o d Linguistic Inquiry
Adams Beverly C Jr Charles E Clifton and Don C Mitchell Lexical guidance in
sentence parsing
Aho Alfred and S C Johnson LR parsing ACM Computing Surveys
Altmann Gerry T Alan Garnham and Yvette Dennis Avoiding the garden path Eye
movements in context Journal of Memory and Language
Altmann Gerry T and Mark J Steedman Interaction with context during human
sentence pro cessing Cognition
Aone Chinatsu and Kent Wittenburg Zero morphemes in uni cation based combinatory
categorial grammar In Proceedings of the th Annual Meeting of the Association for
Computational Linguistics
Bach Emmon Colin Brown and William D Marslen Wilson Crossed and nested dep en
dencies in Dutch and German A psycholinguistic study Language and Cognitive Processses
Bever Tom G The cognitive basis for linguistic structures In John R Hayes Ed
Cognition and the Development of Language chapter New York Wiley
Boland Julie Michael Tanenhaus Greg Carlson and Susan Garnsey Lexical pro jection
and the interaction of syntax and semantics in parsing Journal of Psycholinguistic Research
Brill Eric A simple rule based part of sp eech tagger In Proceedings of the Third Con
ference on Applied Computational Linguistics Trento Italy
Chomsky Noam Lectures on Government and Binding Dordrecht Foris
Chomsky Noam Know ledge of Language Its Nature Origin and Use New York
Praeger
Chomsky Noam and George Miller Intro duction to the formal analysis of natural lan
guages In R Duncan Luce Rob ert R Bush and Eugene Galanter Eds Handbook of
Mathematical Psychology Vol I I New York Wiley
Church Kenneth W A sto chastic parts program and noun phrase parser for unrestricted
texts In Proceedings of the Second Conference on AppliedNatural Language Processing
Connine Cynthia Fernanda Ferreira Charlie Jones Charles Clifton and Lyn Frazier
Verb frame preferences Descriptive norms Journal of Psycholinguistic Research
Cowp er Elizab eth A Constraints on Sentence Complexity A Model for Syntactic Pro
cessing PhD thesis Brown University Providence Rho de Island
Crain Stephen and Janet D Fo dor How can grammars help parsers In David Dowty
Lurie Kartunnen and Arnold Zwicky Eds Natural language parsing Psychological Com
putational and Theoretical Perspectives Cambridge University Press
Crain Stephen and Mark J Steedman On not b eing led up the garden path the
use of context by the psychological syntax pro cessor In David Dowty Lurie Kartunnen
and Arnold Zwicky Eds Natural Language Parsing Psychological Computational And
theoretical Perspectives Cambridge University Press
Curry Haskell B and Rob ert Feys Combinatory Logic Vol I Amsterdam North
Holland
Davidson Donald The logical form of action sentences In Nicholas Rescher Ed The
Logic of Decision and Action University of Pittsburgh Press
Dowty David Typ e raising functional comp osition and non constituent conjunction In
Richard T Oehrle Emmon Bach and Deirdre Wheeler Eds Categorial Grammars and
Natural Language Structures Reidel
Eady J and Janet D Fo dor Is center emb edding a source of pro cessing di culty In
Presented at the Linguistics Society of AmericaAnnual Meeting
Earley Jay An e cientcontext free parsing algorithm Communications of the Associa
tion for Computing Machinery
Ferreira Fernanda and John M Henderson Use of verb information in syntactic parsing
Evidence from eyemovements and word by word self paced reading Journal of Experimen
tal Psychology Learning Memory and Cognition
Fo dor Janet D Empty categories in sentence pro cessing Language and Cognitive
Processses SI
Fo dor Jerry Thomas G Bever and Merrill Garrett The Psychology of Language An
Introduction to Psycholinguistics and Generative Grammar New York McGraw Hill
Ford Marilyn Joan Bresnan and Ronald Kaplan A comp etence based theory of syntactic
MIT closure In Joan Bresnan Ed The Mental Representation of Grammatical Relations
Press
Frank Rob ert E Syntactic Locality and TreeAdjoining Grammar Grammatical Acqui
sition and Processing Perspectives PhD thesis UniversityofPennsylvania
Frazier Lyn On comprehending Sentences Syntactic Parsing Strategies PhD thesis
University of Massachusetts
Frazier Lyn Exploring the architecture of the language pro cessing system In Gerry T
Altmann Ed Cognitive ModelsofSpeech Processing MIT press
Frazier Lyn and Janet D Fo dor The sausage machine A new two stage parsing mo del
Cognition
Frazier Lyn and Keith Rayner Making and correcting errors during sentence compre
hension Eyemovements in the analysis of structurally ambiguous sentences Cognitive
Psychology
Garnsey Susan M Melanie A Loto cky and George W McConkie Verb usage knowledge
in sentence comprehension Novemb er
Garnsey Susan M Michael K Tanenhaus and Rob ert M Chapman Evoked p otentials
and the study of sentence comprehension Journal of Psycholinguistic Research
Geach Peter AProgram for Syntax Vol ofSynthese Reidel
Gibson Edward A F A Computational Theory of Human Linguistic Processing Memory
Limitations and Processing Breakdown PhD thesis Carnegie Mellon University May
Gorrell Paul Establishing the lo ci of serial and parallel e ects in syntactic pro cessing
Journal of Psycholinguistic Research
Grice H Paul Logic and conversation In Peter Cole and Jerry Morgan Eds Syntax
ork Academic Press and Semantics III Speech Acts New Y
Haddo ck Nick Incremental interpretation and combinatory categorial grammar In
Proceedings of the th International Joint ConferenceonArti cial Intel ligence
Haddo ck Nick Incremental Semantics andInteractive Syntactic Processing PhD thesis
Univesity of Edinburgh
Haviland Susan E and Herb ert H Clark What s new acquiring new information as a
pro cess in comprehension Journal of Verbal Learning and Verbal Behavior
Hawkins John A parsing theory of word order universals Linguistic Inquiry
Hepple Mark Metho ds for parsing combinatory grammars and the spurious ambiguity
problem Master s thesis University of Edinburgh
Hepple Mark E cient incremental pro cessing with categorial grammar In Proceedings
of the th Annual Meeting of the Association for Computational Linguistics
Hepple Mark and Glyn Morrill Parsing and derivational equivalence In Proceedings of
the Annual Meeting of the European Chapter of the Association for Computational Linguis
tics
Hickok Greg Parallel parsing Evidence from reactivation in garden path sentences
Journal of Psycholinguistic R esearch in press
Hindle Don and Mats Ro oth Structural ambiguity and lexical relations Computational
Linguistics
Hobbs Jerry Mark Stickel Douglas App elt and Paul Martin Interpretation as ab duc
tion Arti cial Intel ligence Journal
Hobbs Jerry R Ontological promiscuity InProceedings of the rdAnnual Meeting of
the Association for Computational Linguistics
Holmes Virginia M Alan Kennedy and Wayne S Murray Syntactic structure and the
garden path Quarterly Journal of Experimental Psychology A
Holmes Virginia M Laurie Stowe and Linda Cupples Lexical exp ectations in parsing
complement verb sentences Journal of Memory and Language
Joshi Aravind K Tree adjoining grammars Howmuchcontext sensitivity is required to
provide reasonable structural description In David Dowty Lauri Karttunen and Arnold
Zwicky Eds Natural Language Processing Psycholinguistic Computational and Theoret
ical Perspectives New York Cambridge University Press
Joshi Aravind K Pro cessing crossed and nested dep endencies an automaton p ersp ective
on the psycholinguistic results Language and Cognitive Processses
Joshi Aravind K TAGs in categorial clothing Presented at rd Meeting on Mathematics
vemb er of Language MOL No
Joshi Aravind K and Yves Schab es Tree adjoining grammars and lexicalized grammars
In Maurice Nivat and Andreas Po delski Eds De nability and Recognizability of Sets of
Trees Elsevier
Juliano Cornell and Michael K Tanenhaus Contingent frequency e ects in syntactic
ambiguity resolution In Proceedings of the th Annual ConferenceoftheCognitive Science
Society Lawrence Erlbaum Asso ciates June
Karttunen Lauri Radical lexicalism In Mark Baltin and Anthony S Kro ch Eds
Alternative Conceptions of Phrase Structure Chicago University of Chicago Press
Kennedy Alan Wayne S Murray Francis Jennings and Claire Reed Parsing com
plements Comments on the principle of minimal attachment Language and Cognitive
Processses SI
Kimball John Seven principles of surface structure parsing in natural language Cogni
tion
K onig Esther Parsing as natural deduction In Proceedings of the th Annual Me eting
of the Association for Computational Linguistics June
Kro ch Anthony S Re exes of grammar patterns of language change Language Variation
and Change
Ladd D Rob ert Comp ound proso dic domains Technical rep ort University of Edin
burgh February
Lamb ek J The mathematics of sentence structure American Mathematical Monthly
Le Chenadec Philipp e Canonical Forms in Finitely PresentedAlgebras Wiley
MacDonland Maryellen Adam Just and Patricia Carp enter Working memory con
straints on the pro cessing of syntactic ambiguity Cognitive Psychology
Marcus Mitchell A Theory of Syntactic Recognition for Natural Language Cambridge
Mass MIT Press
Marcus Mitchell Donald Hindle and Margaret Fleck D theory Talking ab out talking
ab out trees In Proceedings of the st Annual Meeting of the Association for Computational
Linguistics Cambridge Mass
Marlsen Wilson William and Lorraine K Tyler The temp oral structure of sp oken
language understanding Cognition
Marlsen Wilson William and Lorraine K Tyler Against mo dularity In Jay Gar eld
Ed Modularity in Know ledge Representation and Natural Language Understanding MIT
Press
bridge Mass MIT Press May Rob ert Logical Form Its Structure and Derivation Cam
McClelland Jay L Mark St John and Roman Taraban Sentence comprehension A
parallel distributed pro cessing approach Language and Cognitive Processses SI
Mitchell Don C Lexical guidance in human parsing Lo cus and pro cessing characteristics
In Max Coltheart Ed Attention and Peformance XII The Psychology of Reading
Hillsdale NJ Lawrence Erlbaum Asso ciates
Mitchell Don C Martin M B Corley and Alan Garnham E ects of context in hu
man sentence parsing Evidence against a discourse based prop osal mechanism Journal of
Experimental Psychology Learning Memory and Cognition
Mo ore Rob ert C Uni cation based semantci interpretation In Proceedings of the th
Annual Meeting of the Association for Computational Linguistics June
Mo ortgat Michael Categorial Investigations Dordrecht Foris
Morrill Glyn V Extraction and Coordination in Phrase StructureGrammar and Catego
rial Grammar PhD thesis University of Edinburgh
Newman M H A On theories with a combinatorial de nition of equivalence In
Annals of Mathematics chapter Princeton University Press
Nicol Janet L and Martin J Pickering Pro cessing syntactically ambiguous sentences
esearch Evidence from semantic priming Journal of Psycholinguistic R
Pareschi Remo and Mark J Steedman Alazywaytochart parse with combinatory
grammars In Proceedings of the th Annual Meeting of the Association for Computational
Linguistics
Park Jong C A uni cation based semantic interpretation for co ordinate constructs In
Proceedings of the th Annual Meeting of the Association for Computational Linguistics
Pearlmutter Neal J and Maryellen C MacDonald Plausibility and syntactic ambiguity
resolution In Proceedings of the th Annual ConferenceoftheCognitive ScienceSociety
Lawrence Erlbaum Asso ciates
Pereira Fernando C N A new characterization of attachment preferences In David
Dowty Lurie Kartunnen and Arnold Zwicky Eds Natural Language Parsing Psycholog
ical Computational And theoretical Perspectives Cambridge University Press
Pereira Fernando C N and Stuart M Shieb er Prolog and Natural Language Analysis
Vol Stanford CSLI
Prince Ellen F Toward a taxonomyofgiven new information In P Cole Ed Radical
Pragmatics New York Academic Press
anguage Pritchett Bradley L Garden Path Phenomena and the the Grammatical Basis of L
Processing PhD thesis Harvard University
Pritchett Bradley L Garden path phenomena and the the grammatical basis of language
pro cessing Language
Pritchett Bradley L Grammatical Competence and Parsing Performance Chicago
University of Chicago Press
Quine Willard V O Variables explained away In SelectedLogic Papers New York
Random House
Rambow Owen a A linguistic and computational analysis of the German Third Con
struction In Proceedings of the th Annual Meeting of the Association for Computational
Linguistics
Rambow Owen b Natural language syntax and formal systems Dissertation Prop osal
UniversityofPennsylvania
Rambow Owen and Aravind Joshi A pro cessing mo del for free word order languages
presented at the CUNY Sentence Pro cessing Conference
Randolph Quirk Sidney Greenbaum Geo rey Leech and Jan Svartik A Comprehensive
Grammar of the English Language London Longman
Rayner Keith Marcia Carlson and Lyn Frazier The interaction of syntax and semantics
during sentence pro cessing Eyemo vement in the analysis of semantically biased sentences
Journal of Verbal Learning and Verbal Behavior
Rayner Keith and Lyn Frazier Parsing temp orarily ambiguous complements Quarterly
Journal of Experimental Psychology A
Resnik Philip R Selection and Information PhD thesis UniversityofPennsylvania
Sag Ivan Gerald Gazdar Thomas Wasow and Steven Weisler Co ordination and how
to distinguish categories Natural Language and Linguistic Theory
Schub ert Lenhart Are there preference trade o s in attachment decisions In Proceedings
of the Fifth National ConferenceonArti cial Intel ligence
Sedivy Julie and Michael J Spivey Knowlton The e ect of NP de niteness on pars
ing attachmentambiguity InProceedings of the rdAnnual Meeting of the North East
Linguistics Society
Shastri Lokendra and Venkat Ajjanagadde From simple asso ciates to systematic reason
ing A connectionist representation of rules variables and dynamic bindings using temp oral
synchrony Behavioral and Brain Sciences
Shieb er Stuart M Sentence disambiguation by a shift reduce parsing technique In
Proceedings of the st Annual Meeting of the Association for Computational Linguistics
Shieb er Stuart M and Mark Johnson Variations on incremental interpretation Journal
of Psycholinguistic Research to app ear
hronous tree adjoining grammars In Proceed Shieb er Stuart M and Yves Schab es Sync
ings of the th International Conference on Computational Linguistics
Spivey Knowlton Michael J John C Trueswell and Michael K Tanehaus Context ef
fects in syntactic ambiguity resolution Discourse and semantic in uences in parsing reduced
relative clauses Canadian Journal of Psychology
Steedman Mark Dep endency and co ordination in the grammar of dutch and english
Language
Steedman Mark J Combinatory grammars and parasitic gaps Natural Language and
Linguistic Theory
Steedman Mark J Gapping as constituent co ordination Linguistics and Philosophy
Steedman Mark J Structure and Intonation Language
Steedman Mark J Surface structure Technical Rep ort MS CIS Universityof
Pennsylvania
Steedman Mark J Grammars and pro cessors In Hans Kamp and Christian Rohrer
Eds Aspects of Computational Linguistics Springer Verlag to app ear
Stowe Laurie A Thematic structures and sentence comprehension In Greg N Carl
son and Michael K Tanehaus Eds Linguistic StructureinLanguage Processing Kluwer
Academic Press
Swinney David A Marilyn Ford Uli Frauenfelder and Joan Bresnan On the temp oral
course of Gap Filling and antecedent assignment during sen tence comprehension In Barbara
Grosz Ronald Kaplan M MacKen and Ivan Sag Eds Language structure and processing
Stanford CSLI
Szab olsci Anna ECP in categorial grammar Max Planck Institute Nijmegen
Trueswell John C and Michael K Tanenhaus Tense temp oral context and syntactic
ambiguity resolution Language and Cognitive Processses
Trueswell John C and Michael K Tanenhaus Consulting temp oral context during sen
tence comprehension Evidence from the monitoring of eyemovements in reading In Pro
ceedings of the th Annual Conference of the Cognitive ScienceSociety
Trueswell John C Michael K Tanenhaus and Susan M Garnsey Semantic in uences
on parsing Use of thematic role information in syntactic ambiguity resolution manuscript
UniversityofRochester
Trueswell John C Michael K Tanenhaus and Christopher Kello Verb sp eci c con
straints in sentence pro cessing Separating e ects of lexical preferences from garden paths
Journal of Experimental Psychology Learning Memory and Cognition in press
Wall Rob ert and Kent Wittenburg Predictive normal forms for function comp osition in
categorial grammars In Proceedings of the International Workshop on Parsing Technolo
gies
Weinb erg Amy A parsing theory for the nineties Minimal commitment manuscript
ersity of Maryland Univ
Weischedel Ralph Damaris Ayuso Rob ert Bobrow Sean Boisen Rob ert Ingria and Je Pal
mucci Partial parsing A rep ort on work in progress In Proceedings of the DARPA
Speech and Natural Language Workshop
Whittemore Greg Kathleen Ferrara and Hans Brunner Empirical study of predictive
powers of simple attachmentschemes for p ost mo di er prep ositional phrases In Proceedings
of the th Annual Meeting of the Association for Computational Linguistics
Wilks Yorick Rightattachment and preference semantics In Proceedings of the rd
Annual Meeting of the Association for Computational Linguistics
Wittenburg Kent Natural Language Parsing with Combinatory Categorial Grammar in
aGraph Uni cation BasedFormalism PhD thesis UniversityofTexas
Wright B and Merrill Garrett Lexical decision in sentences E ects of syntactic struc
ture Memory and Cognition