XNL-Soar, Incremental Parsing, and the Minimalist Program Deryle Lonsdale, LaReina Hingson, Jamison Cooper-Leavitt, Warren Casbeer, Rebecca Madsen BYU Department of Linguistics Syntax in the Minimalist Program (MP) Minimalist Principles (Chomsky 1995) Abstract Merge: Build structure based on uninterpretable categorial features First: head+complement The object: a new incremental language modeling parser Second: spec + head/X-bar Operator trace (partial) Syntactic representation based on the Minimalist Program (Chomsky 1995) Move: Generate grammaticality in the surface representation 1: Updates the prior GB (Principles & Parameters) representation Head-head movement 2:-3: 1: lexaccess(their) The framework: Soar Phrasal movement 5: 2: lexaccess(investigation) Symbolic, rule-based, goal-directed cognitive modeling approach (Newell 1990) Hierarchy of Projections (Adger 2003) 3: project (NP) Machine learning architecture (Laird et al. 1987) Inherent knowledge for specifying structural projections 4: hop (n) Models on-line human language processing. 6: Nominal: D > (Poss) > n > N 5: merge1(n’) The goal: explore MP correlations with prior psycholinguistic findings in human language processing (Lewis 1993) Clausal: C > T > (Neg) > (Perf) > (Prog) > (Pass) > v > V 6: merge2(nP) The approach: process incoming lexical items incrementally via operators Features play a central role 7: movehead (N) Lexical access operator for each incoming word Strong (movement) and weak (merge) 8: hop(D) Build MP syntactic structure via projection, merge, and movement operators Feature checking 9: merge1(D’) Constrain operators using subcategorization and thematic information NP, VP symmetry including shells 10: merge2(DP) Allow strategies for limited, local structural reanalysis for unproblematic 11: lexaccess(should) ambiguities 12: lexaccess(have) Related issues: 13: lexaccess(exonerated) 7: Semantic interpretation incrementally mapped from syntactic structures Incremental parsing 14: lexaccess(the) Machine learning: chunking up prior decisions and reusing them later 15: lexaccess(defendant) Performance issues: memory usage across time, parsing difficulties Human language processing is incremental 16: project(NP) References: Processes are largely lexically driven 17: hop(n) Chomsky, N. 1995. The Minimalist Program. MIT Press. 18: merge1(n’) Laird, J., A. Newell, and P. Rosenbloom. 1987. Soar: An architecture for Word-by-word processing: words... 9:-10: general intelligence. Artificial Intelligence 33:1-64. 19: merge2(nP) enter system’s perceptual buffer Lewis, R. 1993. An architecturally-based theory of human sentence 20: movehead(N) attended to via lexical access operator comprehension. PhD thesis, Carnegie Mellon, School of Computer Science. 21: hop(D) Newell, A. 1990. Unified theories of cognition. Harvard University Press: disappear if unattended to after specified duration 11:-15: 22: merge1(D’) Cambridge, MA. Structure (syntactic and semantic) constructed piecemeal 23: merge2(DP) Pritchett, B. 1992. Grammatical Competence and Parsing Performance. 24: merge1(V’) University of Chicago Press. Chicago, IL. Open question: Is parsing within the MP incrementally feasible in a 25: merge2(VP) cognitively plausible way? 26: hop(v) 27: merge1(v’) 28: merge2(vP) 29: movehead(V) Background: Soar WordNet data 30: hop(Perf) 31: merge1(Perf’) Operators and operator types General theory of human problem solving 32: merge2(PerfP) Overview of verb exonerate Cognition: language, action, performance (in all their varieties) 16:-24: 33: hop(T) Operator: basic unit of cognitive processing The verb exonerate has 1 sense (first 1 from tagged 34: merge1(T’) Cognitive modeling architecture instantiated as intelligent agent texts) Stepwise progress toward specified goal 35: merge2(TP) Observable mechanisms, time course of behaviors, deliberation Various types and functions in XNL-Soar (1)acquit, assoil, clear, discharge, exonerate, Knowledge levels and their use exculpate -- (pronounce not guilty of criminal Lexical access: retrieve and store lexically-related information charges; "The suspect was cleared of the murder Instantiate the model as a computational system Merge: construct syntax via MP-specified merge operations charges") Symbolic rule-based architecture Movehead: perform head-to-head movement (via adjunction) Semantic class: verb.communication Subgoal-directed problem specification HoP: consult hierarchy of projections, return next possible target level Verb frames: Operator-based problem solving Project: create bare-structure maximal projection from lexical item Somebody ---s somebody. Machine learning Somebody ---s somebody of something. Applications: robotics, video games and simulation, tutorial dialogue, etc. NL-Soar: natural language processing engine built on Soar English LCS lexicon data Background: NL-Soar 10.6.a#1#_ag_th,mod- External knowledge sources poss(of)#exonerate#exonerate#exonerate#exonerate+ed# (2.0,00874318_exonerate%2:32:00::) Soar extension for modeling language use “10.6.a” “Verbs of Possessional Deprivation: Cheat Verbs/-of” WordNet 2.0 (wordnet.princeton.edu) WORDS (absolve acquit balk bereave bilk bleed burgle cheat Unified theory of cognition Lexical semantics: part-of-speech, word senses, subcategorization cleanse con cull cure defraud denude deplete depopulate + deprive despoil disabuse disarm disencumber dispossess divest Inflectional and derivational morphology drain ease exonerate fleece free gull milk mulct pardon Soar cognitive modeling system English LCS lexicon (www.umiacs.umd.edu/~bonnie/verbs-English.lcs) plunder purge purify ransack relieve render rid rifle rob sap 25:-29: + strip swindle unburden void wean) NL components Thematic information: θ-grids, θ-roles ((1 "_ag_th,mod-poss()") = Used to derive uninterpretable features (1 "_ag_th,mod-poss(from)") (1 "_ag_th,mod-poss(of)")) Unified cognitive architecture for overall cognition including NL Triggers syntactic construction Used specifically to model language tasks: acquisition, language use, "He !!+ed the people (of their rights); He !!+ed him of his Aligned with WordNet information sins" language/task integration, etc. Different modalities supported Parsing/comprehension: derive semantics of incoming utterances Generation/production: output utterances expressing semantic content Rules for XNL-Soar Mapping: convert between semantic representations Discourse/dialogue: learn and execute dialogue plans IF→THEN (productions) XNL-Soar updates the syntactic component of NL-Soar to use the MP If certain conditions are met, then the agent performs some action Represent a priori knowledge Several rule/production firings can be bundled together as operators Current XNL-Soar system: about 60 productions External knowledge sources: interfaced via 1000+ lines of Tcl/Perl 30:-34: NL-Soar system: 3500 productions Our goals Integrate the MP into a cognitive modeling engine Explore language/task-integrations using the MP Current status Test cross-linguistic implementation possibilities with the MP Ultimately, determine whether the MP supports incremental, operator- Proof of concept for fundamental syntactic structures based processing Basic transitive sentences work, ditransitives soon Unergatives and unaccusatives work Our approach All functional and lexical projections in syntactic structure Feature percolation, feature checking mostly works Map the syntactic parsing task onto an operator-based framework Some constraints derived from thematic information Specify goals, subgoals, etc. for parsing Develop operator types for various MP syntactic operations Implement constraints, preconditions, precedence hierarchies Future functionality Integrate necessary and relevant external knowledge sources Strengths: XP adjunction We have already done this for a prior syntactic model. More semantics/deeper semantics. The MP has an operator-like feel to it. Quantifier raising The Soar operator-based framework is versatile and flexible. Scopal relationships Weaknesses: C-command and other interpretive mechanisms The MP literature does not address incremental parsing in-depth. More detailed LCS structures The external knowledge sources are somewhat incommensurable. Web-based interactive Minimalist Parser grapher 35: Our knowledge of human performance data is sketchy. Issue: Find a balance between generation and parsing Most MP descriptions are generative, not recognitional in focus Is it advisable and well motivated to “undo” or “reverse” movements? If not, is generate-and-test the right mechanism for parsing input? What are the implications for learning and bootstrapping language capabilities (e.g. parsing in the service of generation)? Similar Work Incremental parsing in general (Phillips 2003) Future applications Other linguistic theories for incremental parsing GB (Kolb 1991) Integrate syntax/semantics into discourse/conversation component Dependency grammar (Milward 1994, Ait-Mokhtar et al. 2002) Develop human-agent and agent-agent communication Categorial Grammar (Izuo 2004) Parameterize XNL-Soar for processing of other languages besides English Finite-state methods (Ait-Mokhtar & Chanod 1997) Model cognition in reading Minimalist parsing in other frameworks (Stabler 1997, Harkema 2001) Model real-time language/task integrations Thematic information and parsing (Schlesewsky & Bornkessel 2004) Crosslinguistic considerations in incremental parsing (Schneider 2000) Note: all of the above have been implemented in NL-Soar, the predecessor Human studies on ambiguity, reanalysis Eye tracking (Kamide, Altmann, & Haywood 2003) ERP (Bornkessel, Schlesewsky, & Friederici 2003) For more information ... on Soar: http://sitemaker.umich.edu/soar on NL-Soar: http://linguistics.byu.edu/nlsoar.