The Natural Language Understanding Problem A literature review of recent work using an embodied, usage-based, cognitive linguistic approach MSc ProjectMSc Painting by John William Waterhouse, Pandora, 1896 Steven Bos, June 2012 Milestone 1 for MSc Project TU Delft Supervisor: Dr. Ir. Pascal Wiggers Painting by John William Waterhouse, Pandora, 1896 The Natural Language Understanding Problem 2012 Contents About this research assignment .......................................................................................................... 5 1 Introduction ................................................................................................................................ 6 2 Embodiment and the Neural Theory of Language ........................................................................ 8 3 A new philosophical framework for NLU ................................................................................... 10 3.1 The current definitions of meaning and understanding ...................................................... 10 3.2 The relation between meaning and understanding ............................................................ 11 3.3 The Embodied definitions of meaning and understanding ................................................. 13 3.3.1 Defining Meaning....................................................................................................... 13 3.3.2 Defining Understanding ............................................................................................. 15 3.4 Epistemology: Study of knowledge .................................................................................... 17 3.4.1 Defining Experience ................................................................................................... 17 3.4.2 Defining Knowledge ................................................................................................... 20 3.4.3 Defining Logic ............................................................................................................ 22 3.4.4 Defining Truth ............................................................................................................ 24 3.4.5 Defining Belief ........................................................................................................... 25 4 The human NLU system: Early developments ............................................................................ 28 4.1 The genesis of cognition .................................................................................................... 28 4.2 Early cognitive development .............................................................................................. 29 4.3 Development of language skills.......................................................................................... 30 4.3.1 Passive language learning .......................................................................................... 30 4.3.2 Active language learning ............................................................................................ 31 5 The human NLU system: Components in the brain .................................................................... 33 5.1 Anatomy of the brain ......................................................................................................... 34 5.1.1 The brain at large ....................................................................................................... 34 5.1.2 Cerebral cortex .......................................................................................................... 35 5.1.3 Neurons ..................................................................................................................... 35 5.1.4 Training neurons ....................................................................................................... 36 L i t e r a t u r e r e v i e w b y S t e v e n B o s ([email protected]) Pagina 2 The Natural Language Understanding Problem 2012 5.1.5 The central nervous system and information flow ...................................................... 37 5.2 Functional structures in the brain ...................................................................................... 39 5.2.1 Broca’s area ............................................................................................................... 40 5.2.2 Wernicke’s area ......................................................................................................... 41 5.3 Functionality of the brain: Memory ................................................................................... 43 5.3.1 Sensorial memory ...................................................................................................... 44 5.3.2 Short term memory ................................................................................................... 45 5.3.3 Long term memory .................................................................................................... 46 5.3.4 Working memory ....................................................................................................... 46 6 The human NLU system: Theories on enabling NLU ................................................................... 49 6.1 Embodied theory of language structure ............................................................................. 50 6.2 Embodied theory of language aquistion ............................................................................. 53 6.2.1 Language learning: a usage-based theory ................................................................... 54 6.2.2 Grammar learning ...................................................................................................... 55 6.2.3 Infant teaching methods of language ......................................................................... 57 6.3 Embodied theory of language use ...................................................................................... 58 6.3.1 Modelling motivation and personality ........................................................................ 58 6.3.2 Modelling human inference ....................................................................................... 61 7 The artificial NLU system ........................................................................................................... 64 7.1 Developing (Embodied) AI systems .................................................................................... 65 7.1.1 A brief history of problems with developing AI systems ............................................. 65 7.1.2 Principles of engineering Embodied AI systems .......................................................... 67 7.1.3 Key directions in engineering Embodied AI systems ................................................... 68 7.2 Developing (Embodied) NLU systems ................................................................................. 70 7.2.1 General NLU Architecture .......................................................................................... 70 7.2.2 General Components ................................................................................................. 71 7.2.3 Problems of traditional NLU systems .......................................................................... 72 7.2.4 Computational approaches to Language Aquisition .................................................... 73 7.2.5 Computational approaches to Language Structure ..................................................... 74 L i t e r a t u r e r e v i e w b y S t e v e n B o s ([email protected]) Pagina 3 The Natural Language Understanding Problem 2012 7.2.6 Computational approaches to Language Usage .......................................................... 75 7.3 Benchmarking and validating artificial NLU systems ........................................................... 78 7.4 Existing NLU Systems ......................................................................................................... 79 7.4.1 ITALK ......................................................................................................................... 80 7.4.2 Watson ...................................................................................................................... 81 7.4.3 XPERO ........................................................................................................................ 83 7.4.4 Protothinker .............................................................................................................. 84 7.4.5 NTL ............................................................................................................................ 85 8 Conclusion and final remarks .................................................................................................... 88 References ........................................................................................................................................ 90 Appendix 1: Normal language development checklist .................................................................. 100 Appendix 2: The three general learning theories ......................................................................... 101 Appendix 3: Conversations with the Loebner 2011 chatbot ......................................................... 103 Appendix 4: Encoding of typical communication episode ............................................................ 105 Appendix 5: Principles of Embodiment .......................................................................................