Computational Measures of Information Gain and Reinforcement in Inference Processes

Computational Measures of Information Gain and Reinforcement in Inference Processes

Universitat de València Doctoral Dissertation / Tesis Doctoral Computational Measures of Information Gain and Reinforcement in Inference Processes Medidas Computacionales de Ganancia de Información y Refuerzo en Procesos de Inferencia José Hernández Orallo Supervisor/ Director : Prof. Dr. Rafael Beneyto Torres Catedrático de Lógica y Filosofía de la Ciencia Universitat de València A thesis submitted to the Universitat de València in accordance with the requirements of the degree of Doctor of Philosophy in the Department of Logic and Philosophy of Science. September 1999 0. II III IV Abstract and Keywords This work is devoted to the formal study of inductive and deductive concept synthesis usefulness and aftermath in terms of information gain and reinforcement inside inference systems. The set of measures which are introduced allow a detailed and unified analysis of the value of the output of any inference process with respect to the input and the context (background knowledge or axiomatic system). Although the main measures, computational information gain, reinforcement and intensionality, are defined independently, they (alone or combined) make it possible to formalise or better comprehend several notions which have been traditionally treated in a rather ambiguous way: novelty, explicitness/implicitness, informativeness, surprise, interestingness, plausibility, confirmation, comprehensibility, ‘consilience’, utility and unquestionability. Most of the measures are applied to different kinds of theories and systems, from the appraisal of predictiveness, the representational optimality and the axiomatic power of logical theories, software systems and databases, to the justified evaluation of the intellectual abilities of cognitive agents and human beings. Keywords : Inference Processes, Evaluation Measures, Induction, Deduction, Information, Kolmogorov Complexity, Reasoning, Inference Paradox, Information Gain, Inference Confirmation, Reinforcement, Intensionality, Measurement of Cognitive Abilities, Evaluation of Logical Theories, Knowledge-Based Systems, Machine Learning, Inductive Logic Programming, Intensionality. V Resumen y Palabras Clave Esta tesis se centra en el estudio formal de la utilidad y resultados de la síntesis de conceptos inductivos y deductivos en términos de ganancia de información y refuerzo en sistemas de inferencia. El conjunto de medidas que se introducen permiten un análisis detallado y unificado del valor del resultado de cualquier proceso de inferencia con respecto a la entrada y el contexto (conocimiento previo o sistema axiomático). Aunque las medidas más importantes, ganancia computacional de información, refuerzo e intensionalidad, se definen de manera independiente, permiten (solas o combinadas) formalizar o comprender mejor varias nociones que han sido tratadas tradicionalmente de una manera bastante ambigua: novedad, la diferencia entre explícito e implícito, informatividad, sorpresa, interés, plausibilidad, confirmación, comprensibilidad, ‘consiliencia’, utilidad e incuestionabilidad. La mayoría de las medidas se aplican a diferentes tipos de teorías y sistemas, desde la estimación de la capacidad de predicción, la optimalidad de representación, o el poder axiomático de teorías lógicas, sistemas software y bases de datos, hasta la evaluación justificada de las habilidades intelectuales de agentes cognitivos y seres humanos. Palabras Clave : Procesos de Inferencia, Medidas de Evaluación, Inducción, Deducción, Información, Complejidad Kolmogorov, Razonamiento, Paradoja de la Inferencia, Ganancia de Información, Confirmación de la Inferencia, Refuerzo, Medición de Capacidades Cognitivas, Sistemas Basados en el Conocimiento, Aprendizaje Computacional, Programación Lógica Inductiva, Intensionalidad . VI Contents Abstract and Keywords .................................................................................................... V Contents .......................................................................................................................... VII Extended Abstract ....................................................................................................... XIII Resumen Extendido ...................................................................................................... XV Authorship .................................................................................................................. XVII Acknowledgements...................................................................................................... XIX 1. INTRODUCTION ............................................................................... 1 1.1 Introduction .......................................................................................................... 2 1.2 Motivation and Precedents ................................................................................... 4 1.3 Aims .................................................................................................................... 10 1.4 Overview and Organisation ................................................................................ 12 1.5 Terminology and Notation ................................................................................. 17 2. ON INFERENCE PROCESSES AND THEIR RELATIONSHIP .............. 19 2.1 Introduction ........................................................................................................ 20 2.2 Deduction ........................................................................................................... 24 2.2.1 Automated Deduction and Logic Programming .......................................................................... 26 2.2.2 Resource-Bounded and Non-omniscient Deduction .................................................................. 29 2.3 Induction ............................................................................................................ 30 2.3.1 The MDL principle and other Selection Criteria .......................................................................... 32 2.3.2 Grammatical Inference and Induction of Functional Programs ............................................... 35 2.3.3 Inductive Logic Programming (ILP) .............................................................................................. 36 2.4 Abduction ........................................................................................................... 39 2.5 Reasoning by Analogy ........................................................................................ 41 2.5.1 Case-Based Reasoning ....................................................................................................................... 41 2.6 On the Relation between Inference Processes ................................................... 42 2.6.1 Inference Processes, Effort and Lazy/Eager Methods ............................................................... 45 2.6.2 Inference Processes and Confirmation .......................................................................................... 47 2.6.3 Towards a Combination of Inference Processes .......................................................................... 50 VII 3. INFORMATION AND REPRESENTATION GAINS ............................. 53 3.1 Introduction ........................................................................................................ 54 3.2 Resource Consumption and Gain ...................................................................... 57 3.3 Relative Information Value ................................................................................ 59 3.3.1 Properties ............................................................................................................................................. 60 3.4 Time-Ignoring Information Gain ...................................................................... 62 3.5 Computational Information Gain ...................................................................... 64 3.5.1 Fundamental Properties .................................................................................................................... 65 3.5.2 Unique Interface Formulation ......................................................................................................... 66 3.5.3 Other Properties ................................................................................................................................. 67 3.6 Information Gain and Complexity ..................................................................... 68 3.7 True Information Gain ....................................................................................... 70 3.8 Representation Gain............................................................................................ 71 3.8.1 Universal Simplification .................................................................................................................... 72 3.8.2 Representational Optimality ............................................................................................................. 74 3.9 Comparison with Related Information Measures.............................................. 76 3.9.1 Kirsh’s Theory of Explicitness ......................................................................................................... 77 3.9.2 Nake’s Theory of Aesthetics and Schmidhuber’s Interestingness ............................................. 78 3.10 Summary and Contributions of This Chapter .................................................. 79 4. INFORMATION

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    367 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us