Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12 Download Book (5,071 KB) Download Chapter (273 KB) Algorithmic Learning Theory Lecture Notes in Computer Science Volume 8139, 2013, pp 158-172 Citations 368 Downloads 200 Citations 9 Comments Abstract We define an optimal Bayesian knowledge-seeking agent, KL-KSA, designed for countable hypothesis classes of stochastic environments and whose goal is to gather as much information about the unknown world as possible. Although this agent works for arbitrary countable classes and priors, we focus on the especially interesting case where all stochastic computable environments are considered and the prior is based on Solomonoff’s universal prior. Among other properties, we show that KL-KSA learns the true environment in the sense that it learns to predict the consequences of actions it does not take. We show that it does not consider noise to be information and avoids taking actions leading to inescapable traps. We also present a variety of toy experiments demonstrating that KL-KSA behaves according to expectation. 1 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12 Related Content References (13) 1. Baranes, A., Oudeyer, P.-Y.: Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots. Robotics and Autonomous Systems 61(1), 69–73 (2013) CrossRef 2. Hutter, M.: Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability. Springer (2005) 3. Lattimore, T., Hutter, M.: Asymptotically optimal agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 368–382. Springer, Heidelberg (2011) CrossRef 4. Lattimore, T., Hutter, M.: Time Consistent Discounting. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 383–397. Springer, Heidelberg (2011) CrossRef 5. Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications, 3rd edn. Springer, New York (2008) CrossRef 6. Orseau, L.: Universal Knowledge-Seeking Agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 353–367. Springer, Heidelberg (2011) CrossRef 7. Orseau, L.: Asymptotic non-learnability of universal agents with computable horizon functions. Theoretical Computer Science 473, 149–156 (2013) CrossRef 8. Rathmanner, S., Hutter, M.: A philosophical treatise of universal induction. Entropy 13(6), 1076–1136 (2011) CrossRef 9. Sutton, R., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998) 10. Schmidhuber, J.: Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science 18(2), 173–188 (2006) CrossRef 11. Sun, Y., Gomez, F., Schmidhuber, J.: Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS, vol. 6830, pp. 41–51. Springer, Heidelberg (2011) CrossRef 12. Storck, J., Hochreiter, S., Schmidhuber, J.: Reinforcement driven information acquisition in non-deterministic environments. In: Proceedings of 2 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12 the International Conference on Artificial Neural Networks, Paris, vol. 2, pp. 159–164. EC2 & Cie (1995) 13. Solomonoff, R.: Complexity-based induction systems: comparisons and convergence theorems. IEEE Transactions on Information Theory 24(4), 422–432 (1978) CrossRef About this Chapter Title Universal Knowledge-Seeking Agents for Stochastic Environments Book Title Algorithmic Learning Theory Book Subtitle 24th International Conference, ALT 2013, Singapore, October 6-9, 2013. Proceedings Pages pp 158-172 Copyright 2013 DOI 10.1007/978-3-642-40935-6_12 Print ISBN 978-3-642-40934-9 Online ISBN 978-3-642-40935-6 Series Title Lecture Notes in Computer Science Series Volume 8139 Series ISSN 0302-9743 Publisher Springer Berlin Heidelberg Copyright Holder Springer-Verlag Berlin Heidelberg Additional Links About this Book 3 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12 Topics Artificial Intelligence (incl. Robotics) Mathematical Logic and Formal Languages Algorithm Analysis and Problem Complexity Computation by Abstract Devices Logics and Meanings of Programs Pattern Recognition Keywords Universal artificial intelligence exploration reinforcement learning algorithmic information theory Solomonoff induction Industry Sectors Electronics Telecommunications IT & Software eBook Packages eBook Package english Computer Science eBook Package english full Collection Editors 4 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12 Sanjay Jain (19) Rémi Munos (20) Frank Stephan (19) Thomas Zeugmann (21) Editor Affiliations 19. National University of Singapore 20. Inria Lille - Nord Europe, Villeneuve d’Ascq 21. Hokkaido University Authors Laurent Orseau (22) (23) Tor Lattimore (24) Marcus Hutter (24) Author Affiliations 22. UMR 518 MIA, AgroParisTech, F-75005, Paris, France 23. UMR 518 MIA, INRA, F-75005, Paris, France 24. RSCS, Australian National University, Canberra, ACT, 0200, Australia Continue reading... To view the rest of this content please follow the download PDF link above. 8,348,055 scientific documents at your fingertips © Springer, Part of Springer Science+Business Media 5 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12 6 of 6 29/11/2013 2:04 PM.