Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12
Download Book (5,071 KB) Download Chapter (273 KB) Algorithmic Learning Theory Lecture Notes in Computer Science Volume 8139, 2013, pp 158-172
Citations 368 Downloads 200 Citations 9 Comments Abstract We define an optimal Bayesian knowledge-seeking agent, KL-KSA, designed for countable hypothesis classes of stochastic environments and whose goal is to gather as much information about the unknown world as possible. Although this agent works for arbitrary countable classes and priors, we focus on the especially interesting case where all stochastic computable environments are considered and the prior is based on Solomonoff’s universal prior. Among other properties, we show that KL-KSA learns the true environment in the sense that it learns to predict the consequences of actions it does not take. We show that it does not consider noise to be information and avoids taking actions leading to inescapable traps. We also present a variety of toy experiments demonstrating that KL-KSA behaves according to expectation.
1 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12
Related Content
References (13)
1. Baranes, A., Oudeyer, P.-Y.: Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots. Robotics and Autonomous Systems 61(1), 69–73 (2013) CrossRef
2. Hutter, M.: Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability. Springer (2005)
3. Lattimore, T., Hutter, M.: Asymptotically optimal agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 368–382. Springer, Heidelberg (2011) CrossRef
4. Lattimore, T., Hutter, M.: Time Consistent Discounting. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 383–397. Springer, Heidelberg (2011) CrossRef
5. Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications, 3rd edn. Springer, New York (2008) CrossRef
6. Orseau, L.: Universal Knowledge-Seeking Agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 353–367. Springer, Heidelberg (2011) CrossRef
7. Orseau, L.: Asymptotic non-learnability of universal agents with computable horizon functions. Theoretical Computer Science 473, 149–156 (2013) CrossRef
8. Rathmanner, S., Hutter, M.: A philosophical treatise of universal induction. Entropy 13(6), 1076–1136 (2011) CrossRef
9. Sutton, R., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
10. Schmidhuber, J.: Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science 18(2), 173–188 (2006) CrossRef
11. Sun, Y., Gomez, F., Schmidhuber, J.: Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS, vol. 6830, pp. 41–51. Springer, Heidelberg (2011) CrossRef
12. Storck, J., Hochreiter, S., Schmidhuber, J.: Reinforcement driven information acquisition in non-deterministic environments. In: Proceedings of
2 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12
the International Conference on Artificial Neural Networks, Paris, vol. 2, pp. 159–164. EC2 & Cie (1995)
13. Solomonoff, R.: Complexity-based induction systems: comparisons and convergence theorems. IEEE Transactions on Information Theory 24(4), 422–432 (1978) CrossRef
About this Chapter
Title Universal Knowledge-Seeking Agents for Stochastic Environments Book Title Algorithmic Learning Theory Book Subtitle 24th International Conference, ALT 2013, Singapore, October 6-9, 2013. Proceedings Pages pp 158-172 Copyright 2013 DOI 10.1007/978-3-642-40935-6_12 Print ISBN 978-3-642-40934-9 Online ISBN 978-3-642-40935-6 Series Title Lecture Notes in Computer Science Series Volume 8139 Series ISSN 0302-9743 Publisher Springer Berlin Heidelberg Copyright Holder Springer-Verlag Berlin Heidelberg Additional Links
About this Book
3 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12
Topics
Artificial Intelligence (incl. Robotics)
Mathematical Logic and Formal Languages
Algorithm Analysis and Problem Complexity
Computation by Abstract Devices
Logics and Meanings of Programs
Pattern Recognition Keywords
Universal artificial intelligence
exploration
reinforcement learning
algorithmic information theory
Solomonoff induction Industry Sectors
Electronics
Telecommunications
IT & Software eBook Packages
eBook Package english Computer Science
eBook Package english full Collection
Editors
4 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12
Sanjay Jain (19)
Rémi Munos (20)
Frank Stephan (19)
Thomas Zeugmann (21) Editor Affiliations
19. National University of Singapore
20. Inria Lille - Nord Europe, Villeneuve d’Ascq
21. Hokkaido University Authors
Laurent Orseau (22) (23)
Tor Lattimore (24)
Marcus Hutter (24) Author Affiliations
22. UMR 518 MIA, AgroParisTech, F-75005, Paris, France
23. UMR 518 MIA, INRA, F-75005, Paris, France
24. RSCS, Australian National University, Canberra, ACT, 0200, Australia
Continue reading... To view the rest of this content please follow the download PDF link above.
8,348,055 scientific documents at your fingertips © Springer, Part of Springer Science+Business Media
5 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12
6 of 6 29/11/2013 2:04 PM