Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12

Download Book (5,071 KB) Download Chapter (273 KB) Algorithmic Learning Theory Lecture Notes in Computer Science Volume 8139, 2013, pp 158-172

Citations 368 Downloads 200 Citations 9 Comments Abstract We define an optimal Bayesian knowledge-seeking agent, KL-KSA, designed for countable hypothesis classes of stochastic environments and whose goal is to gather as much information about the unknown world as possible. Although this agent works for arbitrary countable classes and priors, we focus on the especially interesting case where all stochastic computable environments are considered and the prior is based on Solomonoff’s universal prior. Among other properties, we show that KL-KSA learns the true environment in the sense that it learns to predict the consequences of actions it does not take. We show that it does not consider noise to be information and avoids taking actions leading to inescapable traps. We also present a variety of toy experiments demonstrating that KL-KSA behaves according to expectation.

1 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12

Related Content

References (13)

1. Baranes, A., Oudeyer, P.-Y.: Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots. Robotics and Autonomous Systems 61(1), 69–73 (2013) CrossRef

2. Hutter, M.: Universal Artificial : Sequential Decisions based on Algorithmic Probability. Springer (2005)

3. Lattimore, T., Hutter, M.: Asymptotically optimal agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 368–382. Springer, Heidelberg (2011) CrossRef

4. Lattimore, T., Hutter, M.: Time Consistent Discounting. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 383–397. Springer, Heidelberg (2011) CrossRef

5. Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications, 3rd edn. Springer, New York (2008) CrossRef

6. Orseau, L.: Universal Knowledge-Seeking Agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 353–367. Springer, Heidelberg (2011) CrossRef

7. Orseau, L.: Asymptotic non-learnability of universal agents with computable horizon functions. Theoretical Computer Science 473, 149–156 (2013) CrossRef

8. Rathmanner, S., Hutter, M.: A philosophical treatise of universal induction. Entropy 13(6), 1076–1136 (2011) CrossRef

9. Sutton, R., Barto, A.G.: : An Introduction. MIT Press, Cambridge (1998)

10. Schmidhuber, J.: Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science 18(2), 173–188 (2006) CrossRef

11. Sun, Y., Gomez, F., Schmidhuber, J.: Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS, vol. 6830, pp. 41–51. Springer, Heidelberg (2011) CrossRef

12. Storck, J., Hochreiter, S., Schmidhuber, J.: Reinforcement driven information acquisition in non-deterministic environments. In: Proceedings of

2 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12

the International Conference on Artificial Neural Networks, Paris, vol. 2, pp. 159–164. EC2 & Cie (1995)

13. Solomonoff, R.: Complexity-based induction systems: comparisons and convergence theorems. IEEE Transactions on Information Theory 24(4), 422–432 (1978) CrossRef

About this Chapter

Title Universal Knowledge-Seeking Agents for Stochastic Environments Book Title Algorithmic Learning Theory Book Subtitle 24th International Conference, ALT 2013, Singapore, October 6-9, 2013. Proceedings Pages pp 158-172 Copyright 2013 DOI 10.1007/978-3-642-40935-6_12 Print ISBN 978-3-642-40934-9 Online ISBN 978-3-642-40935-6 Series Title Lecture Notes in Computer Science Series Volume 8139 Series ISSN 0302-9743 Publisher Springer Berlin Heidelberg Copyright Holder Springer-Verlag Berlin Heidelberg Additional Links

About this Book

3 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12

Topics

Artificial Intelligence (incl. Robotics)

Mathematical Logic and Formal Languages

Algorithm Analysis and Problem Complexity

Computation by Abstract Devices

Logics and Meanings of Programs

Pattern Recognition Keywords

Universal

exploration

reinforcement learning

algorithmic information theory

Solomonoff induction Industry Sectors

Electronics

Telecommunications

IT & Software eBook Packages

eBook Package english Computer Science

eBook Package english full Collection

Editors

4 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12

Sanjay Jain (19)

Rémi Munos (20)

Frank Stephan (19)

Thomas Zeugmann (21) Editor Affiliations

19. National University of Singapore

20. Inria Lille - Nord Europe, Villeneuve d’Ascq

21. Hokkaido University Authors

Laurent Orseau (22) (23)

Tor Lattimore (24)

Marcus Hutter (24) Author Affiliations

22. UMR 518 MIA, AgroParisTech, F-75005, Paris, France

23. UMR 518 MIA, INRA, F-75005, Paris, France

24. RSCS, Australian National University, Canberra, ACT, 0200, Australia

Continue reading... To view the rest of this content please follow the download PDF link above.

8,348,055 scientific documents at your fingertips © Springer, Part of Springer Science+Business Media

5 of 6 29/11/2013 2:04 PM Universal Knowledge-Seeking Agents for Stochastic Environments - Springer http://link.springer.com/chapter/10.1007/978-3-642-40935-6_12

6 of 6 29/11/2013 2:04 PM