Universal Algorithmic Intelligence A Mathematical Top!Down Approach Marcus Hutter Canberra, ACT, 0200, Australia http://www.hutter1.net/ ANU RSISE NICTA Marcus Hutter - 2 - Universal Arti¯cial Intelligence Abstract The dream of creating arti¯cial devices that reach or outperform human intelligence is many centuries old. Nowadays most research is more modest, focussing on solving narrower, speci¯c problems, associated with only some aspects of intelligence, like playing chess or natural language translation, either as a goal in itself or as a bottom-up approach. The dual top down approach investigates theories of general-purpose intelligent agents: the power of such theoretical agents, how to scale them down, and the involved key concepts. Necessary ingredients seem to be Occam's razor; Turing machines; Kolmogorov complexity; probability theory; Solomono® induction; Bayesian sequence prediction; minimum description length principle; agent framework; sequential decision theory; adaptive control theory; reinforcement learning; Levin search and extensions, which are all important subjects in their own right. From a mathematical point of view these concepts also seem to be su±cient. Marcus Hutter - 3 - Universal Arti¯cial Intelligence Contents ² What is (Arti¯cial) Intelligence. ² Philosophical Foundations. (Ockham, Epicurus, Induction) ² Mathematical Foundations. (Information, Complexity, Bayesian & Algorithmic Probability, Solomono® induction, Sequential Decision) ² Framework: Rational Agents. (in Known and Unknown Environments) ² AIXI: Universal Arti¯cial Intelligence. ² Approximations & Applications. (Universal Search, AIXItl, AI», MC-AIXI-CTW, ©MDP) ² Comparison to Other Approaches. ² Discussion. (Summary, Criticism, Questions, Next Steps, Literature) Marcus Hutter - 4 - Universal Arti¯cial Intelligence What is (Arti¯cial) Intelligence? Intelligence can have many faces ) formal de¯nition di±cult ² reasoning What is AI? Thinking Acting ² creativity ² association humanly Cognitive Turing test, ² generalization Science Behaviorism ² pattern recognition ² problem solving rationally Laws Doing the ² memorization Thought Right Thing ² planning ² achieving goals ² learning Collection of 70+ Defs of Intelligence ² optimization http://www.vetta.org/ ² self-preservation definitions-of-intelligence/ ² vision ² language processing ² classi¯cation Real world is nasty: partially unobservable, ² induction ² deduction uncertain, unknown, non-ergodic, reactive, ² ... vast, but luckily structured, ... Marcus Hutter - 5 - Universal Arti¯cial Intelligence Informal De¯nition of (Arti¯cial) Intelligence Intelligence measures an agent's ability to achieve goals in a wide range of environments. [S. Legg and M. Hutter] Emergent: Features such as the ability to learn and adapt, or to understand, are implicit in the above de¯nition as these capacities enable an agent to succeed in a wide range of environments. The science of Arti¯cial Intelligence is concerned with the construction of intelligent systems/artifacts/agents and their analysis. What next? Substantiate all terms above: agent, ability, utility, goal, success, learn, adapt, environment, ... === ===== Never trust experimenta theory if it is not supported by an experiment theory Marcus Hutter - 6 - Universal Arti¯cial Intelligence Induction!Prediction!Decision!Action Having or acquiring or learning or inducing a model of the environment an agent interacts with allows the agent to make predictions and utilize them in its decision process of ¯nding a good next action. Induction infers general models from speci¯c observations/facts/data, usually exhibiting regularities or properties or relations in the latter. Example Induction: Find a model of the world economy. Prediction: Use the model for predicting the future stock market. Decision: Decide whether to invest assets in stocks or bonds. Action: Trading large quantities of stocks influences the market. Marcus Hutter - 7 - Universal Arti¯cial Intelligence Science ¼ Induction ¼ Occam's Razor ² Grue Emerald Paradox: Hypothesis 1: All emeralds are green. Hypothesis 2: All emeralds found till y2020 are green, thereafter all emeralds are blue. ² Which hypothesis is more plausible? H1! Justi¯cation? ² Occam's razor: take simplest hypothesis consistent with data. is the most important principle in machine learning and science. Marcus Hutter - 8 - Universal Arti¯cial Intelligence Information Theory & Kolmogorov Complexity ² Quanti¯cation/interpretation of Occam's razor: ² Shortest description of object is best explanation. ² Shortest program for a string on a Turing machine T leads to best extrapolation=prediction. KT (x) = minfl(p): T (p) = xg p ² Prediction is best for a universal Turing machine U. Kolmogorov-complexity(x) = K(x) = KU (x) · KT (x) + cT ² Recent impressive applications: universal similarity metric "[LV08]. [Clustering by Compression, Cilibrasi & Vitanyi 2005] Marcus Hutter - 9 - Universal Arti¯cial Intelligence Bayesian Probability Theory Given (1): Models P (DjHi) for probability of observing data D, when Hi is true. Given (2): Prior probability over hypotheses P (Hi). Goal: Posterior probability P (HijD) of Hi, after having seen data D. Solution: P (DjHi) ¢ P (Hi) Bayes' rule: P (HijD) = P i P (DjHi) ¢ P (Hi) (1) Models P (DjHi) usually easy to describe (objective probabilities) (2) But Bayesian prob. theory does not tell us how to choose the prior P (Hi) (subjective probabilities) Marcus Hutter - 10 - Universal Arti¯cial Intelligence Algorithmic Probability Theory ² Epicurus: If more than one theory is consistent with the observations, keep all theories. ² ) uniform prior over all Hi? ² Re¯nement with Occam's razor quanti¯ed in terms of Kolmogorov complexity: ¡KT =U (Hi) P (Hi) := 2 ² Fixing T we have a complete theory for prediction. Problem: How to choose T . ² Choosing U we have a universal theory for prediction. Observation: Particular choice of U does not matter much. Problem: Incomputable. Marcus Hutter - 11 - Universal Arti¯cial Intelligence Inductive Inference & Universal Forecasting ² Solomono® combined Occam, Epicurus, Bayes, and Turing into one formal theory of sequential prediction. ² M(x) = probability that a universal Turing machine outputs x when provided with fair coin flips on the input tape. ² A posteriori probability of y given x is M(yjx) = M(xy)=M(x). ² Given x1; ::; xt¡1, the probability of xt is M(xtjx1:::xt¡1). ² Immediate \applications": - Weather forecasting: xt 2 fsun,raing. - Stock-market prediction: xt 2 fbear,bullg. - Continuing number sequences in an IQ test: xt 2 IN. ² Optimal universal inductive reasoning system! Marcus Hutter - 12 - Universal Arti¯cial Intelligence Sequential Decision Theory Setup: For t = 1; 2; 3; 4; ::: Given sequence x1; x2; :::; xt¡1 (1) predict/make decision yt, (2) observe xt, (3) su®er loss Loss(xt; yt), (4) t ! t + 1, goto (1) Goal: Minimize expected Loss. Greedy minimization of expected loss is optimal if: Important: Decision yt does not influence env. (future observations). Loss function is known. Problem: Expectation w.r.t. what? Solution: W.r.t. universal distribution M if true distr. is unknown. Marcus Hutter - 13 - Universal Arti¯cial Intelligence Example: Weather Forecasting Observation xt 2 X = fsunny, rainyg Decision yt 2 Y = fumbrella, sunglassesg Loss sunny rainy umbrella 0.1 0.3 sunglasses 0.0 1.0 Taking umbrella/sunglasses does not influence future weather (ignoring butterfly e®ect) Marcus Hutter - 14 - Universal Arti¯cial Intelligence Agent Model with Reward if actions/decisions a influence the environment q r1 j o1 r2 j o2 r3 j o3 r4 j o4 r5 j o5 r6 j o6 ... © HYH ©© H ©¼© H H Agent Environ- work tape ... work tape ... p ment q P ³1 PP ³³ PP ³³ PPq³³ a1 a2 a3 a4 a5 a6 ... Marcus Hutter - 15 - Universal Arti¯cial Intelligence Rational Agents in Known Environment ² Setup: Known deterministic or probabilistic environment ² Greedy maximization of reward r (=¡Loss) no longer optimal. Example: Chess ² Exploration versus exploitation problem. ) Agent has to be farsighted. ² Optimal solution: Maximize future (expected) reward sum, called value. ² Problem: Things drastically change if environment is unknown Marcus Hutter - 16 - Universal Arti¯cial Intelligence Rational Agents in Unknown Environment Additional problem: (probabilistic) environment unknown. Fields: reinforcement learning and adaptive control theory Bayesian approach: Mixture distribution. 1. What performance does Bayes-optimal policy imply? It does not necessarily imply self-optimization (Heaven&Hell example). 2. Computationally very hard problem. 3. Choice of horizon? Immortal agents are lazy. Universal Solomono® mixture ) universal agent AIXI. Represents a formal (math., non-comp.) solution to the AI problem? Most (all?) problems are easily phrased within AIXI. Marcus Hutter - 17 - Universal Arti¯cial Intelligence The AIXI Model in one Line complete & essentially unique & limit-computable X X X ¡`(q) AIXI: ak := arg max ::: max [rk + ::: + rm] 2 ak am okrk omrm q : U(q;a1::am)=o1r1::omrm action, reward, observation, Universal TM, qrogram, k=now AIXI is an elegant mathematical theory of AI Claim: AIXI is the most intelligent environmental independent, i.e. universally optimal, agent possible. Proof: For formalizations, quanti¯cations, and proofs, see [Hut05]. Potential Applications: Agents, Games, Optimization, Active Learning, Adaptive Control, Robots. Marcus Hutter - 18 - Universal Arti¯cial Intelligence Computational Issues: Universal Search ² Levin search: Fastest algorithm for inversion and optimization