DOCSLIB.ORG

De-Randomization Under a Uniform Assumption

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Randomness vs. Time: De-randomization under a uniform assumption Russell Impagliazzo∗ Avi Wigdersony Department of Computer Science Institute of Computer Science University of California Hebrew University San Diego, CA 91097-0114 Jerusalem, Israel 91904 [email protected] [email protected] Abstract 1 Introduction, History, and Intuition We prove that if BPP = EXP, then every prob- lem in BPP can be solved6 deterministically in 1.1 Motivation subexponential time on almost every input ( The introduction of randomization into efficient on every samplable ensemble for infinitely many computation has been one of the most fertile and input sizes). This is the first derandomiza- useful ideas in computer science. In cryptogra- tion result for BP P based on uniform, non- phy and asynchronous computing, randomization cryptographic hardness assumptions. It implies makes possible tasks that are impossible to per- the following gap in the average-instance com- form deterministically. Even for function com- plexities of problems in BP P : either these com- putation, many examples are known in which plexities are always sub-exponential or they con- randomization allows considerable savings in re- tain arbitrarily large exponential functions. sources like space and time over deterministic al- We use a construction of a small \pseudo- gorithms, or even \only" simplifies them. random" set of strings from a \hard function" But to what extent is this seeming power of in EXP which is identical to that used in the randomness over determinism real? The most analogous non-uniform results of [21, 3]. How- famous concrete version of this question regards ever, previous proofs of correctness assume the the power of BP P , the class of problems solv- \hard function" is not in P=poly. They give a able by probabilistic polynomial time algorithms non-constructive argument that a circuit distin- making small constant error. What is the rela- guishing the pseudo-random strings from truly tive power of such algorithms compared to de- random strings implies that a similarly-sized cir- terministic ones? This is largely open. On the cuit exists computing the \hard function". Our one hand, it is possible that P = BP P , i.e., ran- main technical contribution is to show that, if the domness is useless for solving new problems in \hard function" has certain properties, then this polynomial time. On the other, we might have argument can be made constructive. We then BP P = EXP , which would say that random- show that, assuming EXP P=poly, there are ness would be a nearly omnipotent tool for al- EXP -complete functions with⊆ these properties. gorithm design. A priori, neither extreme seems likely: there are some problems where random- ness seems exponentially helpful, but many hard problems are not susceptible to randomized solu- ∗Research supported by NSF Award CCR-92-570979, Sloan Research Fellowship BR-3311, grant #93025 of the tions. joint US-Czechoslovak Science and Technology Program, In this paper, we show that the intuition that and USA-Israel BSF Grant 92-00043 randomness is a resource basically incomparable yThis research was supported by grant number 69/96 of the Israel Science Foundation, founded by the Israel to time is wrong. Either there is a non-trivial de- Academy for Sciences and Humanities terministic simulation of BP P , or BP P = EXP ! Either time can non-trivially substitute for ran- \cryptographically secure" (BMY-type) pseudo- domness, or randomness can non-trivially substi- random generator based on one-way functions tute for time. In other words, either universal [6, 24, 16, 7, 8, 11]; the NW-generator based on de-randomization is possible, or randomization a Boolean function with no circuit that approxi- is a panacea for intractability. (There are some mates it [21, 3]; and the hitting set method [1, 2]. technical provisos: the deterministic simulation To state our results, we will need some notation only works for infinitely many input lengths, and for complexity classes. Let Size(T (n)) be the may fail on a negligible fraction of inputs even class of functions computable by circuit families of these lengths.) We consider the former much where the number of gates in the circuit with more plausible than the latter. n inputs is at most T (n). For C a complexity class and t(n) a function, let C=t(n) be the class 1.2 History: Hardness vs. Ran- of functions computable in C with t(n) bits of \advice" depending only on the input size, i.e., domness f C=t(n) g C and a function h : Z 2 () 9 2 ! While counter to most people's first intuition, Z with h(n) t(n) and f(x) = g(x; h( x )). A j j ≤ j j c our result should be less surprising to those who result of [15] shows that P=poly = c 1Size(n ). [ ≥ are aware of the literature on de-randomization. For C a complexity class, let i:o: C be the class − The fundamental paradigm in de-randomization of functions that agree with a function in C for is to trade \hardness" for \randomness". This all inputs of length n for infinitely many n. was first elucidated in the remarkable sequence of The first set of papers construct a pseudo- papers [22, 6, 24]. Roughly speaking, \computa- random generator from a one-way function. tionally hard" functions can be used to construct The pseudo-random generator quickly converts “efficient pseudo-random generators". These in a small random string to a polynomially larger turn lower the randomness requirements of any string that seems random in the following sense: efficient probabilistic algorithm, allowing for a Any adversary that can distinguish an output of \nontrivial" deterministic simulation. this generator from a truly random string of the In many such results, there is a quantitative same length can be used to invert the function. trade-off between the hardness assumption and A BP P algorithm that had a markedly different the time to perform the deterministic simula- behaviour on a pseudo-random input than a ran- tion. The stronger the assumption, the faster dom one would be such an adversary. So if no the simulation. Here, we are concentrating on such invertor exists, the deterministic algorithm the \low end" of the curve in this trade-off: what that enumerates the multi-set of outputs of the is the weakest assumption one can make and still generator and simulates the BPP algorithm on have some version of universal derandomization? each, taking the majority answer, would always Our results also have some implications for the be correct. Informally, this is stated as: \higher end" of the curve, but these are much less clean, and we will not fully describe them in Theorem A 1 [6, 24, 8, 7, 11] this abstract. If there are one-way functions that cannot be in- verted with a non-negligible probability in P=poly, We will thus compare our results mainly to the then BP P SUBEXP \low end" version of the known results. In partic- ⊂ ular, we will use as our standard for \nontrivial" The NW-generator [20, 21, 3] considerably nδ the class SUBEXP = δ>0DT IME(2 ). The weakened the hardness assumption needed in the statement BP P SUBEXP\ (read \random- ⊂ nonuniform setting. It achieves the same deter- ness is weak"), while falling short of P = BP P , ministic simulation of BP P , from any function would be a great result to prove uncondition- in EXP P=poly. ally, and it certainly implies BP P = EXP . − 6 There have been a sequence of papers getting Theorem A 2 [20, 21, 3] weaker and weaker hardness assumptions suffi- If EXP P=poly, then BP P i:o: SUBEXP cient to prove such a result. These papers use 6⊂ ⊂ − one of three basic methods for converting hard This was the best result known at the \low functions into pseudo-random sequences: the end" of the hardness vs. randomness curve. 2 There has also been a sequence of papers [21, then M(n; R) is distributed according to µn. As 1, 2, 14] at the \high end" of the curve, where usual, we can extend this notion to allow M ac- the desired goal is to obtain P = BP P under cess to an oracle, in which case we say µ is poly- the weakest possible assumption. The strongest nomially sampleable given the oracle. result in this sequence is stated below: Let T and be functions of n. HeurT IME (T (n)) is the class of pairs (f; µ) Theorem A 3 [14] (n) of functions f : 0; 1 ∗ 0; 1 and probability If EXP i:o: SIZE(2o(n)), then BP P = P . f g ! f g 6⊂ − ensembles µ so that there is an algorithm A(x) running in deterministic time T ( x ) so that n, n j j 8 1.3 Average-Case Derandomiza- for x µ 0; 1 , P rob[A(x) = f(x)] < (n). 2 n f g 6 tion Under Uniform Assump- AvgT IME(n)(T (n)) is the class of pairs (f; µ) tions of functions f : 0; 1 ∗ 0; 1 and probabil- ity ensembles µ fso thatg ! there f isg an algorithm Both the high-end and low-end results above A(x) running in deterministic time T ( x ) so that require non-uniform hardness assumptions, i.e., j j n A(x) f(x); ? and for all n, for x µn 0; 1 , that the hard problems in question are hard for P rob[2A( fx) = f(gx)] < (n). 2 f g circuits. The reason for this, intuitively, is that If membership6 in one of the above classes holds if a function is hard uniformly but easy non- for f together with any polynomially sampleable uniformly, there is some advice, or trap-door, ensemble, then we omit mention of the ensem- that makes computing the function easy.
Recommended publications
  • 3 Autocorrelation

    3 Autocorrelation

    3 Autocorrelation Autocorrelation refers to the correlation of a time series with its own past and future values. Autocorrelation is also sometimes called “lagged correlation” or “serial correlation”, which refers to the correlation between members of a series of numbers arranged in time. Positive autocorrelation might be considered a specific form of “persistence”, a tendency for a system to remain in the same state from one observation to the next. For example, the likelihood of tomorrow being rainy is greater if today is rainy than if today is dry. Geophysical time series are frequently autocorrelated because of inertia or carryover processes in the physical system. For example, the slowly evolving and moving low pressure systems in the atmosphere might impart persistence to daily rainfall. Or the slow drainage of groundwater reserves might impart correlation to successive annual flows of a river. Or stored photosynthates might impart correlation to successive annual values of tree-ring indices. Autocorrelation complicates the application of statistical tests by reducing the effective sample size. Autocorrelation can also complicate the identification of significant covariance or correlation between time series (e.g., precipitation with a tree-ring series). Autocorrelation implies that a time series is predictable, probabilistically, as future values are correlated with current and past values. Three tools for assessing the autocorrelation of a time series are (1) the time series plot, (2) the lagged scatterplot, and (3) the autocorrelation function. 3.1 Time series plot Positively autocorrelated series are sometimes referred to as persistent because positive departures from the mean tend to be followed by positive depatures from the mean, and negative departures from the mean tend to be followed by negative departures (Figure 3.1).
  • The Bayesian Approach to Statistics

    The Bayesian Approach to Statistics

    THE BAYESIAN APPROACH TO STATISTICS ANTHONY O’HAGAN INTRODUCTION the true nature of scientific reasoning. The fi- nal section addresses various features of modern By far the most widely taught and used statisti- Bayesian methods that provide some explanation for the rapid increase in their adoption since the cal methods in practice are those of the frequen- 1980s. tist school. The ideas of frequentist inference, as set out in Chapter 5 of this book, rest on the frequency definition of probability (Chapter 2), BAYESIAN INFERENCE and were developed in the first half of the 20th century. This chapter concerns a radically differ- We first present the basic procedures of Bayesian ent approach to statistics, the Bayesian approach, inference. which depends instead on the subjective defini- tion of probability (Chapter 3). In some respects, Bayesian methods are older than frequentist ones, Bayes’s Theorem and the Nature of Learning having been the basis of very early statistical rea- Bayesian inference is a process of learning soning as far back as the 18th century. Bayesian from data. To give substance to this statement, statistics as it is now understood, however, dates we need to identify who is doing the learning and back to the 1950s, with subsequent development what they are learning about. in the second half of the 20th century. Over that time, the Bayesian approach has steadily gained Terms and Notation ground, and is now recognized as a legitimate al- ternative to the frequentist approach. The person doing the learning is an individual This chapter is organized into three sections.
  • Analysis of Variance and Analysis of Variance and Design of Experiments of Experiments-I

    Analysis of Variance and Analysis of Variance and Design of Experiments of Experiments-I

    Analysis of Variance and Design of Experimentseriments--II MODULE ––IVIV LECTURE - 19 EXPERIMENTAL DESIGNS AND THEIR ANALYSIS Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur 2 Design of experiment means how to design an experiment in the sense that how the observations or measurements should be obtained to answer a qqyuery inavalid, efficient and economical way. The desigggning of experiment and the analysis of obtained data are inseparable. If the experiment is designed properly keeping in mind the question, then the data generated is valid and proper analysis of data provides the valid statistical inferences. If the experiment is not well designed, the validity of the statistical inferences is questionable and may be invalid. It is important to understand first the basic terminologies used in the experimental design. Experimental unit For conducting an experiment, the experimental material is divided into smaller parts and each part is referred to as experimental unit. The experimental unit is randomly assigned to a treatment. The phrase “randomly assigned” is very important in this definition. Experiment A way of getting an answer to a question which the experimenter wants to know. Treatment Different objects or procedures which are to be compared in an experiment are called treatments. Sampling unit The object that is measured in an experiment is called the sampling unit. This may be different from the experimental unit. 3 Factor A factor is a variable defining a categorization. A factor can be fixed or random in nature. • A factor is termed as fixed factor if all the levels of interest are included in the experiment.
  • A Review of Graph and Network Complexity from an Algorithmic Information Perspective

    A Review of Graph and Network Complexity from an Algorithmic Information Perspective

    entropy Review A Review of Graph and Network Complexity from an Algorithmic Information Perspective Hector Zenil 1,2,3,4,5,*, Narsis A. Kiani 1,2,3,4 and Jesper Tegnér 2,3,4,5 1 Algorithmic Dynamics Lab, Centre for Molecular Medicine, Karolinska Institute, 171 77 Stockholm, Sweden; [email protected] 2 Unit of Computational Medicine, Department of Medicine, Karolinska Institute, 171 77 Stockholm, Sweden; [email protected] 3 Science for Life Laboratory (SciLifeLab), 171 77 Stockholm, Sweden 4 Algorithmic Nature Group, Laboratoire de Recherche Scientifique (LABORES) for the Natural and Digital Sciences, 75005 Paris, France 5 Biological and Environmental Sciences and Engineering Division (BESE), King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia * Correspondence: [email protected] or [email protected] Received: 21 June 2018; Accepted: 20 July 2018; Published: 25 July 2018 Abstract: Information-theoretic-based measures have been useful in quantifying network complexity. Here we briefly survey and contrast (algorithmic) information-theoretic methods which have been used to characterize graphs and networks. We illustrate the strengths and limitations of Shannon’s entropy, lossless compressibility and algorithmic complexity when used to identify aspects and properties of complex networks. We review the fragility of computable measures on the one hand and the invariant properties of algorithmic measures on the other demonstrating how current approaches to algorithmic complexity are misguided and suffer of similar limitations than traditional statistical approaches such as Shannon entropy. Finally, we review some current definitions of algorithmic complexity which are used in analyzing labelled and unlabelled graphs. This analysis opens up several new opportunities to advance beyond traditional measures.
  • Introduction to Bayesian Inference and Modeling Edps 590BAY

    Introduction to Bayesian Inference and Modeling Edps 590BAY

    Introduction to Bayesian Inference and Modeling Edps 590BAY Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2019 Introduction What Why Probability Steps Example History Practice Overview ◮ What is Bayes theorem ◮ Why Bayesian analysis ◮ What is probability? ◮ Basic Steps ◮ An little example ◮ History (not all of the 705+ people that influenced development of Bayesian approach) ◮ In class work with probabilities Depending on the book that you select for this course, read either Gelman et al. Chapter 1 or Kruschke Chapters 1 & 2. C.J. Anderson (Illinois) Introduction Fall 2019 2.2/ 29 Introduction What Why Probability Steps Example History Practice Main References for Course Throughout the coures, I will take material from ◮ Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., & Rubin, D.B. (20114). Bayesian Data Analysis, 3rd Edition. Boco Raton, FL, CRC/Taylor & Francis.** ◮ Hoff, P.D., (2009). A First Course in Bayesian Statistical Methods. NY: Sringer.** ◮ McElreath, R.M. (2016). Statistical Rethinking: A Bayesian Course with Examples in R and Stan. Boco Raton, FL, CRC/Taylor & Francis. ◮ Kruschke, J.K. (2015). Doing Bayesian Data Analysis: A Tutorial with JAGS and Stan. NY: Academic Press.** ** There are e-versions these of from the UofI library. There is a verson of McElreath, but I couldn’t get if from UofI e-collection. C.J. Anderson (Illinois) Introduction Fall 2019 3.3/ 29 Introduction What Why Probability Steps Example History Practice Bayes Theorem A whole semester on this? p(y|θ)p(θ) p(θ|y)= p(y) where ◮ y is data, sample from some population.
  • Autocorrelation

    Autocorrelation

    Autocorrelation David Gerbing School of Business Administration Portland State University January 30, 2016 Autocorrelation The difference between an actual data value and the forecasted data value from a model is the residual for that forecasted value. Residual: ei = Yi − Y^i One of the assumptions of the least squares estimation procedure for the coefficients of a regression model is that the residuals are purely random. One consequence of randomness is that the residuals would not correlate with anything else, including with each other at different time points. A value above the mean, that is, a value with a positive residual, would contain no information as to whether the next value in time would have a positive residual, or negative residual, with a data value below the mean. For example, flipping a fair coin yields random flips, with half of the flips resulting in a Head and the other half a Tail. If a Head is scored as a 1 and a Tail as a 0, and the probability of both Heads and Tails is 0.5, then calculate the value of the population mean as: Population Mean: µ = (0:5)(1) + (0:5)(0) = :5 The forecast of the outcome of the next flip of a fair coin is the mean of the process, 0.5, which is stable over time. What are the corresponding residuals? A residual value is the difference of the corresponding data value minus the mean. With this scoring system, a Head generates a positive residual from the mean, µ, Head: ei = 1 − µ = 1 − 0:5 = 0:5 A Tail generates a negative residual from the mean, Tail: ei = 0 − µ = 0 − 0:5 = −0:5 The error terms of the coin flips are independent of each other, so if the current flip is a Head, or if the last 5 flips are Heads, the forecast for the next flip is still µ = :5.
  • Mild Vs. Wild Randomness: Focusing on Those Risks That Matter

    Mild Vs. Wild Randomness: Focusing on Those Risks That Matter

    with empirical sense1. Granted, it has been Mild vs. Wild Randomness: tinkered with using such methods as Focusing on those Risks that complementary “jumps”, stress testing, regime Matter switching or the elaborate methods known as GARCH, but while they represent a good effort, Benoit Mandelbrot & Nassim Nicholas Taleb they fail to remediate the bell curve’s irremediable flaws. Forthcoming in, The Known, the Unknown and the The problem is that measures of uncertainty Unknowable in Financial Institutions Frank using the bell curve simply disregard the possibility Diebold, Neil Doherty, and Richard Herring, of sharp jumps or discontinuities. Therefore they editors, Princeton: Princeton University Press. have no meaning or consequence. Using them is like focusing on the grass and missing out on the (gigantic) trees. Conventional studies of uncertainty, whether in statistics, economics, finance or social science, In fact, while the occasional and unpredictable have largely stayed close to the so-called “bell large deviations are rare, they cannot be dismissed curve”, a symmetrical graph that represents a as “outliers” because, cumulatively, their impact in probability distribution. Used to great effect to the long term is so dramatic. describe errors in astronomical measurement by The good news, especially for practitioners, is the 19th-century mathematician Carl Friedrich that the fractal model is both intuitively and Gauss, the bell curve, or Gaussian model, has computationally simpler than the Gaussian. It too since pervaded our business and scientific culture, has been around since the sixties, which makes us and terms like sigma, variance, standard deviation, wonder why it was not implemented before. correlation, R-square and Sharpe ratio are all The traditional Gaussian way of looking at the directly linked to it.
  • Random Variables and Applications

    Random Variables and Applications

    Random Variables and Applications OPRE 6301 Random Variables. As noted earlier, variability is omnipresent in the busi- ness world. To model variability probabilistically, we need the concept of a random variable. A random variable is a numerically valued variable which takes on different values with given probabilities. Examples: The return on an investment in a one-year period The price of an equity The number of customers entering a store The sales volume of a store on a particular day The turnover rate at your organization next year 1 Types of Random Variables. Discrete Random Variable: — one that takes on a countable number of possible values, e.g., total of roll of two dice: 2, 3, ..., 12 • number of desktops sold: 0, 1, ... • customer count: 0, 1, ... • Continuous Random Variable: — one that takes on an uncountable number of possible values, e.g., interest rate: 3.25%, 6.125%, ... • task completion time: a nonnegative value • price of a stock: a nonnegative value • Basic Concept: Integer or rational numbers are discrete, while real numbers are continuous. 2 Probability Distributions. “Randomness” of a random variable is described by a probability distribution. Informally, the probability distribution specifies the probability or likelihood for a random variable to assume a particular value. Formally, let X be a random variable and let x be a possible value of X. Then, we have two cases. Discrete: the probability mass function of X specifies P (x) P (X = x) for all possible values of x. ≡ Continuous: the probability density function of X is a function f(x) that is such that f(x) h P (x < · ≈ X x + h) for small positive h.
  • How to Randomize?

    How to Randomize?

    TRANSLATING RESEARCH INTO ACTION How to Randomize? Roland Rathelot J-PAL Course Overview 1. What is Evaluation? 2. Outcomes, Impact, and Indicators 3. Why Randomize and Common Critiques 4. How to Randomize 5. Sampling and Sample Size 6. Threats and Analysis 7. Project from Start to Finish 8. Cost-Effectiveness Analysis and Scaling Up Lecture Overview • Unit and method of randomization • Real-world constraints • Revisiting unit and method • Variations on simple treatment-control Lecture Overview • Unit and method of randomization • Real-world constraints • Revisiting unit and method • Variations on simple treatment-control Unit of Randomization: Options 1. Randomizing at the individual level 2. Randomizing at the group level “Cluster Randomized Trial” • Which level to randomize? Unit of Randomization: Considerations • What unit does the program target for treatment? • What is the unit of analysis? Unit of Randomization: Individual? Unit of Randomization: Individual? Unit of Randomization: Clusters? Unit of Randomization: Class? Unit of Randomization: Class? Unit of Randomization: School? Unit of Randomization: School? How to Choose the Level • Nature of the Treatment – How is the intervention administered? – What is the catchment area of each “unit of intervention” – How wide is the potential impact? • Aggregation level of available data • Power requirements • Generally, best to randomize at the level at which the treatment is administered. Suppose an intervention targets health outcomes of children through info on hand-washing. What is the appropriate level of randomization? A. Child level 30% B. Household level 23% C. Classroom level D. School level 17% E. Village level 13% 10% F. Don’t know 7% A. B. C. D.
  • Randomized Experimentsexperiments Randomized Trials

    Randomized Experimentsexperiments Randomized Trials

    Impact Evaluation RandomizedRandomized ExperimentsExperiments Randomized Trials How do researchers learn about counterfactual states of the world in practice? In many fields, and especially in medical research, evidence about counterfactuals is generated by randomized trials. In principle, randomized trials ensure that outcomes in the control group really do capture the counterfactual for a treatment group. 2 Randomization To answer causal questions, statisticians recommend a formal two-stage statistical model. In the first stage, a random sample of participants is selected from a defined population. In the second stage, this sample of participants is randomly assigned to treatment and comparison (control) conditions. 3 Population Randomization Sample Randomization Treatment Group Control Group 4 External & Internal Validity The purpose of the first-stage is to ensure that the results in the sample will represent the results in the population within a defined level of sampling error (external validity). The purpose of the second-stage is to ensure that the observed effect on the dependent variable is due to some aspect of the treatment rather than other confounding factors (internal validity). 5 Population Non-target group Target group Randomization Treatment group Comparison group 6 Two-Stage Randomized Trials In large samples, two-stage randomized trials ensure that: [Y1 | D =1]= [Y1 | D = 0] and [Y0 | D =1]= [Y0 | D = 0] • Thus, the estimator ˆ ˆ ˆ δ = [Y1 | D =1]-[Y0 | D = 0] • Consistently estimates ATE 7 One-Stage Randomized Trials Instead, if randomization takes place on a selected subpopulation –e.g., list of volunteers-, it only ensures: [Y0 | D =1] = [Y0 | D = 0] • And hence, the estimator ˆ ˆ ˆ δ = [Y1 | D =1]-[Y0 | D = 0] • Only estimates TOT Consistently 8 Randomized Trials Furthermore, even in idealized randomized designs, 1.
  • Algorithmic Information Theory and Novelty Generation

    Algorithmic Information Theory and Novelty Generation

    Computational Creativity 2007 Algorithmic Information Theory and Novelty Generation Simon McGregor Centre for Research in Cognitive Science University of Sussex, UK [email protected] Abstract 100,000 digits in the binary expansion of π can be gener- ated by a program far shorter than 100,000 bits. A string This paper discusses some of the possible contributions of consisting of the binary digit 1 repeated 1,000 times can be algorithmic information theory, and in particular the cen- generated by a program shorter than 1,000 bits. However, tral notion of data compression, to a theoretical exposition it can be shown (Li and Vitanyi, 1997) that most strings of computational creativity and novelty generation. I note are incompressible, i.e. they cannot be generated by a pro- that the formalised concepts of pattern and randomness gram shorter than themselves. Consequently, if you flip due to algorithmic information theory are relevant to com- a perfectly random coin 100,000 times, the likelihood is puter creativity, briefly discuss the role of compression that the sequence of heads and tails you obtain cannot be in machine learning theory and present a general model described by a program shorter than 100,000 bits. In algo- for generative algorithms which turns out to be instanti- rithmic information theory a string is described as random ated by decompression in a lossy compression scheme. I if and only if it is incompressible. also investigate the concept of novelty using information- Note that randomness in algorithmic information the- theoretic tools and show that a purely “impersonal” formal ory applies to strings, and not to the physical processes notion of novelty is inadequate; novelty must be defined which generate those strings.
  • Random Numbers and the Central Limit Theorem

    Random Numbers and the Central Limit Theorem

    Random numbers and the central limit theorem Alberto García Javier Junquera Bibliography No se puede mostrar la imagen en este momento. Cambridge University Press, Cambridge, 2002 ISBN 0 521 65314 2 Physical processes with probabilistic character Certain physical processes do have a probabilistic character Desintegration of atomic nuclei: Brownian movement of a particle in a liquid: The dynamic (based on Quantum We do not know in detail the Mechanics) is strictly probabilistic dynamical variables of all the particles involved in the problem We need to base our knowledge in new laws that do not rely on dynamical variables with determined values, but with probabilistic distributions Starting from the probabilistic distribution, it is possible to obtain well defined averages of physical magnitudes, especially if we deal with very large number of particles The stochastic oracle Computers are (or at least should be) totally predictible Given some data and a program to operate with them, the results could be exactly reproduced But imagine, that we couple our code with a special module that generates randomness program randomness real :: x $<your compiler> -o randomness.x randomness.f90 do $./randomness call random_number(x) print "(f10.6)" , x enddo end program randomness The output of the subroutine randomness is a real number, uniformly distributed in the interval "Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.” (John von Neumann) Probability distribution The subroutine “random number”