DOCSLIB.ORG

Monte-Carlo Simulation Balancing

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Monte-Carlo Simulation Balancing Monte-Carlo Simulation Balancing David Silver [email protected] Department of Computing Science, University of Alberta, Edmonton, AB Gerald Tesauro [email protected] IBM Watson Research Center, 19 Skyline Drive, Hawthorne, NY Abstract in a search-tree by Monte-Carlo simulation. It has proven surprisingly successful in deterministic two- In this paper we introduce the first algo- player games, achieving master-level at 9 × 9 Go rithms for efficiently learning a simulation (Gelly & Silver, 2007; Coulom, 2007) and winning policy for Monte-Carlo search. Our main idea the General Game-Playing competition (Finnsson & is to optimise the balance of a simulation pol- Bj¨ornsson,2008). icy, so that an accurate spread of simulation outcomes is maintained, rather than optimis- In these algorithms, many games of self-play are sim- ing the direct strength of the simulation pol- ulated, using a simulation policy to select actions for icy. We develop two algorithms for balanc- both players. The overall performance of Monte-Carlo ing a simulation policy by gradient descent. search is largely determined by the simulation policy. The first algorithm optimises the balance of A simulation policy with appropriate domain knowl- complete simulations, using a policy gradient edge can dramatically outperform a uniform random algorithm; whereas the second algorithm op- simulation policy (Gelly et al., 2006). Automatically timises the balance over every two steps of improving the simulation policy is a major goal of simulation. We compare our algorithms to current research in this area (Gelly & Silver, 2007; reinforcement learning and supervised learn- Coulom, 2007; Chaslot et al., 2008). Two approaches ing algorithms for maximising the strength have previously been taken to improving the simula- of the simulation policy. We test each al- tion policy. gorithm in the domain of 5 × 5 and 6 × 6 The first approach is to directly construct a strong Computer Go, using a softmax policy that is simulation policy that performs well by itself, either parameterised by weights for a hundred sim- by hand (Billings et al., 1999), reinforcement learning ple patterns. When used in a simple Monte- (Tesauro & Galperin, 1996; Gelly & Silver, 2007), or Carlo search, the policies learnt by simulation supervised learning (Coulom, 2007). Unfortunately, balancing achieved significantly better per- a stronger simulation policy can actually lead to a formance, with half the mean squared error of weaker Monte-Carlo search (Gelly & Silver, 2007), a a uniform random policy, and similar overall paradox that we explore further in this paper. performance to a sophisticated Go engine. The second approach to learning a simulation policy is by trial and error, adjusting parameters and testing for 1. Introduction improvements in the performance of the Monte-Carlo player, either by hand (Gelly et al., 2006), or by hill- Monte-Carlo search algorithms use the average out- climbing (Chaslot et al., 2008). However, each param- come of many simulations to evaluate candidate ac- eter evaluation usually requires many complete games, tions. They have achieved human master level in thousands of positions, and millions of simulations to a variety of stochastic two-player games, including be executed. Furthermore, hill-climbing methods do Backgammon (Tesauro & Galperin, 1996), Scrabble not scale well with increasing dimensionality, and fare (Sheppard, 2002) and heads up Poker (Billings et al., poorly with complex policy parameterisations. 1999). Monte-Carlo tree search evaluates each state Handcrafting an effective simulation policy is partic- Appearing in Proceedings of the 26 th International Confer- This work was supported in part under the DARPA ence on Machine Learning, Montreal, Canada, 2009. Copy- GALE project, contract No. HR0011-08-C-0110. right 2009 by the author(s)/owner(s). Monte-Carlo Simulation Balancing ularly problematic in Go. Many of the top Go pro- In real-world domains, knowledge of the true min- grams utilise a small number of simple patterns and imax values is not available. In practice, we use rules, based largely on the default policy used in MoGo the values V^ ∗(s) computed by deep Monte-Carlo tree (Gelly et al., 2006). Adding further Go knowledge searches, which converge on the minimax value in the without breaking MoGo's \magic formula" has proven limit (Kocsis & Szepesvari, 2006), as an approximation to be surprisingly difficult. V^ ∗(s) ≈ V ∗(s). In this paper we introduce a new paradigm for learning At every time-step t, each player's move incurs some ∗ ∗ a simulation policy. We define an objective function, error δt = V (st+1) − V (st) with respect to the min- ∗ which we call imbalance, that explicitly measures the imax value V (st). We will describe a policy with a performance of a simulation policy for Monte-Carlo small error as strong, and a policy with a small ex- evaluation. We introduce two new algorithms that pected error as balanced. Intuitively, a strong policy minimise the imbalance of a simulation policy by gradi- makes few mistakes, whereas a balanced policy allows ent descent. These algorithms require very little com- many mistakes, as long as they cancel each other out putation for each parameter update, and are able to on average. Formally, we define the strength J(θ) and learn expressive simulation policies with hundreds of k-step imbalance Bk(θ) of a policy πθ, parameters. 2 J(θ) = Eρ Eπθ δt jst = s (2) We evaluate our simulation balancing algorithms in 2 23 0 2k−1 31 the game of Go. We compare them to reinforcement X B (θ) = 6 δ js = s 7 (3) learning and supervised learning algorithms for max- k Eρ 4@Eπθ 4 t+j t 5A 5 imising strength, and to a well-known simulation pol- j=0 icy for this domain, handcrafted by trial and error. h 2i = ( [V ∗(s ) − V ∗(s )js = s]) The simulation policy learnt by our new algorithms Eρ Eπθ t+k t t significantly outperforms prior approaches. We consider two choices of k in this paper. The two- step imbalance B2(θ) is specifically appropriate to two- 2. Strength and Balance player games. It allows errors by one player, as long as We consider deterministic two-player games of finite they are on average cancelled out by the other player's error on the next move. The full imbalance B allows length with a terminal outcome or score z 2 R. During 1 simulation, move a is selected in state s according to errors to be committed at any time, as long as they cancel out by the time the game is finished. It is ex- a stochastic simulation policy πθ(s; a) with parameter vector θ, that is used to select moves for both players. actly equivalent to the mean squared bias that we are The goal is to find the parameter vector θ∗ that max- aiming to optimise in Equation 1, imises the overall playing strength of a player based h ∗ ∗ 2i B1(θ) = Eρ (Eπθ [V (sT ) − V (s)jst = s]) on Monte-Carlo search. Our approach is to make the h i Monte-Carlo evaluations in the search as accurate as ∗ 2 = Eρ (Eπθ [zjst = s] − V (s)) (4) possible, by minimising the mean squared error be- 1 N P where sT is the terminal state with outcome z. Thus, tween the estimated values V (s) = N i=1 zi and the minimax values V ∗(s). while the direct performance of a policy is largely de- termined by its strength, the performance of a policy When the number of simulations N is large, the mean in Monte-Carlo simulation is determined by its full im- squared error is dominated by the bias of the sim- balance. ulation policy with respect to the minimax value, ∗ ∗ If the simulation policy is optimal, π [zjs] = V (s), V (s) − Eπθ [zjs], and the variance of the estimate (i.e. E θ the error caused by only seeing a finite number of sim- then perfect balance is achieved, B1(θ) = 0. This ulations) can be ignored. Our objective is to minimise suggests that optimising the strength of the simula- the mean squared bias, averaged over the distribution tion policy, so that individual moves become closer to of states ρ(s) that are evaluated during Monte-Carlo optimal, may be sufficient to achieve balance. How- search. ever, even small errors can rapidly accumulate over the course of long simulations if they are not well- ∗ h ∗ 2i θ = argmin Eρ (V (s) − Eπθ [zjs]) (1) balanced. It is more important to maintain a diverse θ spread of simulations, which are on average representa- where Eρ denotes the expectation over the distribution tive of strong play, than for individual moves or sim- of actual states ρ(s), and Eπθ denotes the expectation ulations to be low in error. Figure 1 shows a sim- over simulations with policy πθ. ple scenario in which the error of each player is i.i.d Monte-Carlo Simulation Balancing 10 10 Simulations Simulations Mean Mean Monte-Carlo value Monte-Carlo value 5 5 0 0 Minimax value Minimax value -5 -5 -10 -10 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Time steps Time steps Figure 1. Monte-Carlo simulation in an artificial two-player game. 30 simulations of 100 time steps were executed from an initial state with minimax value 0. Each player selects moves imperfectly during simulation, with an error that is exponentially distributed with respect to the minimax value, with rate parameters λ1 and λ2 respectively. a) The simulation players are strong but imbalanced: λ1 = 10; λ2 = 5, b) the simulation players are weak but balanced: λ1 = 2; λ2 = 2.
Load more

Recommended publications
  • A Politico-Social History of Algolt (With a Chronology in the Form of a Log Book)
    A Politico-Social History of Algolt (With a Chronology in the Form of a Log Book) R. w. BEMER Introduction This is an admittedly fragmentary chronicle of events in the develop­ ment of the algorithmic language ALGOL. Nevertheless, it seems perti­ nent, while we await the advent of a technical and conceptual history, to outline the matrix of forces which shaped that history in a political and social sense. Perhaps the author's role is only that of recorder of visible events, rather than the complex interplay of ideas which have made ALGOL the force it is in the computational world. It is true, as Professor Ershov stated in his review of a draft of the present work, that "the reading of this history, rich in curious details, nevertheless does not enable the beginner to understand why ALGOL, with a history that would seem more disappointing than triumphant, changed the face of current programming". I can only state that the time scale and my own lesser competence do not allow the tracing of conceptual development in requisite detail. Books are sure to follow in this area, particularly one by Knuth. A further defect in the present work is the relatively lesser availability of European input to the log, although I could claim better access than many in the U.S.A. This is regrettable in view of the relatively stronger support given to ALGOL in Europe. Perhaps this calmer acceptance had the effect of reducing the number of significant entries for a log such as this. Following a brief view of the pattern of events come the entries of the chronology, or log, numbered for reference in the text.
    [Show full text]
  • Security Applications of Formal Language Theory
    Dartmouth College Dartmouth Digital Commons Computer Science Technical Reports Computer Science 11-25-2011 Security Applications of Formal Language Theory Len Sassaman Dartmouth College Meredith L. Patterson Dartmouth College Sergey Bratus Dartmouth College Michael E. Locasto Dartmouth College Anna Shubina Dartmouth College Follow this and additional works at: https://digitalcommons.dartmouth.edu/cs_tr Part of the Computer Sciences Commons Dartmouth Digital Commons Citation Sassaman, Len; Patterson, Meredith L.; Bratus, Sergey; Locasto, Michael E.; and Shubina, Anna, "Security Applications of Formal Language Theory" (2011). Computer Science Technical Report TR2011-709. https://digitalcommons.dartmouth.edu/cs_tr/335 This Technical Report is brought to you for free and open access by the Computer Science at Dartmouth Digital Commons. It has been accepted for inclusion in Computer Science Technical Reports by an authorized administrator of Dartmouth Digital Commons. For more information, please contact [email protected]. Security Applications of Formal Language Theory Dartmouth Computer Science Technical Report TR2011-709 Len Sassaman, Meredith L. Patterson, Sergey Bratus, Michael E. Locasto, Anna Shubina November 25, 2011 Abstract We present an approach to improving the security of complex, composed systems based on formal language theory, and show how this approach leads to advances in input validation, security modeling, attack surface reduction, and ultimately, software design and programming methodology. We cite examples based on real-world security flaws in common protocols representing different classes of protocol complexity. We also introduce a formalization of an exploit development technique, the parse tree differential attack, made possible by our conception of the role of formal grammars in security. These insights make possible future advances in software auditing techniques applicable to static and dynamic binary analysis, fuzzing, and general reverse-engineering and exploit development.
    [Show full text]
  • Data General Extended Algol 60 Compiler
    DATA GENERAL EXTENDED ALGOL 60 COMPILER, Data General's Extended ALGOL is a powerful language tial I/O with optional formatting. These extensions comple­ which allows systems programmers to develop programs ment the basic structure of ALGOL and significantly in­ on mini computers that would otherwise require the use of crease the convenience of ALGOL programming without much larger, more expensive computers. No other mini making the language unwieldy. computer offers a language with the programming features and general applicability of Data General's Extended FEATURES OF DATA GENERAL'S EXTENDED ALGOL Character strings are implemented as an extended data ALGOL. type to allow easy manipulation of character data. The ALGOL 60 is the most widely used language for describ­ program may, for example, read in character strings, search ing programming algorithms. It has a flexible, generalized, for substrings, replace characters, and maintain character arithmetic organization and a modular, "building block" string tables efficiently. structure that provides clear, easily readable documentation. Multi-precision arithmetic allows up to 60 decimal digits The language is powerful and concise, allowing the systems of precision in integer or floating point calculations. programmer to state algorithms without resorting to "tricks" Device-independent I/O provides for reading and writ­ to bypass the language. ing in line mode, sequential mode, or random mode.' Free These characteristics of ALGOL are especially important form reading and writing is permitted for all data types, or in the development of working prototype systems. The output can be formatted according to a "picture" of the clear documentation makes it easy for the programmer to output line or lines.
    [Show full text]
  • Enhancing Reliability of RTL Controller-Datapath Circuits Via Invariant-Based Concurrent Test Yiorgos Makris, Ismet Bayraktaroglu, and Alex Orailoglu
    IEEE TRANSACTIONS ON RELIABILITY, VOL. 53, NO. 2, JUNE 2004 269 Enhancing Reliability of RTL Controller-Datapath Circuits via Invariant-Based Concurrent Test Yiorgos Makris, Ismet Bayraktaroglu, and Alex Orailoglu Abstract—We present a low-cost concurrent test methodology controller states for enhancing the reliability of RTL controller-datapath circuits, , , control signals based on the notion of path invariance. The fundamental obser- , environment signals vation supporting the proposed methodology is that the inherent transparency behavior of RTL components, typically utilized for hierarchical off-line test, renders rich sources of invariance I. INTRODUCTION within a circuit. Furthermore, additional sources of invariance are obtained by examining the algorithmic interaction between the HE ability to test the functionality of a circuit during usual controller, and the datapath of the circuit. A judicious selection T operation is becoming an increasingly desirable property & combination of modular transparency functions, based on the of modern designs. Identifying & discarding faulty results be- algorithm implemented by the controller-datapath pair, yields a powerful set of invariant paths in a design. Compliance to the in- fore they are further used constitutes a powerful design attribute variant behavior is checked whenever the latter is activated. Thus, of ASIC performing critical computations, such as DSP, and such paths enable a simple, yet very efficient concurrent test capa- ALU. While this capability can be provided by hardware dupli- bility, achieving fault security in excess of 90% while keeping the cation schemes, such methods incur considerable cost in terms hardware overhead below 40% on complicated, difficult-to-test, sequential benchmark circuits. By exploiting fine-grained design of area overhead, and possible performance degradation.
    [Show full text]
  • FPGA-Accelerated Evaluation and Verification of RTL Designs
    FPGA-Accelerated Evaluation and Verification of RTL Designs Donggyu Kim Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2019-57 http://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-57.html May 17, 2019 Copyright © 2019, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. FPGA-Accelerated Evaluation and Verification of RTL Designs by Donggyu Kim A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Krste Asanovi´c,Chair Adjunct Assistant Professor Jonathan Bachrach Professor Rhonda Righter Spring 2019 FPGA-Accelerated Evaluation and Verification of RTL Designs Copyright c 2019 by Donggyu Kim 1 Abstract FPGA-Accelerated Evaluation and Verification of RTL Designs by Donggyu Kim Doctor of Philosophy in Computer Science University of California, Berkeley Professor Krste Asanovi´c,Chair This thesis presents fast and accurate RTL simulation methodologies for performance, power, and energy evaluation as well as verification and debugging using FPGAs in the hardware/software co-design flow. Cycle-level microarchitectural software simulation is the bottleneck of the hard- ware/software co-design cycle due to its slow speed and the difficulty of simulator validation.
    [Show full text]
  • Behavioral Types in Programming Languages
    Foundations and Trends R in Programming Languages Vol. 3, No. 2-3 (2016) 95–230 c 2016 D. Ancona et al. DOI: 10.1561/2500000031 Behavioral Types in Programming Languages Davide Ancona, DIBRIS, Università di Genova, Italy Viviana Bono, Dipartimento di Informatica, Università di Torino, Italy Mario Bravetti, Università di Bologna, Italy / INRIA, France Joana Campos, LaSIGE, Faculdade de Ciências, Univ de Lisboa, Portugal Giuseppe Castagna, CNRS, IRIF, Univ Paris Diderot, Sorbonne Paris Cité, France Pierre-Malo Deniélou, Royal Holloway, University of London, UK Simon J. Gay, School of Computing Science, University of Glasgow, UK Nils Gesbert, Université Grenoble Alpes, France Elena Giachino, Università di Bologna, Italy / INRIA, France Raymond Hu, Department of Computing, Imperial College London, UK Einar Broch Johnsen, Institutt for informatikk, Universitetet i Oslo, Norway Francisco Martins, LaSIGE, Faculdade de Ciências, Univ de Lisboa, Portugal Viviana Mascardi, DIBRIS, Università di Genova, Italy Fabrizio Montesi, University of Southern Denmark Rumyana Neykova, Department of Computing, Imperial College London, UK Nicholas Ng, Department of Computing, Imperial College London, UK Luca Padovani, Dipartimento di Informatica, Università di Torino, Italy Vasco T. Vasconcelos, LaSIGE, Faculdade de Ciências, Univ de Lisboa, Portugal Nobuko Yoshida, Department of Computing, Imperial College London, UK Contents 1 Introduction 96 2 Object-Oriented Languages 105 2.1 Session Types in Core Object-Oriented Languages . 106 2.2 Behavioral Types in Java-like Languages . 121 2.3 Typestate . 134 2.4 Related Work . 139 3 Functional Languages 140 3.1 Effects for Session Type Checking . 141 3.2 Sessions and Explicit Continuations . 143 3.3 Monadic Approaches to Session Type Checking .
    [Show full text]
  • Algorithms, Computation and Mathewics (Algol -SE102?
    ' DOCUMENT RESUME 'ID 143 509 -SE102? 986 . AUTHOR ' Charp, Sylvia; And Cthers TITLE Algorithms, Computation and MatheWics (Algol SApplement). Teacher's Commentary. ReViged Edition. / IATITUTION. Stanford Univ.., Calif. School ,Mathematics Study Group. , SPONS, AGENCY National Science Foundation-, Washington, D.C. PUB DATE 66 'NOTE 113p.;,For related docuuents, see SE 022 983-988; Not available in hard copy dug to marginal legibility off original doctiment TIDRS PRICE MF-$0,.83 Plus Postage. HC Not Available frOm SDRS. 1?'ESCRI2TORS Algorithms; *Computers; *Mathematics Education; *Programing Languages; Secondary Education; *Secondary School Mathematics; *Teaching Guides IDENTIFIERS *itGeL; *,School Mathematics Study Group I. ABSTRACT.- A is the teacher's guide and commentary for the SIISG textboot,Algorithms, Computation andMathematics' (Algol Supplement) .This teacher's commentary provides~ background inf8rmation for'the teacher, suggestions for activities fOund in the student's lgoI 'Supplement, and answers to exercises and activities: The course is ,designed for high school students in grades 11 and 12. Acdess to a computer is high recommended. (RH) f ******************************4**********#5*****i******Lii*************- '* Documents acquired by ERIC ittlUde many informal unPgablished *, * materials n'ot available from other, sources. ERIC makes every effort pic' * to obtain,the best copy available. NeVerfheless, items of 'marginal * reprodUcibillty'are often encountered and this, affects the quality * * of the microfiche and hardcopy reproductions ERIp_makes.available_ * * via the-.ERIC-DocumentReproduction. Service IEDR4.' lint is note * * respon-siile for the quality of the original.document.'Reproductions * supplied by EURS are the` best that can be. made ftom the original. ********,*****************************A********************************* 414 yt r r 4) ALGORITHMS, COMPUTATION AN D MATH EMATICS (Algol Supplement) Teacher's Commentary Revised Edition 1 6 The following is a' list of, ail thKeji'vho participated in thepreparation of this.
    [Show full text]
  • A Logic Programming Language with Lambda-Abstraction, Function Variables, and Simple Unification
    A Logic Programming Language with Lambda-Abstraction, Function Variables, and Simple Unification Dale Miller Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104–6389 USA Abstract: It has been argued elsewhere that a logic programming language with function variables and ¸-abstractions within terms makes a good meta-programming language, especially when an object-language contains notions of bound variables and scope. The ¸Prolog logic programming language and the related Elf and Isabelle systems provide meta-programs with both function variables and ¸-abstractions by containing implementations of higher-order unification. This paper presents a logic programming language, called L¸, that also contains both function variables and ¸-abstractions, al- though certain restrictions are placed on occurrences of function variables. As a result of these restrictions, an implementation of L¸ does not need to implement full higher- order unification. Instead, an extension to first-order unification that respects bound variable names and scopes is all that is required. Such unification problems are shown to be decidable and to possess most general unifiers when unifiers exist. A unification algorithm and logic programming interpreter are described and proved correct. Several examples of using L¸ as a meta-programming language are presented. 1. Introduction A meta-programming language should be able to represent and manipulate such syntactic structures as programs, formulas, types, and proofs. A common characteristic of all these structures is that they involve notions of abstractions, scope, bound and free variables, substitution instances, and equality up to alphabetic change of bound variables. Although the data types available in most computer programming languages are, of course, rich enough to represent all these kinds of structures, such data types do not have direct support for these common characteristics.
    [Show full text]
  • The History of the ALGOL Effort
    Technische Universiteit Eindhoven Department of Mathematics and Computer Science The History of the ALGOL Effort by HT de Beer supervisors: C. Hemerik L.M.M. Royakkers Eindhoven, August 2006 Abstract This report studies the importancy of the ALGOL effort for computer science. Therefore the history of the ALGOL effort is told, starting with the compu- tational context of the 1950s when the ALGOL effort was initiated. Second, the development from IAL to ALGOL 60 and the role the BNF played in this development are discussed. Third, the translation of ALGOL 60 and the establishment of the scientific field of translator writing are treated. Finally, the period of ALGOL 60 maintenance and the subsequent period of creating a successor to ALGOL 60 are described. ii Preface This history on the ALGOL effort was written as a Master thesis in Com- puter Science and Engineering (Technische Informatica) at the Eindhoven University of Technology, the Netherlands. iii Contents Abstract ii Preface iii Contents iv 0 Introduction 1 0.0 On the sources used ........................ 2 0.1 My own perspective ........................ 3 0.2 The ALGOL effort: a definition .................. 4 0.3 Notes ................................ 4 1 Creation 5 1.0 The start of the ALGOL effort ................... 5 1.1 The need for a universal algorithmic language ........... 7 1.1.0 The American field of computing ............. 7 1.1.1 The European field of computing ............. 9 1.1.2 The difference between Europe and the USA and the need for universalism ...................... 11 1.2 Why was IAL chosen over other algorithmic languages? ..... 11 1.2.0 IT ............................
    [Show full text]
  • A Language-Independent Static Checking System for Coding Conventions
    A Language-Independent Static Checking System for Coding Conventions Sarah Mount A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy 2013 This work or any part thereof has not previously been presented in any form to the University or to any other body whether for the purposes of as- sessment, publication or for any other purpose (unless otherwise indicated). Save for any express acknowledgements, references and/or bibliographies cited in the work, I confirm that the intellectual content of the work is the result of my own efforts and of no other person. The right of Sarah Mount to be identified as author of this work is asserted in accordance with ss.77 and 78 of the Copyright, Designs and Patents Act 1988. At this date copyright is owned by the author. Signature: . Date: . Abstract Despite decades of research aiming to ameliorate the difficulties of creat- ing software, programming still remains an error-prone task. Much work in Computer Science deals with the problem of specification, or writing the right program, rather than the complementary problem of implementation, or writing the program right. However, many desirable software properties (such as portability) are obtained via adherence to coding standards, and there- fore fall outside the remit of formal specification and automatic verification. Moreover, code inspections and manual detection of standards violations are time consuming. To address these issues, this thesis describes Exstatic, a novel framework for the static detection of coding standards violations. Unlike many other static checkers Exstatic can be used to examine code in a variety of lan- guages, including program code, in-line documentation, markup languages and so on.
    [Show full text]
  • Simulation-Based Approach to Efficient Commonsense Reasoning in Very Large Knowledge Bases
    The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19) Simulation-Based Approach to Efficient Commonsense Reasoning in Very Large Knowledge Bases Abhishek Sharma , Keith M. Goolsbey Cycorp Inc., 7718 Wood Hollow Drive, Suite 250, Austin, TX 78731 [email protected], [email protected] Abstract paths. Therefore, to make the search more efficient in such Cognitive systems must reason with large bodies of general a KBS, an inference engine is expected to assess the utility knowledge to perform complex tasks in the real world. of further expanding each incomplete path. A naïve order- However, due to the intractability of reasoning in large, ex- ing algorithm can cause unproductive backtracking. To pressive knowledge bases (KBs), many AI systems have solve this problem, researchers have used two types of limited reasoning capabilities. Successful cognitive systems have used a variety of machine learning and axiom selection search control knowledge: (i) Axiom/premise selection methods to improve inference. In this paper, we describe a heuristics: These heuristics attempt to find the small num- search heuristic that uses a Monte-Carlo simulation tech- ber of axioms that are the most relevant for answering a set nique to choose inference steps. We test the efficacy of this of queries, and (ii) Certain researchers have worked on approach on a very large and expressive KB, Cyc. Experi- ordering heuristics for improving the order of rule and mental results on hundreds of queries show that this method is highly effective in reducing inference time and improving node expansions. question-answering (Q/A) performance. In the current work, we describe a simulation-based ap- proach for learning an ordering heuristic for controlling search in large knowledge-based systems (KBS).
    [Show full text]
  • Parallel Implementations of Random Time Algorithm for Chemical Network Stochastic Simulations
    Parallel implementations of random time algorithm for chemical network stochastic simulations Chuanbo Liua, Jin Wanga,b,∗ aState Key Laboratory of Electroanalytical Chemistry, Changchun Institute of Applied Chemistry, Chinese Academy of Sciences, Jilin, People’s Republic of China bDepartment of Chemistry, Physics and Applied Mathematics, State University of New York at Stony Brook, Stony Brook, USA Abstract In this study, we have developed a parallel version of the random time simulation algorithm. Firstly, we gave a rigorous basis of the random time description of the stochastic process of chemical reaction network time evolution. And then we reviewed the random time simulation algorithm and gave the implementations for the parallel version of next reaction random time algorithm. The discussion of computational complexity suggested a factor of M (which is the connection number of the network) folds time consuming reduction for random time simulation algorithm as compared to other exact stochastic simulation algorithms, such as the Gillespie algorithm. For large-scale system, such like the protein- protein interaction network, M is on order of 108. We further demonstrate the power of random time simulation with a GPGPU parallel implementation which achieved roughly 100 folds acceleration as compared with CPU implementations. Therefore the stochastic simulation method we developed here can be of great application value for simulating time evolution process of large-scale network. Keywords: Random time, Stochastic simulations, Parallel algorithm, GPGPU 1. Introduction such like StochKit. We here demonstrated a new imple- mentation of random time algorithm with GPGPU for Chemical reaction network time evolution is intrinsi- stochastic simulations. By carefully arranging the data cally stochastic.
    [Show full text]