DOCSLIB.ORG

Parallel Algorithms Combinatorial Search Some Combinatorial Search Methods • Divide and Conquer

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Parallel Algorithms Combinatorial Search Some Combinatorial Search Methods • Divide and Conquer Parallel Programming Parallel algorithms Combinatorial Search Some Combinatorial Search Methods • Divide and conquer • Backtrack search • Branch and bound • Game tree search (minimax, alpha-beta) 2010@FEUP Parallel Algorithms - Combinatorial Search 2 Terminology • Combinatorial algorithm: computation performed on discrete structure • Combinatorial search: finding one or more optimal or suboptimal solutions in a defined problem space • Kinds of combinatorial search problem • Decision problem (exists (find 1 solution); doesn’t exist) • Optimization problem (the best solution) 2010@FEUP Parallel Algorithms - Combinatorial Search 3 Examples of Combinatorial Search • Laying out circuits in VLSI • Find the smallest area • Planning motion of robot arms • Smallest distance to move (with or without constraints) • Assigning crews to airline flights • Proving theorems • Playing games 2010@FEUP Parallel Algorithms - Combinatorial Search 4 Search Tree • Each node represents a problem or sub- problem • Root of tree: initial problem to be solved • Children of a node created by adding constraints (one move from the father) • AND node: to find solution, must solve problems represented by all children • OR node: to find a solution, solve any of the problems represented by the children 2010@FEUP Parallel Algorithms - Combinatorial Search 5 Search Tree (cont.) • AND tree • Contains only AND nodes • Divide-and-conquer algorithms • OR tree • Contains only OR nodes • Backtrack search and branch and bound • AND/OR tree • Contains both AND and OR nodes • Game trees 2010@FEUP Parallel Algorithms - Combinatorial Search 6 Divide and Conquer • Divide-and-conquer methodology • Partition a problem into subproblems • Solve the subproblems • Combine solutions to subproblems • Recursive: sub-problems may be solved using the divide-and-conquer methodology • Example: quicksort 2010@FEUP Parallel Algorithms - Combinatorial Search 7 Best for Centralized Multiprocessor • Unsolved subproblems kept in one shared stack • Processors needing work can access the stack • Processors with extra work can put it on the stack • Effective workload balancing mechanism • Stack can become a bottleneck as number of processors increases 2010@FEUP Parallel Algorithms - Combinatorial Search 8 Multicomputer Divide and Conquer • Subproblems must be distributed among memories of individual processors • Two designs • Original problem and final solution stored in memory of a single processor • Both original problem and final solution distributed among memories of all processors 2010@FEUP Parallel Algorithms - Combinatorial Search 9 Design 1 • Algorithm has three phases • Phase 1: problems divided and propagated throughout the parallel computer • Phase 2: processors compute solutions to their subproblems • Phase 3: partial results are combined • Maximum speedup limited by propagation and combining overhead 2010@FEUP Parallel Algorithms - Combinatorial Search 10 Design 2 • Both original problem and final solution are distributed among processors’ memories • Eliminates starting up and winding down phases of first design • Allows maximum problem size to increase with number of processors • Used this approach for parallel quicksort algorithms • Challenge: keeping workloads balanced among processors 2010@FEUP Parallel Algorithms - Combinatorial Search 11 Backtrack Search • Uses depth-first search to consider alternative solutions to a combinatorial search problem • Recursive algorithm • Backtrack occurs when • A node has no children (“dead end”) • All of a node’s children have been explored 2010@FEUP Parallel Algorithms - Combinatorial Search 12 Example: Crossword Puzzle Creation • Given • Blank crossword puzzle • Dictionary of words and phrases • Assign letters to blank spaces so that all puzzle’s horizontal and vertical “words” are from the dictionary • Halt as soon as a solution is found 2010@FEUP Parallel Algorithms - Combinatorial Search 13 Crossword Puzzle Problem Given a blank crossword puzzle and a dictionary ............. find a way to fill in the puzzle. 2010@FEUP Parallel Algorithms - Combinatorial Search 14 A Search Strategy • Identify longest incomplete word in puzzle (break ties arbitrarily) • Look for a word of that length • If cannot find such a word, backtrack • Otherwise, find longest incomplete word that has at least one letter assigned (break ties arbitrarily) • Look for a word of that length • If cannot find such a word, backtrack • Recurse until a solution is found or all possibilities have been attempted 2010@FEUP Parallel Algorithms - Combinatorial Search 15 State Space Tree Root of tree is initial, blank puzzle. Choices for word 1 Choices for word 2 Word 3 choices etc. 2010@FEUP Parallel Algorithms - Combinatorial Search 16 Backtrack Search ? ? ? ? ? ? ? 2010@FEUP Parallel Algorithms - Combinatorial Search 17 Backtrack Search T R ? ? O ? ? ? ? L L E Y 2010@FEUP Parallel Algorithms - Combinatorial Search 18 Backtrack Search T ? R ? C L O S E T S L ? L ? E ? Y ? 2010@FEUP Parallel Algorithms - Combinatorial Search 19 Backtrack Search T ? R ? C L O S E T S L ? L ? E ? Cannot find word. Must backtrack. Y ? 2010@FEUP Parallel Algorithms - Combinatorial Search 20 Backtrack Search T R ? ? O ? ? ? ? L L E Cannot find word. Must backtrack. Y 2010@FEUP Parallel Algorithms - Combinatorial Search 21 Backtrack Search T ? R ? C R O Q U E T L ? L ? E ? Y ? 2010@FEUP Parallel Algorithms - Combinatorial Search 22 Backtrack Search T T R R C R O Q U E T L M ? ? L ? P ? ? E E Y D 2010@FEUP Parallel Algorithms - Combinatorial Search 23 Time and Space Complexity • Suppose average branching factor in state space tree is b • Searching a tree of depth k requires examining bk1 b 1b b2 bk 1 (bk ) b 1 nodes in the worst case (exponential time) • Amount of memory usually required is (k) 2010@FEUP Parallel Algorithms - Combinatorial Search 24 Parallel Backtrack Search • First strategy: give each processor a subtree • Suppose p = bk • A process searches all nodes to depth k • It then explores only one of subtrees rooted at level k • If d (depth of search) > 2k, time required by each process to traverse first k levels of state space tree is negligible 2010@FEUP Parallel Algorithms - Combinatorial Search 25 Parallel Backtrack when p = bk Subtree Subtree Subtree Subtree Searched Searched Searched Searched by by by by Process 0 Process 1 Process 2 Process 3 2010@FEUP Parallel Algorithms - Combinatorial Search 26 What If p bk ? • A process can perform sequential search to level m (where bm > p) of state space tree • Each process explores its share of the subtrees rooted by nodes at level m • As m increases, there are more subtrees to divide among processes, which can make workloads more balanced • Increasing m also increases number of redundant computations 2010@FEUP Parallel Algorithms - Combinatorial Search 27 Maximum Speedup when p bk In this example 5 processors are exploring a state space tree with branching factor 3 and depth 10. 2010@FEUP Parallel Algorithms - Combinatorial Search 28 Disadvantage of Allocating One Subtree per Process • In most cases state space tree is not balanced • Example: in crossword puzzle problem, some word choices lead to dead ends quicker than others • Alternative: make sequential search go deeper, so that each process handles many subtrees (cyclic allocation) 2010@FEUP Parallel Algorithms - Combinatorial Search 29 Allocating Many Subtrees per Process b = 3; p = 4; m = 3; allocation rule (subtree nr) % p == rank 2010@FEUP Parallel Algorithms - Combinatorial Search 30 Backtrack Algorithm cutoff_count – nr of nodes at cutoff_depth cutoff_depth – depth at which subtrees are divided among processes depth – maximum search depth in the state space tree moves – records the path to the current node (moves made so far) p, id – number of processes, process rank Parallel_Backtrack(node, level) if (level == depth) if (node is a solution) Print_Solution(moves) else if (level == cutoff_depth) cutoff_count ++ if (cutoff_count % p != id) return possible_moves = Count_Moves(node) // nr of possible moves from current node for i = 1 to possible_moves node = Make_Move(node, i) moves[ level ] = i Parallel_Backtrack(node, level+1) node = Unmake_Move(node, i) return 2010@FEUP Parallel Algorithms - Combinatorial Search 31 Distributed Termination Detection • Suppose we only want to print one solution • We want all processes to halt as soon as one process finds a solution • This means processes must periodically check for messages • Every process calls MPI_Iprobe every time search reaches a particular level (such as the cutoff depth) • A process sends a message after it has found a solution 2010@FEUP Parallel Algorithms - Combinatorial Search 32 Simple (Incorrect) Algorithm • A process halts after one of the following events has happened: • It has found a solution and sent a message to all of the other processes • It has received a message from another process • It has completely searched its portion of the state space tree 2010@FEUP Parallel Algorithms - Combinatorial Search 33 Why Algorithm Fails • If a process calls MPI_Finalize before another active process attempts to send it a message, we get a run-time error • How this could happen? • A process finds a solution after another process has finished searching its share of the subtrees OR • A process finds a solution after another process has found a solution 2010@FEUP Parallel Algorithms - Combinatorial Search 34 Distributed Termination Problem • Distributed termination problem: Ensuring that • all processes are inactive AND • no
Load more

Recommended publications
  • 6 Adversarial Search
    6 ADVERSARIAL SEARCH In which we examine the problems that arise when we try to plan ahead in a world where other agents are planning against us. 6.1 GAMES Chapter 2 introduced multiagent environments, in which any given agent will need to con- sider the actions of other agents and how they affect its own welfare. The unpredictability of these other agents can introduce many possible contingencies into the agent’s problem- solving process, as discussed in Chapter 3. The distinction between cooperative and compet- itive multiagent environments was also introduced in Chapter 2. Competitive environments, in which the agents’ goals are in conflict, give rise to adversarial search problems—often GAMES known as games. Mathematical game theory, a branch of economics, views any multiagent environment as a game provided that the impact of each agent on the others is “significant,” regardless of whether the agents are cooperative or competitive.1 In AI, “games” are usually of a rather specialized kind—what game theorists call deterministic, turn-taking, two-player, zero-sum ZERO­SUM GAMES games of perfect information. In our terminology, this means deterministic, fully observable PERFECT INFORMATION environments in which there are two agents whose actions must alternate and in which the utility values at the end of the game are always equal and opposite. For example, if one player wins a game of chess (+1), the other player necessarily loses (–1). It is this opposition between the agents’ utility functions that makes the situation adversarial. We will consider multiplayer games, non-zero-sum games, and stochastic games briefly in this chapter, but will delay discussion of game theory proper until Chapter 17.
    [Show full text]
  • "Large Scale Monte Carlo Tree Search on GPU"
    Large Scale Monte Carlo Tree Search on GPU GPU+HK大ávモンテカルロ木探索 Doctoral Dissertation ¿7論£ Kamil Marek Rocki ロツキ カミル >L#ク Submitted to Department of Computer Science, Graduate School of Information Science and Technology, The University of Tokyo in partial fulfillment of the requirements for the degree of Doctor of Information Science and Technology Thesis Supervisor: Reiji Suda (½田L¯) Professor of Computer Science August, 2011 Abstract: Monte Carlo Tree Search (MCTS) is a method for making optimal decisions in artificial intelligence (AI) problems, typically for move planning in combinatorial games. It combines the generality of random simulation with the precision of tree search. Research interest in MCTS has risen sharply due to its spectacular success with computer Go and its potential application to a number of other difficult problems. Its application extends beyond games, and MCTS can theoretically be applied to any domain that can be described in terms of (state, action) pairs, as well as it can be used to simulate forecast outcomes such as decision support, control, delayed reward problems or complex optimization. The main advantages of the MCTS algorithm consist in the fact that, on one hand, it does not require any strategic or tactical knowledge about the given domain to make reasonable decisions, on the other hand algorithm can be halted at any time to return the current best estimate. So far, current research has shown that the algorithm can be parallelized on multiple CPUs. The motivation behind this work was caused by the emerging GPU- based systems and their high computational potential combined with the relatively low power usage compared to CPUs.
    [Show full text]
  • Compiler Construction
    Compiler construction PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Sat, 10 Dec 2011 02:23:02 UTC Contents Articles Introduction 1 Compiler construction 1 Compiler 2 Interpreter 10 History of compiler writing 14 Lexical analysis 22 Lexical analysis 22 Regular expression 26 Regular expression examples 37 Finite-state machine 41 Preprocessor 51 Syntactic analysis 54 Parsing 54 Lookahead 58 Symbol table 61 Abstract syntax 63 Abstract syntax tree 64 Context-free grammar 65 Terminal and nonterminal symbols 77 Left recursion 79 Backus–Naur Form 83 Extended Backus–Naur Form 86 TBNF 91 Top-down parsing 91 Recursive descent parser 93 Tail recursive parser 98 Parsing expression grammar 100 LL parser 106 LR parser 114 Parsing table 123 Simple LR parser 125 Canonical LR parser 127 GLR parser 129 LALR parser 130 Recursive ascent parser 133 Parser combinator 140 Bottom-up parsing 143 Chomsky normal form 148 CYK algorithm 150 Simple precedence grammar 153 Simple precedence parser 154 Operator-precedence grammar 156 Operator-precedence parser 159 Shunting-yard algorithm 163 Chart parser 173 Earley parser 174 The lexer hack 178 Scannerless parsing 180 Semantic analysis 182 Attribute grammar 182 L-attributed grammar 184 LR-attributed grammar 185 S-attributed grammar 185 ECLR-attributed grammar 186 Intermediate language 186 Control flow graph 188 Basic block 190 Call graph 192 Data-flow analysis 195 Use-define chain 201 Live variable analysis 204 Reaching definition 206 Three address
    [Show full text]
  • Randomized Parallel Algorithms for Backtrack Search and Branch-And-Bound Computation
    Randomized Parallel Algorithms for Backtrack Search and Branch-and-Bound Computation RICHARD M. KARP AND YANJUN ZHANG University of California at Berkeley, Berkeley, California Abstract. Universal randomized methods for parallelizing sequential backtrack search and branch-and-bound computation are presented. These methods execute on message-passing multi- processor systems, and require no global data structures or complex communication protocols. For backtrack search, it is shown that, uniformly on all instances, the method described in this paper is likely to yield a speed-up within a small constant factor from optimal, when all solutions to the problem instance are required. For branch-and-bound computation, it is shown that, uniformly on all instances, the execution time of this method is unlikely to exceed a certain inherent lower bound by more than a constant factor. These randomized methods demonstrate the effectiveness of randomization in distributed parallel computation. Categories and Subject Descriptors: F.2.2 [Analysis of Algorithms and Problem Complexity]: Non-numerical Algorithms-computation on discrete structures; sorting and searching; F.1.2 [Com- putation by Abstract Devices]: Modes of computation-parallelism and concurrency; probabilistic computation; G.2.1 [Discrete Mathematics]: Combinatorics-combinutoria~ algorithms; G.3 [Prob- ability and Statistics]: probabilistic algotithms; 1.2.8 [Artificial Intelligence]: Problem Solving, Control Methods, and Search-backtracking; graph and tree search strategies General Terms: Algorithms Additional Key Words and Phrases: Backtrack search, branch-and-bound, distributed parallel computation 1. Introduction A combinatorial search problem, or simply search problem, is a problem of finding certain arrangements of some combinatorial objects among a large set of possible arrangements.
    [Show full text]
  • Heuristics Table of Contents
    Part I: Heuristics Table of Contents List of Contributors ......................................................... ix Preface ...................................................................... xi I. Heuristics ........................................................... 1 1. Heuristic Search for Planning Under Uncertainty Blai Bonet and Eric A. Hansen . 3 2. Heuristics, Planning, and Cognition Hector Geffner . 23 3. Mechanical Generation of Admissible Heuristics Robert Holte, Jonathan Schaeffer, and Ariel Felner . 43 4. Space Complexity of Combinatorial Search Richard E. Korf . 53 5. Paranoia Versus Overconfidence in Imperfect-Information Games Austin Parker, Dana Nau and V.S. Subrahmanian . 63 6. Heuristic Search: Pearl's Significance From a Personal Perspective Ira Pohl ................................................................. 89 II. Probability ......................................................... 103 7. Inference in Bayesian Networks: A Historical Perspective Adnan Darwiche . 105 8. Graphical Models of the Visual Cortex Thomas Dean . 121 9. On the Power of Belief Propagation: A Constraint Propagation Perspective Rina Dechter, Bozhena Bidyuk, Robert Mateescu, and Emma Rollon . 143 10. Bayesian Nonparametric Learning: Expressive Priors for Intelligent Systems Michael I. Jordan . 167 v 1 Heuristic Search for Planning under Uncertainty Blai Bonet and Eric A. Hansen 1 Introduction The artificial intelligence (AI) subfields of heuristic search and automated planning are closely related, with planning problems often providing a
    [Show full text]
  • Understanding Sampling-Based Adversarial Search Methods
    UNDERSTANDING SAMPLING-BASED ADVERSARIAL SEARCH METHODS A Dissertation Presented to the Faculty of the Graduate School of Cornell University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy by Raghuram Ramanujan August 2012 c 2012 Raghuram Ramanujan ALL RIGHTS RESERVED UNDERSTANDING SAMPLING-BASED ADVERSARIAL SEARCH METHODS Raghuram Ramanujan, Ph.D. Cornell University 2012 Until 2007, the best computer programs for playing the board game Go per- formed at the level of a weak amateur, while employing the same Minimax algo- rithm that had proven so successful in other games such as Chess and Checkers. Thanks to a revolutionary new sampling-based planning approach named Up- per Confidence bounds applied to Trees (UCT), today’s best Go programs play at a master level on full-sized 19 × 19 boards. Intriguingly, UCT’s spectacular success in Go has not been replicated in domains that have been the traditional stronghold of Minimax-style approaches. The focus of this thesis is on under- standing this phenomenon. We begin with a thorough examination of the vari- ous facets of UCT in the games of Chess and Mancala, where we can contrast the behavior of UCT to that of the better understood Minimax approach. We then introduce the notion of shallow search traps — positions in games where short winning strategies for the opposing player exist — and demonstrate that these are distributed very differently in different games, and that this has a significant impact on the performance of UCT. Finally, we study UCT and Minimax in two novel synthetic game settings that permit mathematical analysis.
    [Show full text]
  • Selective Search in Games of Different Complexity
    Selective Search in Games of Different Complexity Selective Search in Games of Different Complexity PROEFSCHRIFT ter verkrijging van de graad van doctor aan de Universiteit Maastricht, op gezag van de Rector Magnificus, Prof. mr. G.P.M.F. Mols, volgens het besluit van het College van Decanen, in het openbaar te verdedigen op woensdag 25 mei 2011 om 16.00 uur door Maarten Peter Dirk Schadd Promotor: Prof. dr. G. Weiss Copromotor: Dr. M.H.M. Winands Dr. ir. J.W.H.M. Uiterwijk Leden van de beoordelingscommissie: Prof. dr. ir. R.L.M. Peeters (voorzitter) Dr. Y. Bj¨ornsson(Reykjavik University) Prof. dr. H.J.M. Peters Prof. dr. ir. J.A. La Poutr´e(Universiteit Utrecht) Prof. dr. ir. J.C. Scholtes This research has been funded by the Netherlands Organisation for Scientific Research (NWO) in the framework of the project TACTICS, grant number 612.000.525. Dissertation Series No. 2011-16 The research reported in this thesis has been carried out under the auspices of SIKS, the Dutch Research School for Information and Knowledge Systems. Printed by Optima Grafische Communicatie, Rotterdam ISBN 978-94-6169-060-9 c 2011 M.P.D. Schadd, Maastricht, The Netherlands. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronically, mechanically, photo- copying, recording or otherwise, without prior permission of the author. Preface As far as I can think back, I have enjoyed playing all kind of games. A fascinating property of games is that with simple rules, very complex behavior can be created.
    [Show full text]
  • Anytime Tree Search for Combinatorial Optimization Luc Libralesso
    Anytime tree search for combinatorial optimization Luc Libralesso To cite this version: Luc Libralesso. Anytime tree search for combinatorial optimization. Computer Arithmetic. Université Grenoble Alpes [2020-..], 2020. English. NNT : 2020GRALM026. tel-03014697 HAL Id: tel-03014697 https://tel.archives-ouvertes.fr/tel-03014697 Submitted on 19 Nov 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THÈSE Pour obtenir le grade de DOCTEUR DE L’UNIVERSITÉ GRENOBLE ALPES Spécialité : MATHÉMATIQUES & INFORMATIQUE Arrêté ministériel : 25 mai 2016 Présentée par Luc LIBRALESSO Thèse dirigée par Louis ESPERET, Chargé de Recherche CNRS, G-SCOP et co-encadrée par Vincent JOST, Chargé de Recherche CNRS, G-SCOP Thibault Honegger, Chief Scientific Officer, NETRI préparée au sein du Laboratoire des Sciences pour la Conception, l'Optimisation et la Production de Grenoble (G-SCOP) dans l'École Doctorale Mathématiques, Sciences et technologies de l'information, Informatique (ED-MSTII) Recherches arborescentes anytime pour l’optimisation combinatoire Anytime tree search
    [Show full text]
  • Search and Learning Algorithms for Two-Player Games with Application to the Game of Hex Chao
    Search and Learning Algorithms for Two-Player Games with Application to the Game of Hex by Chao Gao A thesis submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computing Science University of Alberta c Chao Gao, 2020 Abstract Two-Player alternate-turn perfect-information zero-sum games have been suggested as a testbed for Artificial Intelligence research since Shannon in 1950s. In this thesis, we summarize and develop algorithms for this line of research. We focus on the game of Hex | a game created by Piet Hein in 1942 and popularized by Jonh Nash in 1949. We continue on previous research, further bringing the strength of machine learning techniques | specifically deep neural networks | to the game of Hex. We develop new methods and apply them to solving and playing Hex. We present state-of-the-art results for Hex and discuss relevant research on other domains. In particular, we develop solving and playing algorithm by combining search and deep learning methods. As a result of our algorithmic innovations, we produce stronger solver and player, winning gold medal at the Computer Olympiad Hex tournaments in 2017 and 2018. ii What I believe is that if it takes 200 years to achieve artificial intelligence, and finally there is a textbook that explains how it is done; the hardest part of that textbook to write will be the part that explains why people didn't think of it 200 years ago, because we are really talking about how to make machines do things that are on the surface of our minds.
    [Show full text]
  • 5 Adversarial Search
    5 ADVERSARIAL SEARCH In which we examine the problems that arise when we try to plan ahead in a world where other agents are planning against us. 5.1 GAMES Chapter 2 introduced multiagent environments, in which each agent needs to consider the actions of other agents and how they affect its own welfare. The unpredictability of these other agents can introduce contingencies into the agent’s problem-solving process, as dis- cussed in Chapter 4. In this chapter we cover competitive environments, in which the agents’ GAME goals are in conflict, giving rise to adversarial search problems—often known as games. Mathematical game theory, a branch of economics, views any multiagent environment as a game, provided that the impact of each agent on the others is “significant,” regardless of whether the agents are cooperative or competitive.1 In AI, the most common games are of a rather specialized kind—what game theorists call deterministic, turn-taking, two-player, ZERO-SUM GAMES zero-sum games of perfect information (such as chess). In our terminology, this means PERFECT INFORMATION deterministic, fully observable environments in which two agents act alternately and in which the utility values at the end of the game are always equal and opposite. For example, if one player wins a game of chess, the other player necessarily loses. It is this opposition between the agents’ utility functions that makes the situation adversarial. Games have engaged the intellectual faculties of humans—sometimes to an alarming degree—for as long as civilization has existed. For AI researchers, the abstract nature of games makes them an appealing subject for study.
    [Show full text]