Heuristic Search Theory and Applications

Stefan Edelkamp

Stefan Schrodl

AMSTERDAM • BOSTON • HEIDELBERG • LONDON

NEW YORK • OXFORD • PARIS • SAN DIEGO

SAN FRANCISCO • SINGAPORE • SYDNEY » TOKYO

ELSEVIER Morgan Kaufmann is an imprint of Elsevier Contents

List of Algorithms

Preface xxul

PART I HEURISTIC SEARCH PRIMER

CHAPTER 1 Introduction 3 1.1 Notational and Mathematical Background 3 1.1.1 Pseudocode 3 1.1.2 Computability Theory 4 1.1.3 Complexity Theory 5 1.1.4 Asymptotic Resource Consumption 7 1.1.5 Symbolic Logic 8 1.2 Search 9

1.3 Success Stories 9 1.4 State Space Problems 12 1.5 Problem Graph Representations 14 1.6 Heuristics 15 1.7 Examples of Search Problems 19 1.7.1 Sliding-Tile Puzzles 19 1.7.2 Rubik's Cube 22

1.7.3 Sokoban 22 1.7.4 Route Planning 24

1.7.5 TSP 25 1.7.6 Multiple Sequence Alignment 26 1.8 General State Space Descriptions 29 1.8.1 Action Planning 29 1.8.2 *Production Systems 32 1.8.3 Markov Decision Processes 35

1.8.4 Generic Search Model 36 1.9 Summary 37 1.10 Exercises 39 1.11 Bibliographic Notes 45

CHAPTER 2 Basic Search Algorithms 47 2.1 Uninformed Graph Search Algorithms 48 2.1.1 Depth-First Search 50 2.1.2 Breadth-First Search 51 2.1.3 Dijkstra's Algorithm 53

V vi Contents

2.1.4 Negatively Weighted Graphs 57 2.1.5 Relaxed Node Selection 58 2.1.6 * Algorithm of Bellman-Ford 60 2.1.7 Dynamic Programming 62 2.2 Informed Optimal Search 68 2.2.1 A* 69 2.2.2 On the Optimal Efficiency of A* 74 2.3 *General Weights 75 2.3.1 Cost Algebras 76 2.3.2 Multiobjective Search 79 2.4 Summary 80 2.5 Exercises 82 2.6 Bibliographic Notes 86

CHAPTER 3 *Dictionary Data Structures 89 3.1 Priority Queues 89 3.1.1 Bucket Data Structures 90 3.1.2 Heap Data Structures 97 3.2 Hash Tables 110

3.2.1 Hash Dictionaries Ill

3.2.2 Hash Functions 112 3.2.3 Hashing Algorithms 121 3.2.4 Memory-Saving Dictionaries 130 3.2.5 Approximate Dictionaries 134 3.3 Subset Dictionaries 138 3.3.1 Arrays and Lists 139 3.3.2 Tries 139 3.3.3 Hashing 139 3.3.4 Unlimited Branching Trees 141 3.4 String Dictionaries 142 3.4.1 Suffix Trees 143

3.4.2 Generalized Suffix Trees 146 3.5 Summary 151 3.6 Exercises 154 3.7 Bibliographic Notes 157

CHAPTER 4 Automatically Created Heuristics 161 4.1 Abstraction Transformations 162 4.2 Valtorta's Theorem 164

4.3 hierarchical A* 165

4.4 Pattern Databases 167 4.4.1 Fifteen-Puzzle 168 4.4.2 Rubik's Cube 169 4.4.3 Directed Search Graphs 170 4.4.4 Korf's Conjecture 170 4.4.5 Multiple Pattern Databases 172 4.4.6 Disjoint Pattern Databases 173 4.5 *Customized Pattern Databases 177

4.5.1 Pattern Selection 178 4.5.2 Symmetry and Dual Pattern Databases 179 4.5.3 Bounded Pattern Databases 180

4.5.4 On-Demand Pattern Databases 183 4.5.5 Compressed Pattern Databases 183 4.5.6 Compact Pattern Databases 185 4.6 Summary 185 4.7 Exercises 187 4.8 Bibliographic Notes 190

PART II HEURISTIC SEARCH UNDER MEMORY CONSTRAINTS

CHAPTER 5 Linear-Space Search 195 5.1 ^Logarithmic Space Algorithms 195 5.1.1 Divide-and-Conquer BFS 196 5.1.2 Divide-and-Conquer Shortest Paths Search 196 5.2 Exploring the Search Tree 197 5.3 Branch-and-Bound 198 5.4 Iterative-Deepening Search 201 5.5 Iterative-Deepening A* 204 5.6 Prediction of IDA* Search 206 5.6.1 Asymptotic Branching Factors 206 5.6.2 IDA* Search Tree Prediction 212 5.6.3 *Convergence Criteria 217 5.7 * Refined Threshold Determination 218 5.8 *Recursive Best-First Search 219 5.9 Summary 220 5.10 Exercises 222 5.11 Bibliographic Notes 223

CHAPTER 6 Memory-Restricted Search 227 6.1 Linear Variants Using Additional Memory 228 6.1.1 Transposition Tables 228 6.1.2 Fringe Search 231 viii Contents

6.1.3 iterative Threshold Search 233 6.1.4 MA*, SMA, and SMAG 235

6.2 Nonadmissible Search 240 6.2.1 Enforced Hill-Climbing 240 6.2.2 Weighted A* 243 6.2.3 Overconsistent A* 244 6.2.4 Anytime Repahing A* 246 6.2.5 Jt-Best-First Search 250

6.2.6 Beam Search 251 6.2.7 Partial A* and Partial IDA* 251 6.3 Reduction of the Closed List 254 6.3.1 Dynamic Programming in Implicit Graphs 255 6.3.2 Divide-and-Conquer Solution Reconstruction 255 6.3.3 Frontier Search 257 6.3.4 *Sparse Memory Graph Search 259 6.3.5 Breadth-First Heuristic Search 261 6.3.6 Locality 264 6.4 Reduction of the Open List 266 6.4.1 Beam-Stack Search 266 6.4.2 Partial Expansion A* 270 6.4.3 Two-Bit Breadth-First Search 271 6.5 Summary 273 6.6 Exercises 275 6.7 Bibliographic Notes 279

CHAPTER 7 Symbolic Search 283 7.1 Boolean Encodings for Set of States 284 7.2 Binary Decision Diagrams 286 7.3 Computing the Image for a State Set 290 7.4 Symbolic Blind Search 291 7.4.1 Symbolic Breadth-First Tree Search 291 7.4.2 Symbolic Breadth-First Search 293 7.4.3 Symbolic Pattern Databases 294 7.4.4 Cost-Optimal Symbolic Breadth-First Search 296 7.4.5 Symbolic Shortest Path Search 298 7.5 Limits and Possibilities of BDDs 299 7.5.1 Exponential Lower Bound 299 7.5.2 Polynomial Upper Bound 300 7.6 Symbolic Heuristic Search 302 7.6.1 Symbolic A* 302 7.6.2 Bucket Implementation 305 Contents ix

7.6.3 Symbolic Best-First Search 307 7.6.4 Symbolic Breadth-First Branch-and-Bound 308 7.7 *Refinements 309 7.7.1 Improving the BDD Size 309 7.7.2 Partitioning 310 7.8 Symbolic Algorithms for Explicit Graphs 311 7.9 Summary 311 7.10 Exercises 313 7.11 Bibliographic Notes 317

CHAPTER 8 External Search 319 8.1 Virtual Memory Management 320 8.2 Fault Tolerance 321 8.3 Model of Computation 321 8.4 Basic Primitives 323 8.5 External Explicit Graph Search 324 8.5.1 * External Priority Queues 324 8.5.2 External Explicit Graph Depth-First Search 325 8.5.3 External Explicit Graph Breadth-First Search 326 8.6 External Implicit Graph Search 328 8.6.1 Delayed Duplicate Detection for BFS 328 8.6.2 *External Breadth-First Branch-and-Bound 330 8.6.3 *External Enforced Hill-Climbing 332 8.6.4 External A* 334 8.6.5 *Lower Bound for Delayed Duplicate Detection 340 8.7 *Refinements 342 8.7.1 Hash-Based Duplicate Detection 342 8.7.2 Structured Duplicate Detection 343 8.7.3 Pipelining 344 8.7.4 External Iterative-Deepening A* Search 345 8.7.5 External Explicit-State Pattern Databases 346 8.7.6 External Symbolic Pattern Databases 348 8.7.7 External Relay Search 348 8.8 *External Value Iteration 349 8.8.1 Forward Phase: State Space Generation 349 8.8.2 Backward Phase: Update of Values 349 8.9 *Flash Memory 353 8.9.1 Hashing 354 8.9.2 Mapping 354 8.9.3 Compressing 356 8.9.4 Flushing 358 x Contents

8.10 Summary 358 8.11 Exercises 361 8.12 Bibliographic Notes 364

PART III HEURISTIC SEARCH UNDER TIME CONSTRAINTS

CHAPTER 9 Distributed Search 369 9.1 Parallel Processing 370 9.1.1 Motivation for Practical Parallel Search 374 9.1.2 Space Partitioning 374 9.1.3 Depth Slicing 375 9.1.4 Lock-Free Hashing 378 9.2 Parallel Depth-First Search 379 9.2.1 *Parallel Branch-and-Bound 379 9.2.2 Stack Splitting 380 9.2.3 Parallel IDA* 382 9.2.4 Asynchronous IDA* 382 9.3 Parallel Best-First Search Algorithms 385

9.3.1 Parallel Global A* 385

9.3.2 Parallel Local A* 387

9.4 Parallel External Search 388

9.4.1 Parallel External Breadth-First Search 388 9.4.2 Parallel Structured Duplicate Detection 391 9.4.3 Parallel External A* 392 9.4.4 Parallel Pattern Database Search 398 9.5 Parallel Search on the GPU 400

9.5.1 GPU Basics 401

9.5.2 GPU-Based Breadth-First Search 401

9.5.3 Bitvector GPU Search 407

9.6 Bidirectional Search 408 9.6.1 Bidirectional Front-to-End Search 409

9.6.2 *Biclirectional Front-to-Front Search 410

9.6.3 Perimeter Search 412

9.6.4 Bidirectional Symbolic Breadth-First Search 416

9.6.5 *Island Search 416

9.6.6 *Multiple-Goal Heuristic Search 418

9.7 Summary 419

9.8 Exercises 422 9.9 Bibliographic Notes 425 Contents xi

CHAPTER 10 State Space Pruning 429 10.1 Admissible State Space Pruning 430 10.1.1 Substring Pruning 430 10.1.2 Pruning Dead-Ends 440 10.1.3 Penalty Tables 445 10.1.4 Symmetry Reduction 447 10.2 Nonadmissible State Space Pruning 449 10.2.1 Macro Problem Solving 449 10.2.2 Relevance Cuts 452 10.2.3 Partial Order Reduction 453 10.3 Summary 458 10.4 Exercises 460 10.5 Bibliographic Notes 463

CHAPTER 11 Real-Time Search 465

11.1 LRTA* 466 11.2 LRTA* with Lookahead One 473 11.3 Analysis of the Execution Cost of LRTA* 474 11.3.1 Upper Bound on the Execution Cost of LRTA* 476

11.3.2 Lower Bound on the Execution Cost of LRTA* 477

11.4 Features of LRTA* 479 11.4.1 Heuristic Knowledge 479 11.4.2 Fine-Grained Control 479 11.4.3 Improvement of Execution Cost 480 11.5 Variants of LRTA* 482 11.5.1 Variants with Local Search Spaces of Varying Sizes 482 11.5.2 Variants with Minimal Lookahead 482 11.5.3 Variants with Faster Value Updates 484 11.5.4 Variants That Detect Convergence 488 11.5.5 Variants That Speed Up Convergence 488 11.5.6 Nonconverging Variants 491 11.5.7 Variants for Nondeterministic and Probabilistic State Spaces 494

11.6 How to Use Real-Time Search 496 11.6.1 Case Study. Off line Search 496 11.6.2 Case Study: Goal-Directed Navigation in Unknown Terrain 497 11.6.3 Case Study: Coverage 500 11.6.4 Case Study: Localization 501 11.7 Summary 507 11.8 Exercises 509 11.9 Bibliographic Notes 514 xii Contents

PART IV HEURISTIC SEARCH VARIANTS

CHAPTER 12 Adversary Search 519 12.1 Two-Player Games 520 12.1.1 Game Tree Search 524 12.1.2 aj3-Pruning 525 12.1.3 Transposition Tables 530 12.1.4 *Searching with Restricted Windows 530 12.1.5 Accumulated Evaluations 534 12.1.6 *Partition Search 535 12.1.7 *Other Improvement Techniques 537 12.1.8 Learning Evaluation Functions 538 12.1.9 Retrograde Analysis 543 12.1.10 *Symbolic Retrograde Analysis 544 12.2 *Multiplayer Games 547 12.3 General Game Playing 550 12.4 AND/OR Graph Search 552 12.4.1 AO* 554 12.4.2 *IDAO* 554 12.4.3 *LAO* 557 12.5 Summary 559 12.6 Exercises 563 12.7 Bibliographic Notes 567

CHAPTER 13 Constraint Search 571 13.1 Constraint Satisfaction 572 13.2 Consistency 575 13.2.1 Arc Consistency 575 13.2.2 Bounds Consistency 577 13.2.3 *Path Consistency 578 13.2.4 Specialized Consistency 579 13.3 Search Strategies 580 13.3.1 Backtracking 581 13.3.2 Backjumping 583 13.3.3 Dynamic Backtracking 584 13.3.4 Backmarking 585 13.3.5 Search Strategies 587 13.4 NP-Hard Problem Solving 591 13.4.1 Boolean Satisfiability 592 13.4.2 Number Partition 596 Contents xiii

13.4.3 *Bin Packing 598 13.4.4 * Rectangle Packing 601 13.4.5 * Vertex Cover, Independent Set, Clique 604 13.4.6 *Graph Partition 606 13.5 Temporal Constraint Networks 609 13.5.1 Simple Temporal Network 609 13.5.2 *PERT Scheduling 611 13.6 * Path Constraints 612 13.6.1 Formula Progression 614 13.6.2 Automata Translation 615 13.7 * Soft and Preference Constraints 617 13.8 Constraint Optimization 618 13.9 Summary 619 13.10 Exercises 623 13.11 Bibliographic Notes 629

CHAPTER 14 Selective Search 633 14.1 From State Space Search to Minimization 634 14.2 Hill-Climbing Search 635 14.3 Simulated Annealing 637 14.4 Tabu Search 638 14.5 Evolutionary Algorithms 639 14.5.1 Randomized Local Search and (1 + 1) EA 639 14.5.2 Simple GA 641 14.5.3 Insights to Genetic Algorithm Search 643 14.6 Approximate Search 645 14.6.1 Approximating TSP 646 14.6.2 Approximating MAX-fc-SAT 646 14.7 Randomized Search 647 14.8 Ant Algorithms 652 14.8.1 Simple Ant System 652 14.8.2 Algorithm Flood 654 14.8.3 Vertex Ant Walk 654 14.9 * Lagrange Multipliers 657 14.9.1 Saddle-Point Conditions 658

14.9.2 Partitioned Problems 661

14.10 *No-Free-Lunch 662 14.11 Summary 662 14.12 Exercises 664 14.13 Bibliographic Notes 668 xiv Contents

PART V HEURISTIC SEARCH APPLICATIONS

CHAPTER 15 Action Planning 673 15.1 Optimal Planning 675 15.1.1 Graphplcm 675 \5A.2 Satplan 677 15.1.3 Dynamic Programming 678 15.1.4 Planning Pattern Databases 680 15.2 Suboptimal Planning 685 15.2.1 Causal Graphs 685 15.2.2 Metric Planning 687 15.2.3 Temporal Planning 691 15.2.4 Derived Predicates 695

15.2.5 Timed Initial Literals 696 15.2.6 State Trajectory Constraints 696 15.2.7 Preference Constraints 697 15.3 Bibliographic Notes 697

CHAPTER 16 Automated System Verification 701 16.1 Model Checking 702 16.1.1 Temporal Logics 702 16.1.2 The Role of Heuristics 703 16.2 Communication Protocols 705

16.2.1 Formula-Based Heuristic 705 16.2.2 Activeness Heuristic 707

16.2.3 Trail-Directed Heuristics 709 16.2.4 Liveness Model Checking 710 16.2.5 Planning Heuristics 710 16.3 Program Model Checking 713 16.4 Analyzing Petri Nets 717 16.5 Exploring Real-Time Systems 721 16.5.1 Timed Automata 721 16.5.2 Linearly Priced Timed Automata 722 16.5.3 Traversal Politics 723 16.6 Analyzing Graph Transition Systems 723 16.7 Anomalies in Knowledge Bases 726 16.8 Diagnosis 727 16.8.1 General Diagnostic Engine 728 16.8.2 Symbolic Propagation 729 Contents xv

16.9 Automated Theorem Proving 730 16.9.1 Heuristics 732 16.9.2 Functional A* Search 733 16.10 Bibliographic Notes 734

CHAPTER 17 Vehicle Navigation 737 17.1 Components of Route Guidance Systems 737 17.1.1 Generation and Preprocessing of Digital Maps 737 17.1.2 Positioning Systems 739 17.1.3 Map Matching 741 17.1.4 Geocoding and Reverse Geocoding 744 17.1.5 User Interface 744 17.2 Routing Algorithms 744 17.2.1 Heuristics for Route Planning 745 17.2.2 Time-Dependent Routing 746 17.2.3 Stochastic Time-Dependent Routing 747 17.3 Cutting Corners 749 17.3.1 Geometric Container Pruning 749 17.3.2 Localizing A* 752 17.4 Bibliographic Notes 756

CHAPTER 18 Computational Biology 759 18.1 Biological Pathway 759 18.2 Multiple Sequence Alignment 760 18.2.1 Bounds 763 18.2.2 Iterative-Deepening Dynamic Programming 763 18.2.3 Main Loop 764 18.2.4 Sparse Representation of Solution Paths 767 18.2.5 Use of Improved Heuristics 769 18.3 Bibliographic Notes 771

CHAPTER 19 Robotics 773 19.1 Search Spaces 773 19.2 Search under Incomplete Knowledge 777 19.3 Fundamental Robot-Navigation Problems 778 19.4 Search Objective 780 19.5 Search Approaches 781 19.5.1 Optimal Offline Search 782 19.5.2 Greedy On line Search 784 xvi Contents

19.6 Greedy Localization 786 19.7 Greedy Mapping 787 19.8 Search with the Freespace Assumption 789 19.9 Bibliographic Notes 791 Bibliography 793

Index 825