Page iii Mastering Algorithms with Perl Jon Orwant, Jarkko Hietaniemi, and John Macdonald Page iv Mastering Algorithms with Perl by Jon Orwant, Jarkko Hietaniemi. and John Macdonald Copyright © 1999 O'Reilly & Associates, Inc. All rights reserved. Printed in the United States of America. Cover illustration by Lorrie LeJeune, Copyright © 1999 O'Reilly & Associates, Inc. Published by O'Reilly & Associates, Inc., 101 Morris Street, Sebastopol, CA 95472. Editors: Andy Oram and Jon Orwant Production Editor: Melanie Wang Printing History: August 1999: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O'Reilly logo are registered trademarks of O'Reilly & Associates, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O'Reilly & Associates, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. The association between the image of a wolf and the topic of Perl algorithms is a trademark of O'Reilly & Associates, Inc. While every precaution has been taken in the preparation of this book, the publisher assumes no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. ISBN: 1-56592-398-7 [1/00] [M]]break Page v Table of Contents Preface xi 1. Introduction 1 What Is an Algorithm? 1 Efficiency 8 Recurrent Themes in Algorithms 20 2. Basic Data Structures 24 Perl's Built-in Data Structures 25 Build Your Own Data Structure 26 A Simple Example 27 Perl Arrays: Many Data Structures in One 37 3. Advanced Data Structures 46 Linked Lists 47 Circular Linked Lists 60 Garbage Collection in Perl 62 Doubly-Linked Lists 65 Doubly-Linked Lists 65 Infinite Lists 71 The Cost of Traversal 72 Binary Trees 73 Heaps 91 Binary Heaps 92 Janus Heap 99 Page vi The Heaps Module 99 Future CPAN Modules 101 4. Sorting 102 An Introduction to Sorting 102 All Sorts of Sorts 119 Sorting Algorithms Summary 151 5. Searching 157 Hash Search and Other Non-Searches 158 Lookup Searches 159 Generative Searches 175 6. Sets 203 Venn Diagrams 204 Creating Sets 205 Set Union and Intersection 209 Set Differences 217 Set Differences 217 Counting Set Elements 222 Set Relations 223 The Set Modules of CPAN 227 Sets of Sets 233 Multivalued Sets 240 Sets Summary 242 7. Matrices 244 Creating Matrices 246 Manipulating Individual Elements 246 Finding the Dimensions of a Matrix 247 Displaying Matrices 247 Adding or Multiplying Constants 248 Transposing a Matrix 254 Multiplying Matrices 256 Extracting a Submatrix 259 Combining Matrices 260 Inverting a Matrix 261 Computing the Determinant 262 Gaussian Elimination 263 Eigenvalues and Eigenvectors 266 Page vii The Matrix Chain Product 269 The Matrix Chain Product 269 Delving Deeper 272 8. Graphs 273 Vertices and Edges 276 Derived Graphs 281 Graph Attributes 286 Graph Representation in Computers 287 Graph Traversal 301 Paths and Bridges 310 Graph Biology: Trees, Forests, DAGS, Ancestors, and Descendants 312 Edge and Graph Classes 316 CPAN Graph Modules 351 9. Strings 353 Perl Builtins 354 String-Matching Algorithms 357 Phonetic Algorithms 388 Stemming and Inflection 389 Parsing 394 Compression 411 10. Geometric Algorithms 425 Distance 426 Area, Perimeter, and Volume 429 Direction 433 Intersection 435 Intersection 435 Inclusion 443 Boundaries 449 Closest Pair of Points 457 Geometric Algorithms Summary 464 CPAN Graphics Modules 464 11. Number Systems 469 Integers and Reals 469 Strange Systems 480 Trigonometry 491 Significant Series 492 Page viii 12. Number Theory 499 Basic Number Theory 499 Prime Numbers 504 Unsolved Problems 522 13. Cryptography 526 Legal Issues 527 Authorizing People with Passwords 528 Authorization of Data: Checksums and More 533 Obscuring Data: Encryption 538 Hiding Data: Steganography 555 Winnowing and Chaffing 558 Winnowing and Chaffing 558 Encrypted Perl Code 562 Other Issues 564 14. Probability 566 Random Numbers 567 Events 569 Permutations and Combinations 571 Probability Distributions 574 Rolling Dice: Uniform Distributions 576 Loaded Dice and Candy Colors: Nonuniform Discrete Distributions 582 If the Blue Jays Score Six Runs: Conditional Probability 589 Flipping Coins over and Over: Infinite Discrete Distributions 590 How Much Snow? Continuous Distributions 591 Many More Distributions 592 15. Statistics 599 Statistical Measures 600 Significance Tests 608 Correlation 620 16. Numerical Analysis 626 Computing Derivatives and Integrals 627 Solving Equations 634 Interpolation, Extrapolation, and Curve Fitting 642 Page ix A. Further Reading 649 B. ASCII Character Set 652 Index 657 Page xi Preface Perl's popularity has soared in recent years. It owes its appeal first to its technical superiority: Perl's unparalleled portability, speed, and expressiveness have made it the language of choice for a million programmers worldwide. Those programmers have extended Perl in ways unimaginable with languages controlled by committees or companies. Of all languages, Perl has the largest base of free utilities, thanks to the Comprehensive Perl Archive Network (abbreviated CPAN; see http://www.perl.com/CPAN/). The modules and scripts you'll find there have made Perl the most popular language for web; text, and database programming. But Perl can do more than that. You can solve complex problems in Perl more quickly, and in fewer lines, than in any other language. This ease of use makes Perl an excellent tool for exploring algorithms. Computer science embraces complexity; the essence of programming is the clean dissection of a seemingly insurmountable problem into a series of simple, computable steps. Perl is ideal for tackling the tougher nuggets of computer science because its liberal syntax lets the programmer express his or her solution in the manner best suited to the task. (After all, Perl's motto is There's More Than One Way To Do It.) Algorithms are complex enough; we don't need a computer language making it any tougher. Most books about computer algorithms don't include working programs. They express their ideas in quasi-English pseudocode instead, which allows the discussion to focus on concepts without getting bogged down in implementation details. But sometimes the details are what matter—the inefficiencies of a bad implementation sometimes cancel the speedup that a good algorithm provides. The devil is in the details.break Page xii And while converting ideas to programs is often a good exercise, it's also just plain time-consuming. So, in this book we've supplied you with not just explanations, but implementations as well. If you read this book carefully, you'll learn more about both algorithms and Perl. About This Book This book is written for two kinds of people: those who want cut and paste solutions and those who want to hone their programming skills. You'll see how we solve some of the classic problems of computer science and why we solved them the way we did. Theory or Practice? Like the wolf featured on the cover, this book is sometimes fierce and sometimes playful. The fierce part is the computer science: we'll often talk like computer scientists talk and discuss problems that matter little to the practical Perl programmer. Other times, we'll playfully explain the problem and simply tell you about ready-made solutions you can find on the Internet (almost always on CPAN). Deciding when to be fierce and when to be playful hasn't been easy for us. For instance, every algorithms textbook has a chapter on all of the different ways to sort a collection of items. So do we, even though Perl provides its own sort() function that might be all you ever need. We do this for four reasons. First, we don't want you thinking you've Mastered Algorithms without understanding the algorithms covered in every college course on the subject. Second, the concepts, processes, and strategies underlying those algorithms will come in handy for more than just sorting. Third, it helps to know how Perl's sort() works under the hood, why its particular algorithm (quicksort) was used, and how to avoid some of the inefficiencies that even experienced Perl programmers fall prey to. Finally, sort() isn't always the best solution! Someday, you might need another of the techniques we provide. When it comes to the inevitable tradeoffs between theory and practice, programmers' tastes vary. We have chosen a middle course, swiftly pouncing from one to the other with feral abandon. If your tastes are exclusively theoretical or practical, we hope you'll still appreciate the balanced diet you'll find here. Organization of This Book The chapters in this book can be read in isolation; they typically don't require knowledge from previous chapters. However, we do recommend that you read at least Chapter 1, Introduction, and Chapter 2, Basic Data Structures, which provide the basic material necessary for understanding the rest of the book.break Page xiii Chapter 1 describes the basics of Perl and algorithms, with an emphasis on speed and general problem-solving techniques. Chapter 2 explains how to use Perl to create simple and very general representations, like queues and lists of lists. Chapter 3, Advanced Data Structures, shows how to build the classic computer science data structures. Chapter 4, Sorting, looks at techniques for ordering data and compares the advantages of each technique. Chapter 5, Searching, investigates ways to extract individual pieces of information from a larger collection. Chapter 6, Sets, discusses the basics of set theory and Perl implementations of set operations. Chapter 7, Matrices, examines techniques for manipulating large arrays of data and solving problems in linear algebra. Chapter 8, Graphs, describes tools for solving problems that are best represented as a graph: a collection of nodes connected by edges. Chapter 9, Strings, explains how to implement algorithms for searching, filtering, and parsing strings of text.