Analysis of Algorithms O Hm

DESIGN AND ANALYSIS OOF

ALGORITHMSHMPress

Associatesociateciate versProfProfessorP Department of Informationformatio Science and Technology Collegee off EngineeEngiEngineering, Guindy Campus Anna UniversityUnUniversity, Chennai

Oxford

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘

Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries.

Published in India by Oxford University Press YMCA Library Building, 1 Jai Singh Road, New Delhi 110001, India

The moral rights of the author/s have been asserted.

First published in 2014

All rights reserved. No part of this publication may be reproduced,ed, storedstore in a retrieval system, or transmitted, in any form or by any means,ns, withoutwithou the prior permission in writing of Oxford University Press, or as expresslypressly permittedpe by law, by licence, or under terms agreed with the appropriatepriateiate reproreprographics rights organization. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxfordo d UniversityUniversi Press, at the address above. Press You must not circulate this workk in any othother form and you must impose this same conditionndition on anyan acquirer.

ISBN-13: 978-0-19-809369-5-0-19-809369-519-80936 ISBN-10:: 0-19-809369-1-19-80936

Typesetet in Times New Roman by Idealall PublishingPublishin Solutions,S Delhi Printed in Indiandia by RadhaRad Press, New Delhi 110031

Third-party websiteebsiteite addressesa dressesdres mentioned in this book are provided by Oxford Universityversity PressP Universityss in good faith and for information only. Oxford University Press disclaimsdisclaim any responsibility for the material contained therein.

Oxford

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Features of the Book

CHAPTER Introducon to 1 Topical Coverage Algorithms CHAPTER The book provides extensive coverage of Basics of Computaonal ƚŽƉŝĐƐƐƚĂƌƟŶŐĨƌŽŵ 18 CHAPTER Complexity ĂůŐŽƌŝƚŚŵďĂƐŝĐƐĂŶĚ ĚĂƚĂƐƚƌƵĐƚƵƌĞƐ͕ŵŽǀŝŶŐ Data Structures—I 5 ŽŶƚŽĚŝīĞƌĞŶƚĂůŐŽƌŝƚŚŵ CHAPTER design techniques, “Bad programmers worry about the code. Good programmers worry about data structures followed by discussions Greedy Algorithms ŽŶĐŽŵƉƵƚĂƟŽŶĂů 11 ĐŽŵƉůĞǆŝƟĞƐĂŶĚWZD “Greed is all right,CHAPTER by the way...TER I think greed is healthy. You can be greedy ĂůŐŽƌŝƚŚŵƐ͘ d till f lss d b t lf ” Dynamic Programming 13 Press y

11.8.1 MŝŶŝŵƵŵ SƉĂŶŶŝŶŐ TƌĞĞƐ Treatment of Concepts The history of an MST is as interesting as its concept. In 1926, Otakerer Boruvka formulatedformulaformers ^ŝŵƉůĞĂŶĚůŽŐŝĐĂů the MST problem. A Polish mathematician, Vojtech Jarnik, describedbedd the problproblem in 1929 ĞǆƉůĂŶĂƟŽŶƐŽĨĐŽŶĐĞƉƚƐ in a letter to Otaker Boruvka. The same problem was conceiveded independentlyndependen by Kruskal in 1956. +ence, Kruskal rediscovered the problem. /ater it wasas de¿ned independently indepi by are provided using Robert Prim in 1957 and by Edsger Dijkstra in 1958 Example 11.1211.12 Consider the graph G shownsh wn in Fig.Fig 11.11. Construct an MST for the ĞƐƐĞŶƟĂůŵĂƚŚĞŵĂƟĐĂů given graph G using Kruskal’s algorithm. UniversityUn ĞǆƉƌĞƐƐŝŽŶƐ͕ŝůůƵƐƚƌĂƟŽŶƐ͕ 6ROXWLRQ The ¿rst step in Kruskal’s algorithmlgorithm is to sort all the edges and form an edge list, ĂŶĚŶƵŵĞƌŽƵƐƐŽůǀĞĚ say E. The edges of graph G shown in Fig. 11.211.21 are sorted and shown in Table 11.13. ĞǆĂŵƉůĞƐ͘ŝīĞƌĞŶƚ ŽŵƉůĞǆŝƚǇŶĂůǇƐŝƐŽĨ<ƌƵƐŬĂů͛ƐůŐŽƌŝƚŚŵŚŵ ƚǇƉĞƐŽĨƉƌŽďůĞŵƐĂůŽŶŐ Since the disjoint set data structure is used,ed, initialiinitializationordor takes at most (|V|) time. The time complexity of the algorithm dependsends on the numnumber of edges. As there are |E| edges, O(| E ǁŝƚŚƚŚĞŝƌĂůŐŽƌŝƚŚŵƐ͕ |log| E |) time is required to sortt these edgeedges. The disjoint set takes at most 2| E | ¿nd opera- ĞǆĂŵƉůĞƐ͕ĂŶĚĐŽŵƉůĞǆŝƚǇ tions and |V |í1 operations. Therefore,refore, ththe total complexity of Kruskal’s algorithm is at most O(| E |log| E |) time. OxforO ĂŶĂůǇƐĞƐĂƌĞŐŝǀĞŶ͘

AlgoritŚm PresentaƟon ůŐŽƌŝƚŚŵƐĂƌĞƉƌĞƐĞŶƚĞĚ Step 1: Create a node x by allocating memory for it. in two ways, that is, step- Step 2: Assign the required value to the item part of node x. wise approach;ŝŶĨŽƌŵĂů item(x) = value Step 3: Set in the pointer to null. ƌĞƉƌĞƐĞŶƚĂƟŽŶͿĂŶĚ next(x) = null pseudocode approach Step 4: ReturnAlgorithm the node x . create(L, x, value) ;ĨŽƌŵĂůƌĞƉƌĞƐĞŶƚĂƟŽŶͿ %% Input: List L and element x with 'value' ĨŽƌƉƌŽǀŝĚŝŶŐĂďĞƩĞƌ %% Output: Node x Begin understanding of the allot(x) %% Allot memory for node x with two ºeldsŹitem and next item(x) = value logic behind solving a next(x) = null ŐŝǀĞŶƉƌŽďůĞŵǁŝƚŚŽƵƚ ƵƐŝŶŐĂŶǇƐƉĞĐŝĮĐ ƉƌŽŐƌĂŵŵŝŶŐůĂŶŐƵĂŐĞ͘

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Historical Notes Box 1.1 Origin of the word ‘algorithm’ The word algorithm is derived from the name of a Persian Europe. He also introduced the simple step-b Historical notes are mathematician, Abu Ja’fer Mohammed Ibn Musa al for addition, subtraction, multiplication, and provided throughout the Khowarizmi, who lived sometime around 780–850 AD. He his book. The word algebra has also been d was from the town of Khowarazm, now in Uzbekistan. He the title of this book. When his book was t ďŽŽŬĂƐƐĞƉĂƌĂƚĞďŽǆĞƐ͘ was a teacher of mathematicsBox 17.2 inGeorge Baghdad. Bernard He wrote Dantzig a book Latin, his name was quoted as Algorismus dŚĞǇƉƌŽǀŝĚĞĂĚĚŝƟŽŶĂů *eorge Bernard Dantzig was born in 1914 at Portland, duality theory. He worked with Fulkerson an Oregon, United States. His father became a professor of in formulating the travelling salesperson pro ŝŶĨŽƌŵĂƟŽŶƌĞůĂƚĞĚƚŽ mathematics at University of Maryland after World War linear programming and solved the TSP proble II. Dantzig’s biggest contribution is that he designed the 49 cities at that time. In 197, he was award ƚŚĞĐŽŶĐĞƉƚƐĚŝƐĐƵƐƐĞĚ͘ simplex method for solving LPPs. Apart from the simplex National Medal of Science, the highest honou

Glossary and Summary Q GLOSSARY Q ŐůŽƐƐĂƌǇŽĨŬĞǇƚĞƌŵƐ Agent A performer of an algorithm $OJRULWKPYHUL¿FDWLRQ The process of provid ĂůŽĂůŽŶŐǁŝƚŚĚĞĮŶŝƟŽŶƐĂŶĚ Algorist A person who is skilled in algorithm development cal proof that the algorithm works correctly f Algorithm A step-by-step procedure for solving a given Algorithm validation The process of chec ĂƉŽŝĂƉŽŝŶƚͲǁŝƐĞƐƵŵŵĂƌǇŝƐ problem ness of an algorithm, this is done by givin givgiven at the end of each Algorithm gap The difference between lower and upper it and checking its results with expected Q SUMMARY Q esesschapter to help readers An algorithm is a step-by-step procedure for solving a Algorithm veri¿cation is a processes of proprovv ƌĞĐĂƉŝƚƵůĂƚĞƚŚĞŝŵƉŽƌƚĂŶƚ given problem. ematical proof that the givenn algorithmalgorithorit wwoo ĐŽŶĐĞƉƚƐĞǆƉůĂŝŶĞĚ͘ A computational problem is characterized by two for all instances of data. PresP factors²speci¿cation of valid input and output param- A proof of an algorithmm is said to existexis if t ity

Revieǁ YuesƟons͕ iversityiv Q REVIEW QUESTIONS Q Eǆercises͕ and 1.1 De¿ne ann algorithm. 1.8 What is the difference between algo 1.2 Whatat are the characteristicschar of an algorithm" and algorithm validation" AddiƟonal Proďlems 1.3 S Surveyy the InteInternet and list out at least ¿ve algorithms 1.9 How is an algorithm validated and EƵŵĞƌŽƵƐƌĞǀŝĞǁ that have Univhugeh impact on our daily lives. with an example. 1.44 What area the stages of problem solving" 1.10 State algorithm classi¿cations. Wha ƋƵĞƐƟŽŶƐ͕ĞǆĞƌĐŝƐĞƐ͕ĂŶĚ Q EXERCISES Q 1.1 Assume that there are two algorithms A and B for a complexities of A, B, and C are 3n ĂĚĚŝƟŽŶĂůĐŽŶĐĞƉƚͲďĂƐĞĚ rd given problem P. The time complexity functions of algo- respectively. Assume that the input i ƉƌŽďůĞŵƐĂƌĞŐŝǀĞŶĂƚ rithms A and B are, respectively, 5n and log2n. Which Assume that the machine executes algorithm should be selected assuming that all other per second. How much time will algo the end of every chapterpter f conditions remain the same for both the problems" C take" Which algorithm will be the Q ADDITIONAL PROBLEM Q to test the readers’er 12 L h f l h di h 1.1 John MacCormick had written a book titled Nine recognition, data compression, datab conceptual knowledgegeOxfordOx Algorithms That Changed the Future: The Ingenious signatures, that changed the world. Ideas that drive Today’s Computers, Princeton University (a) What are these algorithms" Se and also enhance their Press, Princeton, that had listed the nine wonderful and ¿nd what these algorithms ĂůŐŽƌŝƚŚŵǁƌŝƟŶŐƐŬŝůůƐ͘ algorithms, namely, search engine indexing, page rank, (b) Identify one more algorithm that

Q CROSSWORD Q 1 Crossword Puzzles 2 Crossword puzzles, 3 45 ŐŝǀĞŶĂƐĂŶĞǆĐŝƟŶŐĂŶĚ ŝŶƚĞƌĂĐƟǀĞůĞĂƌŶŝŶŐ exercise at the end of each 7 8 ĐŚĂƉƚĞƌ͕ŵŽƟǀĂƚĞĂŶĚĂŝĚ 910 readers to self-check their ƵŶĚĞƌƐƚĂŶĚŝŶŐ͘ 11

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ WƌĞĨĂĐĞ

Computers have become an integral part of our daily life in recent times. They have enormously impacted our personal, professional, as well as social lives. Computers help us in tasks such as document editing, Internet browsing, sending emails, making presentations, performing complex scienti¿c computations, social networking, and playing games. Industries and government of¿ces use computers effectively in production, e-governance, and e-commerce. Considering the increasing demand of computers in society, schools, colleges, and universities have included computer education in their curriculum, to help students become skilled in programming and developing applications which can be used to solve various business, scienti¿c, and social problems. Programming is a process of converting a given problem into an executableutableable cocode for the computer. It involves understanding, analysis, and solving problems to create an algorithm.rithm.hm. Veri¿cation of the algorithm, coding of the algorithm in a speci¿c programming language, testing,ing, debugdebudebugging, and maintaining the source code are also part of the programming process. Therefore, inn order to construct ef¿cient programs, a ¿ne understanding of algorithms is essential. An algorithm is a ssetet ofoPress llogical instructions for solving a given problem. It is expected to give correct results for validalid inputsinput aand should be ef¿cient, consuming less computer resources. An algorithm is implemented ass a program using a programming language. A well-designed algorithm runs faster and consumes lessersser comcompucomputer resources, namely time and space. Therefore, expertise in programming is more relatedatedd to ef¿ciecciency in problem-solving and effective designing of algorithms, rather than developing codes with ththe help of programming languages. Though programming languages are important, theirr roleole is jujust lilimited to the implementation of a well-designed algorithm. For this reason, algorithms aree a central ththeme of computer study. In fact, history of algorithms is muchch older thathan that of computers, dating back to 3000 BC. The ancient people of the Sumerian civilizationvilizatlizat onUniversity wwerwere aware of basic numeric computation like addition. A Sumerian tablet found in the Euphratesuphratesrates rivriver showed how to partition a given quantity of wheat in a way that each person receives thehe speci¿ede quantity. Such tablets were also used by ancient Babylonians (2000 to 1650 BC). Mathematiciansematiciansaticians oof this period such as Euclid, Al-Khwarizmi, Leonardo Pisano (also known as Fibonacci), andnd others producedp procedures that provided the foundations of the concepts of algorithms. Later developmentsvelopmentelopmen in the ¿eld of algorithms were due to the contributions of Gottfried Leibnitz, David Hilbert,rt, and Alan Turing. The history of modern computers, however, only starts from the 1940s. Thus, algorithmsOxford have played a very important role in the development of modern computing. The study of algorithms, called algorithmics, includes three aspects—algorithm design, analysis, and computational complexity of problems. Algorithm design is a creative activity. It includes various techniques (such as divide-and-conquer, greedy approach, dynamic programming, backtracking) that help in producing outputs at a faster pace by consuming lesser computer resources. Algorithm analysis is the estimation of how much resource is required by the algorithm. Computational complexity deals with the analysis and solvability of problems itself. A compulsory course on algorithms design and analysis is generally offered to computer science and information technology students in most universities. The course aims to help students create ef¿cient algorithms in common engineering design situations and analyse the asymptotic performance of algorithms by using the important algorithmic design paradigms and methods of analysis.

ABOUT THE BOOK Design and Analysis of Algorithms is designed to serve as a textbook for the ¿rst level course in algorithms that discusses all the fundamental and necessary information related to the three important aspects of

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Preface vii algorithm study using minimal mathematics and lucid language. This book is suitable for undergradu- ate students of computer science and engineering (CSE) and information technology (IT), as well as for postgraduate students of computer applications. It is also useful for diploma courses, competitive examinations (like GATE), and recruitment interviews for this subject. The book begins with an introduction to algorithms and problem-solving concepts followed by an introduction to algorithm writing and analysis of iterative and recursive algorithms. In-depth explanations and designing techniques of various types of algorithms used for problem-solving such as the brute force technique, divide-and conquer-technique, decrease-and-conquer strategy, greedy approach, transform-and-conquer strategy, dynamic programming, branch-and-bound approach, and backtracking, are provided in the book. Subsequent chapters of the book delve into the discussion of string algorithms, iterative improvement, linear programming, computability theory, NP-hard problems, NP- completeness, probability analysis, randomized algorithms, approximation algorithms, and parallel algorithms, with the appendices throwing light on basic mathematics and proof techniques. The various design techniques have been elucidated with the help off numernumeronumerous problems, solved examples, and illustrations (including schematics, tables, and cartoons).. The algoralgorithms are presented in plain English (informal algorithm presentation) and pseudocode approachproachoach (for(formal algorithm presentation) to make the book programming language-independent and easy-to-comprehend.sy-to-com The book includes a variety of chapter-end pedagogical features such as point-wiseisee ssummary,ummaumm glossary, review questions, exercises, and additional problems to help readers assimilatetee and imimpimplementPressl the concepts learnt.

KEY FEATURES Provides simple and coherent explanations withoutithoutout usingusin eexcessive theorems, proofs, and lemmas Detailed coverage for topics such as greedyy approach, dydynamic programming, transform-and-conquer technique, decrease-and-conquer technique,que, linearlinea prprogramming, and randomized and approximation algorithms Dedicated chapters on backtrackingingng andnd brancbrabranch-and-bound techniques, string matching algorithms, and parallel algorithms University Simple and judicious presentationentationation of algorithms throughout the text in both informal and formal forms, followed by the discussionscussion oof their complexity analysis Numerous review questions,estions,ns, exeexercises, and additional problems given at the end of each chapter to help readers apply and practispractise the concepts learnt Includes glossary and poipoint-wise summary at the end of each chapter to help readers quickly recapitulate the importantrtanOxford concepts Provides historical notes on various topics and crossword puzzles at the end of each chapter to elicit learning interest in students

ORGANIZATION OF THE BOOK The book consists of twenty chapters. A chapter-wise scheme of the book is presented here. Chapter 1 provides an overview of algorithms. It introduces all the basic concepts of algorithms and the fundamental stages of problem-solving. The chapter ends with the classi¿cation of algorithms. Chapter 2 starts with the basic tools used for problem-solving. All the guidelines required for present- ing the pseudocode and Àow charts are provided along with many examples. The focus of this chapter is to provide some practice on writing algorithms. The basics of recursion and algorithm correctness are also covered in this chapter. Chapter 3 covers the basics of algorithm complexity and analysis of iterative algorithms. Step count and operation count methods used for analysing iterative algorithms are discussed in detail. Asymptotic

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ viii Preface analysis is also discussed in detail. Finally, the chapter ends with the concept of analysing the ef¿ciency of algorithms. Chapter 4 discusses the analysis of recursive algorithms. It explains the basics of recurrence equations along with the methods for solving them. Generating functions are also brieÀy discussed as part of this chapter. Data structures are an important component of algorithms. Chapter 5 deals with the fundamentals concepts related to data structures such as stacks, queues, linear lists, and linked lists. Trees and graphs are also explained in this chapter. Chapter 6 covers advanced data structures used for organizing large amounts of dynamic data. Binary search trees and AVL trees used for organizing dictionaries are discussed in this chapter, in addition to priority queues and heaps. Finally, a discussion on disjoint sets and amortized analysis is provided. Chapter 7 deals with the brute force techniques, which use no special logic but instead follow an intuitive way of solving problems using the problem statement. Various problems such as sequential search, bubble sort, and selection sort that can be solved using this approachh area explained. Some basic computational geometry problems such as closest-pair and convex hull aree also covered. The chapter ends with the discussions on exhaustive searching problems such as 15-puzzle5-puzzlepuzzle pproblem, 8-queen problem, magic square problem, knapsack problem, container loading problem,blem, anand assignment problem. Chapter 8 discusses the divide-and-conquer design paradigm. Important problems such as quicksort, merge sort, ¿nding maximum and minimum, multiplication off lonlongong integintintegers, Strassen matrix multiplication, tiling problem, closest-pair problem, and convex hull are solvesolvsolveddPress using this technique. The chapter ends with a discussion of the Fourier transform problem.m. Chapter 9 explains the decrease-and-conquer technique,hnique,ique, wwhich is also known as the incremental or inductive approach. Examples problems such as insertionnsertionrtion sort,sor ttopological sort, generating permutations and subsets, binary search, fake coin detection, andd RussianRussi peasant multiplication problem are used to illustrate the decrease by constant and constantnt factor mmethomethods. Finally, discussions on interpolation search, selection, and ¿nding the median problemsms using theth ddecrease by variable factor method are provided. Chapter 10 deals with time±space tradeoffs.radeoffs. TThThe selection of one type of ef¿ciency over the other and problems related to linear sortingngg aandd HashinHasHashing are discussed. The chapter ends with a discussion on B-trees and their operations. University Chapter 11 explains the greedyeedydy approaapproach concept. This chapter discusses important problems such as coin change problem, scheduling,duling,uling, knaknapsack problem, optimal storage of tapes, Huffman code, minimum spanning tree algorithms,ms, and DijkDijkstra’s shortest path algorithm to illustrate the greedy approach. Chapter 12 discussesssesses the trantratransform-and-conquer approach and its three basic techniques—instance simpli¿cation, representationentationentatio change, and problem reduction. Problems such as Gaussian elimination, decomposition methods,Oxford ¿nding determinant and matrix inverses are discussed. Heap sort, Horner’s method, binary exponentiation algorithm, and reduction problems are also covered in this chapter. Chapter 13 deals with dynamic programming. Important example problems such as Fibonacci problem, binomial coef¿cient multistage graph, Graph algorithms, Floyd-Warshall Algorithm, Bellman±Ford algorithms, travelling salesman problem, chain matrix multiplication, knapsack problem, and optimal binary search tree problem are discussed to illustrate the dynamic programming concept. Finally, the chapter concludes with the Àow-shop scheduling algorithms. Chapter 14 discusses backtracking algorithms. This chapter covers important problems such as N-queen problem, Hamiltonian circuit problem, sum of subsets, vertex colouring problem, graph colouring problems, Graham scan, and generating permutations. Chapter 15 explains branch-and-bound techniques. Search techniques using this concept are discussed. Important problems such as assignment problem and 15-puzzle are covered and the chapter ends with a discussion on traveling salesperson and knapsack problems. Chapter 16 deals with the string algorithms. Some basic string algorithms such as ¿nding the length of strings, ¿nding substrings, concatenation of two strings, longest common sequence, and pattern

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Preface ix recognition algorithms such as Rabin±Karp, Harspool, Knuth±Morris±Pratt and Boyer±Moore algorithms are discussed in this chapter. Finally, approximate string matching algorithm is discussed. Chapter 17 discusses the iterative approach and basics of linear programming. The linear programming formulation of a problem and simplex method is discussed in this chapter. Minimization problem, principle of duality, and max-Àow problems are also explained. Finally, matching algorithms are considered for better understanding of computational complexity. Chapter 18 explains the basics of computational complexity and the upper and lower bound theory. Decision problems, complexity classes, and reduction concepts are also discussed. This chapter also covers theory of NP-complete problems and examples for proving NP-completeness. Chapter 19 covers the basic concepts and types of both randomized and approximation algorithms. Randomized algorithms are illustrated through examples such as hiring problem, primality testing, comparison of strings, and randomized quicksort. Approximation algorithms are illustrated through examples based on heuristic, greedy, linear, and dynamic programming approaches. Chapter 20 begins with an introduction to parallel processing and classissi¿ccation of parallel systems. It then discusses the fundamentals of parallel algorithms and parallelrallel ranrandom access machine (PRAM) model. The concept of parallelism is illustrated through examplesxamplesmples rerelated to parallel searching, parallel sorting, and graph and matrix multiplication problems.lems. There are two appendices in this book. Appendix A explainsns ththee basicbabasics of mathematics such as sets, series and sequences, relations, functions, matrix algebra, andd probabilprobabprobabilityiPress that are necessary for algorithm study. Appendix B deals with mathematical logic and proofooff techniqutechniquetechniques.

ONLINE RESOURCES To aid teachers and students, the book is accompaniedompaniedmpanied by online resources that are available at http:// oupinheonline.com/book/sridhar-Design-Analysis-Algorithms/9780198093695.nalysis-Algorinalysis-Alg The content for the online resources are as follows: For Instructors University PowerPoint slides Solutions manual For Students Answers to the crosswordsswordswordOxford ppuzzles ACKNOWLEDGEMENTS This book would not have been possible without the help and encouragement of many friends, colleagues, and well-wishers. I thank all my students who motivated me by asking interesting questions related to the subject. I acknowledge the assistance of my friend, P. Kanniappan, in typing the manuscripts and for all his advice. I also express my gratitude to all my colleagues at the departments of Computer Science and Engineering and Information Science and Technology, Anna 8niversity and National Institute of Technology, Tiruchirapalli for reviewing my manuscripts and providing constructive suggestions for improvement. I acknowledge all my reviewers whose feedback has helped making this book better. I am also very thankful to the editorial team at Oxford 8niversity Press, India for providing valuable assistance. My sincere thanks are also due to my family members, especially my wife, Dr N. Vasanthy, my mother, Mrs Parameswari, my mother-in-law, Mrs Renuga, and my children, Shobika and Shreevarshika, for their constant support and encouragement during the development of this book. S. Sridhar

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Brief Contents

Features of the Book iv Preface vi Detailed Contents xi

1 Introduction to Algorithms 1 2 Basics of Algorithm Writing 22 3 Basics of Algorithm Analysis 58 4 Mathematical Analysis of Recursive Algorithms 98 5 Data Structures—I 141 6 Data Structures—II 194 7 Brute Force Approaches Press 231 8 Divide-and-conquer Approach 262 9 Decrease-and-conquer Approach 309 10 Time–Space Tradeoffs 342 11 Greedy Algorithms 372 12 Transform-and-conquer Approach 417 13 Dynamic Programming University 455 14 Backtracking 517 15 Branch-and-bound Techniquechniqu 543 16 String Algorithmsmss 569 17 Iterative ImprovementOxford and Linear Programming 601 18 Basics of Computational Complexity 638 19 Randomized and Approximation Algorithms 663 20 Parallel Algorithms 705

Appendix A—Mathematical Basics 735 Appendix B—Proof Techniques 752 Bibliography 763 Index 766

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Detailed Contents

Features of the Book iv Preface vi Brief Contents x

1 IntroducƟon to Algorithms 1 2.3 Non-recursive Algorithms 31 1.1 Introduction 1 2.3.1 Flowcharts 32 1.2 Need for Algorithmic Thinking 1 2.4 Basics of Recursion 42 1.3 Overview of Algorithms 3 2.5 Recursive Algorithms 43 1.3.1 Computational Problems, 2.6 Algorithm Correctnessrre 52 Instance, and Size 5 1.4 Need for Algorithm Ef¿ciency 7 3 Basics of Algorithmgorithmrithm AAnAnalysis 58 1.5 Fundamental Stages of Problem 3.1 Basicscs of AlgorAlgorithm Complexity 58 Solving 9 3.2 Introductionroductionroductio to Time complexity 60 1.5.1 Understanding the Problem 9 3.2.13.2.2 RanRRandomPress Access Machine 60 1.5.2 Planning an Algorithm 9 3.33 AnalysAnal Analysis of Iterative Algorithms 62 1.5.3 Designing an Algorithm 11 3.3.1 Measuring Input Size 63 1.5.4 Validating and Verifying an 3.33.3.2 Measuring Running Time 63 Algorithm 11 3.3.3 Best-, Worst-, and 1.5.5 Analysing an Algorithm 12 Average-case Complexity 71 1.5.6 Implementing an Algorithm 3.4 Rate of Growth 73 and Performing Empiricall 3.4.1 Measuring Larger Inputs 73 Analysis 14 3.4.2 Comparison Framework 74 1.5.7 Post (or Postmortem)em) University3.5 Asymptotic Analysis 76 Analysis 14 3.5.1 Asymptotic Notations 78 1.6 Classi¿cation of Algorithmsgorithms 15 3.5.2 Asymptotic Rules 85 1.6.1 Based onn ImplementImplemenImplementation 15 3.5.3 Asymptotic Complexity Classes 87 1.6.2 Based onn Design 16 3.6 Space Complexity Analysis 88 1.6.3 Based on AreaOxford of 3.7 Empirical Analysis and Algorithm Specialization 17 Visualization 88 1.6.4 Based on Tractability 18 3.7.1 Experimental Purpose 89 3.7.2 Statistical Tests 90 2 Basics of Algorithm WriƟng 22 2.1 Tools for Problem-solving 22 ϰ MathemaƟcal Analysis of Recursive 2.1.1 Stepwise 5e¿nement or Algorithms 98 Top-down design 23 4.1 Introduction to Recurrence 2.1.2 Bottom-up Approach 24 Equations 98 2.1.3 Structured Programming 25 4.1.1 Linear Recurrences 99 2.2 Algorithm Speci¿cations 26 4.1.2 Non-linear Recurrences 101 2.2.1 Guidelines for Writing 4.2 Formulation of Recurrence Algorithms 27 Equations 103

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ xii Detailed Contents

4.3 Techniques for Solving 6 Data Structures—II 194 Recurrence Equations 106 6.1 Introduction to Dictionary 194 4.3.1 Guess-and-verify Method 106 6.1.1 Introduction to Binary 4.3.2 Substitution Method 109 Search Tree 194 4.3.3 Recurrence-tree Method 112 6.1.2 AVL Trees 200 4.3.4 Difference Method 117 6.2 Priority Queues and Heaps 206 4.4 Solving Recurrence Equations 6.2.1 Binary Heaps 207 8sing Polynomial Reduction 118 6.2.2 Binomial Heaps 212 4.4.1 Solving Homogeneous 6.2.3 Fibonacci heap 218 Equations 118 6.3 Disjoint Sets 221 4.4.2 Solving Non-homogeneous 6.3.1 Representation and Equations 122 Operationsion 221 4.5 Generating Functions 123 6.4 Amortizedd AnalysAnalysis 224 4.5.1 Properties of Generating 6.4.1 Aggregategregate MMethod 225 Functions 124 6.4.22 AccountiAccountAccounting Method 226 4.6 Divide-and-conquer Recurrences 127 6.4.34.3 PotenPotePotential Method 226 4.6.1 Master Theorem 127 Press 4.6.2 Transformations 133 7 Brute ForcFoForce Approaches 231 4.6.3 Conditional Asymptotics 135 7.1 IntroIntroduction 231 7.1.17 Advantages and 5 Data Structures—I 1411 Disadvantages of Brute 5.1 Data Structures and Algorithms 141 Force Method 232 5.2 Lists 142 7.2 Sequential Search 232 5.2.1 Linear Lists and Arrays 14314 7.2.1 Analysis of Recursion 5.2.2 Linked Lists 14 146 Programs 234 5.3 Stacks University 152 7.2.2 Recursive Form of Linear 5.3.1 Representationn of and Search Algorithm 235 Operationss onn Stacks 152 7.3 Sorting Problem 236 5.4 Queues 154 7.3.1 Classi¿cation of Sorting 5.4.1 Queuee RepreseRepresenRepresentation 154 Algorithms 237 5.5 Trees 157 7.3.2 Properties of Sorting 5.5.1 Tree Terminologies 157 Algorithms 237 5.5.2 Classi¿cation of Trees 159 7.3.3 Bubble Sort 238 5.5.3 Binary Tree Representation 161 7.3.4 Selection Sort 242 5.5.4 Binary Tree Operations 163 7.4 Computational Geometry Problems 245 5.6 Graphs 167 7.4.1 Closest-pair Problem 245 5.6.1 Terminologies and 7.4.2 Convex Hull Problem 247 Types of Graphs 168 7.5 Exhaustive Searching 249 5.6.2 Graph Representation 172 7.5.1 15-puzzle Problem 250 5.6.3 Graph Traversal 174 7.5.2 8-queen Problem 251 5.6.4 Elementary Graph Algorithms 178 7.5.3 Magic Squares 252 5.6.5 Spanning Tree and Minimum-cost 7.5.4 Container Loading Problem 253 Spanning Tree 186 7.5.6 Assignment Problem 256

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Detailed Contents xiii

8 Divide-and-conquer Approach 262 9.4.1 Interpolation Search 333 8.1 Introduction 262 9.4.2 Selection and Ordered 8.1.1 Recurrence Equation for Statistics 334 Divide and Conquer 263 9.4.3 Finding Median 336 8.1.2 Advantages and Disadvantages of Divide-and-conquer 10 TimeʹSpace Tradeoīs 342 Paradigm 264 10.1 Introduction to Time–Space 8.2 Merge Sort 264 Tradeoffs 342 8.3 Quicksort 269 10.2 Linear Sorting 342 8.3.1 Partitioning Algorithms 270 10.2.1 Counting Sort 342 8.3.2 Variants of Quicksort 275 10.2.2 Bucket Sort 347 8.4 Finding Maximum and Minimum 10.2.3 Radix SoSort 349 Elements 277 10.3 Hashing andd Hash Tables 351 8.5 Multiplication of Long Integers 280 10.3.1 Propertiesroperties of Hash 8.6 Strassen Matrix Multiplication 284 FunctionsFunctioFuncti 353 8.7 Tiling Problem 288 10.3.20.3.2 HashHa Table Operations 354 8.8 Closest-pair Problem 291 10.310.3.30 3Press CollisionC 356 8.8.1 Using Divide-and-conquer 10.4 B-treeB-trB-trees 359 Method 291 10.4.110.410 4 B-tree Balancing 8.9 Convex Hull 293 Operations 361 8.9.1 Quickhull 2933 10.4.2 B-tree Operations 363 8.9.2 Merge Hull 295 95 8.10 Fourier Transform 296 111 Greedy Algorithms 372 8.10.1 Polynomial Multiplicationionon 29629 11.1 Introduction to Greedy Approach 372 8.10.2 Application of Fourierrierer University11.1.1 Components of Greedy Transform 2988 Algorithms 373 8.10.3 Fast Fourierr TransformTransfor 302 11.2 Suitability of Greedy Approach 374 11.3 Coin Change Problem 375 9 Decrease-and-conquernquer AppApproach 309 11.4 Scheduling Problems 376 9.1 Introduction 309 11.4.1 Scheduling without 9.2 Decrease by ConstantonstOxford Method 311 Deadline 377 9.2.1 Insertion Sort 311 11.4.2 Scheduling with Deadline 379 9.2.2 Topological Sort 315 11.4.3 Activity Selection Problem 382 9.2.3 Generating Permutations 321 11.5 Knapsack Problem 384 9.2.4 Generating Subsets 323 11.6 Optimal Storage of Tapes 388 9.3 Decrease by Constant Factor 11.7 Optimal Tree Problems 390 Method 325 11.7.1 Optimal Merge 390 9.3.1 Binary Search 326 11.7.2 Huffman Coding 393 9.3.2 Fake Coin Detection 330 11.7.3 Tree Vertex Splitting Problem 398 9.3.3 Russian Peasant 11.8 Optimal Graph Problems 401 Multiplication Problem 331 11.8.1 Minimum Spanning Trees 401 9.4 Decrease by Variable 11.8.2 Single-source Shortest-path Factor Method 332 Problems 407

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ xiv Detailed Contents

12 Transform-and-conquer Approach 417 13.6 Floyd–Warshall All Pairs 12.1 Introduction to Transform and Shortest-path Algorithm 478 Conquer 417 13.6.1 Shortest-path 12.2 Introduction to Instance Reconstruction 481 Simpli¿cation 418 13.7 Bellman–Ford Algorithm 481 12.3 Matrix Operations 419 13.8 Travelling Salesperson Problem 486 12.3.1 Gaussian Elimination 13.9 Chain Matrix Multiplication 488 Method 420 13.9.1 Dynamic Programming 12.3.2 LU Decomposition 427 Approach for Solving Chain 12.3.3 Crout’s Method of Matrix Multiplication Decomposition 434 Problem 490 12.3.4 Finding Matrix Inverse 436 13.10 Knapsack ProblemPr 496 12.3.5 Finding Matrix 13.11 Optimall Binary Search Trees 500 Determinant 439 13.11.11 Brute FForce Approach 12.4 Change of Representation 441 for ConstructingC Optimal 12.4.1 Heap Sort 441 BSTsB 500 12.4.2 Polynomial Evaluation 13.11.213.11.23.11Press Dynamic Programming Using Horner’s Method 445 Approach for Constructing 12.4.3 Binary Exponentiation 447 Optimal BSTs 502 12.5 Problem Reduction 450 13.1213.1 Flow-shop Scheduling Problem 507 13.12.1 Single-machine 13 Dynamic Programming 455 Sequencing Problem 508 13.1 Basics of Dynamic Programmingg 455 13.12.2 Two-machine Sequencing 13.1.1 Components of Dynamicc Problem 509 Programming 456 13.1.2 Characteristics of DynamicDynamynam University 14 Backtracking 517 Programmingg 458 14.1 Introduction 517 13.2 Fibonacci Problememm 460 14.2 Basics of Backtracking 518 13.3 Computing Binomialomial CoCoef¿cients 463 14.3 N-queen Problem 522 13.4 Multistage Graph ProblemPro 466 14.3.1 State Space of 4-queen 13.4.1 Forwardard ComputationCo Problem 523 ProcedureOxford 467 14.4 Sum of Subsets 525 13.4.2 Backward Computation 14.5 Vertex Colouring Problem 527 Procedure 471 14.6 Hamiltonian Circuit Problem 531 13.5 Transitive Closure and Warshall 14.6.1 Promising (or Bounding Algorithm 472 Function) for Hamiltonian 13.5.1 Finding Transitive Closure Using Problem 531 Brute Force Approach 473 14.7 Generating Permutation 534 13.5.2 Finding Transitive Closure Using 14.8 Graham Scan 536 Warshall Algorithm 473 15 Branch-and-ďound Technique 543 13.5.3 Alternative Method to 15.1 Introduction 543 Warshall Algorithm for 15.2 Search Techniques for Branch- Finding Transitive Closure 476 and-bound Technique 545

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Detailed Contents xv

15.2.1 BFS using Branch-and- 17.8 Max-Àow Problem 623 bound algorithm— 17.9 Bipartite Matching Problem 628 FIFOBB 545 17.10 Stable Marriage Problem 630 15.2.2 LIFO with Branch and Bound 547 18 Basics of ComputaƟonal 15.2.3 Least Cost with Branch Complexity 638 and Bound 547 18.1 Introduction to Computational 15.3 15-puzzle Game 548 Complexity 638 15.4 Assignment Problem 552 18.2 Algorithm Complexity, 15.5 Traveling Salesperson Problem 559 8pper and Lower Bound Theory 639 15.6 Knapsack Problem 562 18.2.1 Upper and Lower Bounds 640 18.3 Decision Problemsobleble and Turing 16 String Algorithms 569 Machine 644 16.1 Introduction to String 18.4 Complexityplexityxity ClassesCla 647 Processing 569 18.4.14.1 Class P 648 16.2 Basic String Algorithms 571 118.4.2.4.2 NP Class 648 16.2.1 Length of Strings 571 18.5.5 TheoTh TheoryryPress of NP-complete 16.2.2 Concatenation of ProblemsProbleProb 650 Two Strings 571 18.6 RedReductions 651 16.2.3 Finding Substrings 572 18.6.1 Turing Reduction 651 16.3 Longest Common Subsequences 573 18.6.2 Karp Reduction 652 16.4 naïve String Matching 18.7 Satis¿ability Problem and Algorithm 576 Cook’s Theorem 654 16.5 Pattern Matching 8sing Finitee 18.8 Example Problems for Proving Automata University5785 NP-completeness 655 16.6 Rabin–Karp Algorithmm 5805 18.8.1 SAT is NP-complete 655 16.7 Knuth–Morris–Pratttt Algorithm 584 18.8.2 Problem 3-CNF-SAT is 16.8 Harspool Algorithmthmm 588 NP-complete 656 16.9 Boyer–Mooree String MatchingMaM 18.8.3 Clique Decision Problem Algorithm 590 is NP-complete 657 16.10 ApproximateeeStri StringStriOxford Matching 594 18.8.4 Sum of Subsets (from 3-CNF-SAT) 658 17 IteraƟve Improvement and Linear Programming 601 19 Randomized and ApproximaƟon 17.1 Introduction to Iterative Algorithms 663 Improvement 601 19.1 Dealing with NP-hard 17.2 Linear Programming 601 Problems 663 17.3 Formulation of LPPs 603 19.2 Introduction to Randomized 17.4 Graphical Method for Algorithms 664 Solving LPPs 606 19.2.1 Generation of Random 17.5 Simplex Method 611 Numbers 666 17.6 Minimization Problems 615 19.2.2 Types of Randomized 17.7 Principle of Duality 619 Algorithms 669

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ xvi Detailed Contents

19.3 Examples of Randomized 20.2 Classi¿cation of Parallel Systems 705 Algorithms 669 20.2.1 Flynn Classi¿cation 706 19.3.1 Hiring Problem 669 20.2.2 Address-space (or Memory 19.3.2 Primality Testing Algorithm 672 Mechanism-based) 19.3.3 Comparing Strings Using Classi¿cation 708 Randomization Algorithm 674 20.2.3 Classi¿cation Based on 19.3.4 Randomized Quicksort 675 Interconnection Networks 710 19.4 Introduction to Approximation 20.3 Introduction to PRAM Model 711 Algorithms 678 20.4 Parallel Algorithm Speci¿cations 19.5 Types of Approximation and Analysis 712 Algorithms 679 20.4.1 Parallel Algorithm 19.6 Examples of Approximation Analysissi 714 Algorithms 680 20.5 Simple Parallelarallelallel AAlgorithms 716 19.6.1 Heuristic-based 20.5.1 Prere¿x ComputationCom 716 Approximation Algorithms 681 20.5.25.2 List RankingRa 718 19.6.2 Greedy Approximation 20.5.30.5.3 EulerEuleEu Tour 719 Algorithms 688 20.66 PaP ParallelrallelrallPress Searching and Parallel 19.6.3 Approximation Algorithm SortingSortiSor 720 Design Using Linear 20.6.120.620 6 Parallel Searching 720 Programming 693 220.6.2 Odd–Even Swap Sort 722 19.6.4 Designing Approximation 20.6.3 Parallel Merge–Split Algorithms Using Dynamic Algorithm 724 Programming 696 20.7 Additional Parallel Algorithms 726 20.7.1 Parallel Matrix 20 Parallel Algorithms 705 Multiplication 726 20.1 Introduction to Parallelel ProcessProcessingngUniversity 705 20.7.2 Parallel Graph Algorithms 729

Appendix A—Mathematicalical Basics 735 Appendix B—Proof Techniquesechniques 752 Bibliography 763 Index 766 Oxford

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ CHAPTER Introducon to Algorithms 1

“Ideas are the beginning point of all fortunes.” —Napolean Hill

1.1 INTRODUCTION Learning OďũecƟves Computers are powerful tools of computing.ing.g. One cacannot ignore the impact of This chapter introduces computers on our modern life. We usee computercomputecomputers for personal needs such as the basics of algorithms. All important de¿nitions typing documents, browsing the Internet,nternernet, Presssenssending emails, playing computer and concepts related to games, performing numeric calculations,alculations,alculation and so on. Industries and govern- the study of algorithms are ments use computers muchh more effeceffectieffectively to perform complicated tasks the focus of this chapter. to improve productivity andd ef¿ciency.ciencycie Applications of computer systems in The reader would be airline reservation, videodeoo surveilsurveillancsurveillance, biometric recognition, e-governance, and familiar with the following concepts by the end of e-commerce are alllll examples of their usefulness in improving ef¿ciency and this chapter: productivity. Thehe increasinincreasing iimportance of computers in our lives has prompted Basic terminologies of schools andd universitiesuniversitie tto introduce computer science as an integral part of algorithm study our moderndernern education.educat Informally, everyone is expected to handle computers to Need for algorithms accomplishmplishplish certaince Universityain basic tasks. This knowledge of using computers to perform Characteristics of ourur day-to-dayday-to-da activities is often called computational thinking. Computational algorithms thinking isi a necessity to survive in this modern world. However, computer Stages of a problem- solving process sciencescien professionals are expected to accomplish much more than acquiring Need for ef¿cient thisth basic skill of computer usage. They are required to write speci¿c computer algorithms programs to provide computer-automated solutions for problems. Writing a Classi¿cation of Oxfordprogram is slightly more complicated than merely learning to use computers, algorithms as it requires ‘algorithmic thinking’. Algorithmic thinking is an important analytical skill that is required for writing effective programs in order to solve given problems. Algorithmic thinking is not con¿ned to computer science only, but it spreads through all disciplines of study. Computer science is a domain where the skills of algorithmic thinking are taught to the aspiring computer professionals for solving computational problems.

1.2 NEED FOR ALGORITHMIC THINKING Norman E. Gibbs and Allen B. Tucker1 proposed a de¿nition for computer science that captures the core truth of computer science study. According to

1 Gibbs, Norman E., Allen B. Tucker, ‘A Model Curriculum for a Liberal Arts Degree in Computer Science’, Communications of ACM, Vol. 29, Issue 3, 1986.

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 2 Design and Analysis of Algorithms

them, ‘Computer science is the systematic study of algorithms and data structures, speci¿cally their formal properties, their mechanical and linguistic realizations, and their applications.’ Hence, there cannot be any dispute regarding the fact that study of algorithms is the central theme of computer science. Let us elaborate this de¿nition further. The formal and mathematical properties of algorithms include the study of algorithm correctness, algorithm design, and algorithm analysis for understanding the behaviour of algorithms. Hardware realizations include the study of computer hardware, which is necessary to run the algorithms in the form of programs. Linguistic realizations include the study of programming languages and their design, translators such as interpreters and compilers, and system software tools such as linkers and loaders, so that the algorithms can be executed by hardware in the form of programs. Applications of algorithms include the study of designes and development of ef ¿cient software packages and software tools so that thesese algoralgorithms can be used to solve speci¿c problems. Thus, computer scientists consider the study of algorithmsrithmshms as ththe core theme of computer science. The art of designing, implementing, and analysingysing algoalgalgorithms is called algorithmics. Algorithmics is a general word that comprises alll aspeaspectsp ctsPress of the study of algorithms. Let us now attempt to de¿ne algorithms formally. An algorithm isi a set of unambiguous instructions or procedures used for solving a given problemblemem to provideprovprovi correct and expected outputs for all valid and legal input data (refer to Boxox 1.1). An algorithm can also be referredd by otheother teterminologies such as recipe, prescription, process, or computational procedure.dure.re. Box 1.21 provides a brief history of algorithms. Algorithms thus serve as prescriptionsrescriptionsrescription ofo how a computer should carry out instructions to solve problems. Hence, algorithms cacan be visualized as strategies for solving a problem. One cannot write a programgramram for a givgiven problem without the necessary analytical skills or a strategy for solving the probleproblem.mTUniversity Thus, the knowledge of how to solve a problem is called algorithmic thinkingkingg. One can comparemparepare a prprogrampro construction with the construction of a house. The blueprint of the housesee incorporatingincorporaincorpor all the planning necessary for the construction of the house can be visualizedizedzed as an algorithm. A computer program is only an algorithm expressed using a programmingngOxford lalanguage such as C or C++. In addition to possessing the essential skills of using a programming language, the ability to conceive a strategy or apply analytical skills are important for solving a problem or constructing a program.

Box 1.1 Origin of the word ‘algorithm’ The word algorithm is derived from the name of a Persian Europe. He also introduced the simple step-by-step rules mathematician, Abu Ja’fer Mohammed Ibn Musa al for addition, subtraction, multiplication, and division in Khowarizmi, who lived sometime around 780–850 AD. He his book. The word algebra has also been derived from was from the town of Khowarazm, now in Uzbekistan. He the title of this book. When his book was translated to was a teacher of mathematics in Baghdad. He wrote a book Latin, his name was quoted as Algorismus from which titled Kitab al Jabr w’al Muqabala (Rules of Restoration the word ‘algorithm’ emerged. Algorithm as a word thus and Reduction) and Algoritmi Numero Indorum where became famous for referring to procedures that are used he introduced the old Arabic–Indian number systems to by computers for solving problems.

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 3

Box 1.2 Short history of compuƟng and algorithms The history of algorithms dates back to ancient civilizations Alan Turing’s Turing machines developed in 193 play that used numbers. Ancient civilizations such as Egyptian, a very important role in computational complexity theory. Indian, Greek, and Chinese were known to have used This Turing machine and Alonzo Church’s lambda calcu- algorithms or procedures to carry out tasks like simple lus formed the basis for formalization of the complexity calculations. This is in contrast to the history of modern theory. Thus, the history of algorithms is much richer than computers that were designed in the 1940s only. Thus, that of computers. the history of algorithms is much older and more exciting. The history of computing is another interesting story. Euclid, who lived in ancient Greece around 400–300 BC, Abacus and other mechanical devices used for computing is credited with writing the ¿rst algorithm in history for com- have been utilized at various stages of human history. A real puting the greatest common divisor (GCD). Archimedes attempt was made in the 17th century by Leibniz, a German created an algorithm for the approximation of the number mathematician and philosopher. He invented in¿nitesimal pi. Eratosthenes introduced to the world the algorithm for calculus, generating the idea of computers.co In 1822, Charles ¿nding prime numbers. Averroes (112–1198) also used Babbage introduced the differenceference engine, which required algorithmic methods for calculations. changing of gears manuallynuallyally to performperf calculations. These A number of inÀuences led to the formalization of the ideas ¿nally led to the invention of computers in the 1940s. theory of algorithms. Some of the noteworthy developments In 1942, the USS governmentgovernmen designed a machine called include the introduction of Boolean algebra by George ENIAC (electronicctro ic numericalnumer integrator and computer). This Boole (1847) and set theory by Gottlob Frege (1879). Set was succeededceedededed bbyyE EDVAC (electronic discrete variable theory has become a foundation of modern mathemat- automaticmaticatic computer)compucompute in 1951. In 1952, IBM introduced ics. The concept of recursion formulated by Kurt Gödel, its mainframeainframe computer.com Developments in the hardware and contributions of Giuseppe Peano (1888), Alfred North domainmain and dedeclining costs have shifted computer usage Whitehead, and Bertrand Russell in mathematical logic from big researchres and military environments to homes and led to the formalization of algorithm as an independentdent educationaleducatioeduc institutions. In 1981, IBM personal computers ¿eld of study. werew introduced for home and of¿ce use.

1.3 OVERVIEW OF ALGORITHMSMSS Un Algorithms are genericneric and nnot limited to computers alone (refer to Box 1.2). We perform many algorithmsms in our daddaily life unknowingly. Consider a few of our daily activities, some of which are listedted as follows:fo 1. Searchingingng for a speci¿c book 2. Arranging bobbooksOxford based on titles 3. Using a recipe to cook a new dish 4. Packing items in a suitcase 5. Scheduling daily activities 6. Finding the shortest path to a friend’s house 7. Searching for a document on the Internet 8. Preparing a CD of compressed personal data 9. Sending messages via email or SMS We perform many tasks without being aware of their inherent algorithms. For example, consider the activity of searching a word in a dictionary. How do we search? It can be noted that indexing the words in a dictionary reduces the effort of searching signi¿cantly. We just open the book, compare the word with the index given, and accordingly decide on the por- tions of the books to be searched for ¿nding the meaning of that particular word. This kind

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 4 Design and Analysis of Algorithms

of a plan or an idea is called a strategy, which is more Agent formally known as an algorithm. The environment of a typical algorithm is shown in Valid Fig. 1.1. Here, agent, which can be a human or a computer, Algorithm Output input is the performer. Fig. 1.1 Typical algorithm To illustrate further, let us consider the example of tea preparation. The hardware used here includes cooking utensils a heater, and a person. To make tea, one needs to have the following ingredients— water, tea powder, and sugar. These ingredients are analogous to the inputs of an algorithm. Here, preparing a cup of tea is called a process. The output of this process is, in this case, tea. This corresponds to the output of an algorithm. The procedure for preparing tea is as follows: 1. Put tea powder in a cup. 4. Pour milk. 2. Boil the water and pour it into the cup. 5. Add sugar if necessary.ecessary.essary 3. Filter it. 6. Pour the teaa intonto a cupcup. This kind of a procedure can be called an algorithmhm. It can be noted that an algorithm consists of step-by-step instructions that are requireded to accomaccomplish a certain task. Humans often perform such procedures intuitivelyivelyvely or eevenPress mechanically without spending much conscious thought, and hence, they labelell such actiactionactions as habitual activity. Many of our day-to-day activities are not very ef¿cient.. However,However alalgorithms that are meant for computers represent a different case altogether. Computermputer pproprocedures should be ef¿cient as computer resources are scarce. Hence, muchh thoughtought is ggiven for writing computer procedures that can solve problems. Problems canan be clasclassi¿ede into two types: computational problems and non-computational problems. Computational problemss can be solvedssolv by a computer system. A computational problem is characterized by twoo factors:factoractor ::(a University(a) thet formalization of all legal inputs and expected outputs of a given problemm andd (b) the characterization of the relationship between problem output and input. Thus,s, an algorialgoritalgorithm is expected to give the expected output for all legal inputs. If an algorithmhm yields tthe correct output for a legal input, then it is called an algorithmic solution. Non-computationalomputatimputati problems cannot be solved by a computer system. This classi¿cation shows the fundamentalunOxford differences between computers and humans in solving problems. Computers are more effective than humans in performing calculations and can crunch numbers in a fraction of seconds with more precision and ef¿ciency. However, humans outscore computers in recognition. The ability of recognizing an object by humans is much better than that by machines. Recognition of an object by computer systems requires lots of programming involving images and concepts of image processing. In addition, some tasks are plainly not possible to be carried out by computer systems. Can a computer offer its opinion or show emotion like humans? To put it simply, problems involving more intellectual complexity are much more dif¿cult to solve by computers as computer systems lack intelligence. Thus, developing algorithms to make computers more intelligent and make them perform tasks like humans becomes much more crucial and challenging. Developing algorithms for computers is both an art and a science. Algorithm design is an art that involves a lot of creative ideas, novelty, and even adventurous strategies and knowledge. Algorithm design is also a science because its construction usually involves the application of some set of principles. The

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 5 review and knowledge of these principles can facilitate the development of better algorithms, based on sound mathematical and scienti¿c principles. 1.3.1 ComputaƟonal Proďlems͕ Instance͕ and Size One encounters many types of computational problems in the domain of computer science. Some of the problems are as follows: Structuring problems In structuring problems, the input is restructured based on certain conditions or properties. For example, sorting a list in an ascending or a descending order is a structuring problem. Search problems A search problem involves searching for a target in a list of all possibilities. All potential solutions may be generated and represented in ththe form of a list or graph. Based on a property or condition, the best solution for the givenn problemroblem is searched. Puzzles are good examples of search problems where the target or goaloal is seasearched in a huge list of possible solutions. Construction problems These problems involvee ththehe constconstruction of solutions based on the constraints associated with the problem. Press Decision problems Decision problems arere yes/no typtype of problems where the algorithm output is restricted to answering yes or no. Let us assuaassume that a road network map of a city is given. The problem, say, ‘Is there anyny road conneconconnectivity between two cities, say Hyderabad and Chennai?’, can be called as a decisionecisioncision proprobleproblem as the output of this algorithm is restricted to either yes or no. Optimization problems Optimizationptimizati problems constitute a very important set of problems that are often encounteredred in cocomputermputempuUniversity science domain. The decision problem about road con- nectivity between Chennainnai and Hyderabad can also be posed as an optimization problem as follows: What is the shortestshortes distance between Chennai and Hyderabad? This is an optimization problem, as thehe probproblproblem involves ¿nding the shortest path. Thus, optimization problems involve a certainertainrtain objecobjectobjective function that is typically of the following form: maximize (say pro¿t) or minimizezee (say eeffort) based on a set of constraints that are associated with the problem. Once the problemro is recognized, the input and output of an algorithm should be identi¿ed. Consider, for example, a problem of ¿nding the factorial of a number. The factorial of a positive number N can be written as follows: N! = N × (N í 1) × (N í 2) × . . . × 1. A valid input can be called an instance of a problem. For example, factorial of a negative number is not possible. Therefore, all valid positive integers {0, 1, 2, . . .} can serve as inputs and every legal input is called an instance. All possible inputs of a problem are often called a domain of the input data. The input should be encoded in a suitable form so that computers can process it. The number of binary bits used to represent the given input, say N, is called the input size. Input size is important, as a larger input size consumes more computer time and space. The core question still remains to be answered: how to solve a given problem? To illustrate problem solving, let us consider a simple problem of counting the number of students in a tuition centre who have passed or failed a test, assuming that the pass mark is 50. Details of the students such as their registration number, name, and more importantly, marks obtained, which are necessary for the given problem, are shown in Table 1.1.

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 6 Design and Analysis of Algorithms

Table 1.1 Students’ course marks

RegistrationRegistration number Student namename Course marks 1 Abraham 80 2 Beena 30 3 Chander 83 4 David 23 5 Elizabeth 90 Fauzia 78

The class strength of the tuition centre is 6. The pass mark of the course is given as 50. How do we manually solve this problem? First, we will read the student marksmarks. Thus, the inputs for this problem are a set of student marks. The goal of this problem iss too print tthe pass and fail counts of the students, which is also the output of this algorithm. Thehe procproceprocess of reading a student’s mark is done manually. Compare the student mark with the pass mmamark, that is, 50. If the student mark is greater than or equal to 50, then pass count shouldhould be adadded by one. Otherwise, the fail count should be added by one. This process is repeatedpeatedated foforraPress all the students. The procedure that is done manually can bee given as an algorithm. Therefore, informally, the algorithm for this problem can be givenen as followsfollowfollows: Step 1: Let counter = 1, number of studentstudentsdents = 6 Step 2: While (counter ≤ number of students)student 2.1: Read the marks off thehe studestudents 2.2: Compare the marksarks of ththe sstudent with 50 2.3: If student mamarkrk is greagreater than or equal to 50 Then incrementcrementment thethUniversity ppass count Else incrementrement tthe fail count 2.4: Incrementcrement theth counter Step 3: Printnt passss councount and fail count Step 4: Exitxit Thus, algorithmicgorith solving can be observed to be much similar to how we solve problems manually. ŚĂƌĂĐƚĞƌŝƐƟĐƐŽĨůŐŽƌŝƚŚŵƐ Some of the important characteristics are listed as follows: Input An algorithm can have zero or more inputs. Output An algorithm should produce at least one or more outputs. 'e¿niteness An algorithm is characterized by de¿niteness. Its instructions should be clear and unambiguous without any confusion. All operations should be well de¿ned. For example, operations involving the division of zero or taking a square root of a negative number are unacceptable. 8niTueness An algorithm should be a well-de¿ned and ordered procedure that consists of a set of instructions in a speci¿c order. The order of the instructions is important as a change in the order of execution leads to a wrong result or uncertainty.

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 7

&orrectness The algorithm should be correct. (IIectiYeness An algorithm should be effective, which implies that it should be traceable manually. Finiteness An algorithm should have a ¿nite number of steps and should terminate after executing the ¿nite set of instructions. Therefore, ¿niteness is an important characteristic of an algorithm. Some of the additional characteristics that an algorithm is supposed to possess are the following: 6iPpOicit\ Ease of implementation is another important characteristic of an algorithm. *eneraOit\ An algorithm should be generic, independent of anyy pprogramming language or operating systems, and able to handle all ranges of inputs;; itt shoulshould not be written for a speci¿c instance. An algorithm that is de¿nite and effective is also called a computationalcomco procedure. In addition, an algorithm should be correct and ef¿cient,ent that is, should work correctly for all valid inputs and yield correct outputs. An algorithmm that exexecutesxecPress fast but gives a wrong result is useless. Thus, ef¿ciency of an algorithm iss secondary ccompared to program correctness. However, if many correct solutions are availableailableable for a ggiven problem, then one has to select the best solution (or the optimal solution)on)n) based on factors such as speed, memory usage, and ease of implementation. ůŐŽƌŝƚŚŵƐǀƐWƌŽŐƌĂŵƐ Algorithms can be contrastedted with propprograms. Algorithms, like blueprints that are used for constructing a house,e,, hehelplp solsolve a given problem logically. A program is an expression of that idea, consistingnsistingting of aUniversity seset of instructions written in any programming language. The development of algorithalgorithms can also be contrasted with software development. At the industrial level,l, softwaresoftwar development is usually undertaken by a team of programmers who developp software,software often for third-party customers, on a commercial basis. Software engineeringngg is a domdodomain that deals with large-scale software development. Project management issues suchOxford as team management, extensive planning, cost estimation, and project scheduling are the main focus of software engineering. On the other hand, algorithm design and analysis as a ¿eld of study take a micro view of program development. Its focus is to develop ef¿cient algorithms for basic tasks that can serve as a building block for various applications. Often project management issues of software development are not relevant to algorithms.

1.4 NEED FOR ALGORITHM EFFICIENCY Computer resources are limited. Hence, many problems that require a large amount of resources cannot be solved. One good example is the travelling salesperson problem (TSP). Its brief history is given in Box 1.3. A TSP is illustrated in Fig. 1.2. It can be modelled as a graph. A graph consists of a set of nodes and edges that interconnect the nodes. In this case, cities are the nodes and the paths that connect the cities are the edges. Edges are undirected in this case. Alternatively, it is

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 8 Design and Analysis of Algorithms

Box 1.3 History of travelling salesperson proďlem A travelling salesperson problem (TSP) is one of the most holes called vertices. The objective is to ¿nd a tour starting important optimization problems studied in history. It is from a city, visiting all other cities only once, and ¿nally very popular among computer scientists. It was studied returning to the city where the tour has started. This is and developed in the 19th century by Sir William Rowan called a Hamiltonian cycle. The TSP is to ¿nd a Hamiltonian Hamilton and Thomas Pennington Kirkman. In 1857, cycle in a graph. The problem was then studied extensively Hamilton created an Icosian game, a pegboard with 20 by Karl Menger in Vienna and by Harvard later in 1920.

A A B possible to move from a particular city (a) (b) to any other city. The complete details of graphshs are pprovided in Chapter 5. A A B TSPP involvesnvolves a travelling person who A B startsrts from a city, visits all other cities only onconce, and returns to the city from wherewherwhPress he started. C C D A brute force technique can be used (c) (d) tyt forf solving this problem. Let us enu- Fig. 1.2 Travelling salesperson problem (a) One city—noy—no—no path ity merate all the possible routes. For a (b) Two cities (c) Three cities (d) Four citiesitiess TSP involving only one city, there is no path. For two cities, there is only one path (A–B). For three cities,s, there are twotw paths. In Fig. 1.2, these two paths are A–B–C and A–C–B, assuming that A is the origoorigin from where the travelling salesperson started. For four cities, the pathshss aarere {A–D–B–C–A,{A–D–{A– A–D–C–B–A, A–B–C–D–A, A–B–D–C–A, A–C–D–B–A, and A–C–D–B–A–C–D–B–A}.C–D–B A}University Thus, every additiondditionion of a city can be noted to increase the path exponentially. Table 1.2 shows the numbermberer of posspossible routes. Therefore,e, it can beb observed that, for N cities, the number of routes would be (N í 1)! for N 2.. The availavailability of an algorithm for a problem does not mean that the problem is solvable byy a computer.comco There are many problems that cannot be solved by a computer and for many problemsOxford algorithms require a huge amount of resources that cannot be provided practically. For example, a TSP cannot be solved in reality. Why? Let us assume that there are 100 cities. As N = 100, the possible routes are then (100 í 1)! = 99!. Table 1.2 Complexity of TSP The value of the number 50! is 3041409320171337804 Number of cities Number of routes 3612608166064768844377641568960512000000000000. 1 0 (as there is no route) Therefore, 99! is a very large number, and even if a computer 2 1 takes one second for exploring a route, the algorithm will 3 2 run for years, which is plainly not acceptable. Just imagine how much time this algorithm will require, if the TSP is 4 tried out for all cities of India or USA. This shows that the 5 24 development of ef¿cient algorithms is very important as 120 computer resources are often limited.

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 9

1.5 FUNDAMENTAL STAGES OF PROBLEM SOLVING Problem solving is both an art and a science. The problem-solving process starts with the understanding of a given problem and ends with the programming code of the given problem. The stages of problem solving are shown in Fig. 1.3. The following are the stages of problem solving: 1. Understanding the problem 5. Analysing an algorithm 2. Planning an algorithm 6. Implementing an algorithm 3. Designing an algorithm 7. Performing empirical analysis (if necessary) 4. Validating and verifying an algorithm 1.5.1 Understanding the Proďlem Problem The study of an algorithm starts withh the computability theory. Theoretically, the primary question off compucomputability theory is the following: Is the given problem solvable?olvable?vable? TTherefore, a guideline for problem solving is necessary to understanunderstand the problem fully. Hence, Planning a proper problem statementt is rrequired.equireequi Any confusion regarding or misunderstanding of a problem stastatementPress will ultimately lead to a wrong result. Often, solvingolvingving the nunumnumerical instances of a given problem Algorithm design can give an insightt too the proprobleproblem. Similarly, algorithmic solutions of a related problememm will pprovprovide more knowledge for solving a given algorithm. Algorithm correctness Generally,ally,y, compucomputer systems cannot solve a problem if it is not properly de¿ned.ed.d. Puzzles ofteoften fall under this category, which needs supreme level offint intintelligence.lligence.lligenc TThese problems illustrate the limitations of computing power.ower. HHumansumaUniversity are better than computers in solving these problems. Algorithm analysis 1.5.21.5. Planning an Algorithm TheT second important stage of problem solving is planning. Some Implementationn and empirical analysisalysis of the important decisions are detailed in the following subsections. OxfordOxfDŽĚĞůŽĨŽŵƉƵƚĂƟŽŶ A computation model or computational model is an abstraction of a Post analysis real-world computer. A model of computation is a theoretical mathematical model and does not exist in physical sense. It is thus a virtual machine, and all programs can be executed on this virtual machine Is theoretically. What is the need for a computing model? It is meaning- OK less to talk about the speed of an algorithm as it varies from machine No to machine. An algorithm may run faster in machine A compared to in Yes machine B. This kind of an analysis is not correct as algorithm analysis should be independent of machines. A computing model thus facilitates Exit such machine-independent analysis. Fig. 1.3 Stages of First, all the valid operations of the model of computation should be problem solving speci¿ed. These valid operations help specify the input, process, and

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 10 Design and Analysis of Algorithms

Box 1.4 Alan Turing Alan Turing (1912–1954) is well known for his contribution a theoretical machine, called the Turing machine that is used towards computing. He is considered by many as the ‘father widely in computability theory and complexity analysis. Alan of modern computing’. His contribution towards computability Turing is also credited with the designing of the ‘Turing Test’ theory and arti¿cial intelligence is monumental. He designed as a measure of testing the intelligence of a machine.

output. In addition, the computing models provide a notion of the necessary steps to compute time and space complexity of a given algorithm. For algorithm study, a computation model such as a random access machine (RAM) or a Turing machine is used to perform complexity analysis of algorithms. RAM is discussed in Chapter 3 and Turing machines are discussed in Chapter 17. Thehe concontcontribution of Alan Turing is vital for the study of computability and computing modelsls (referrefer to Box 1.4). ĂƚĂOƌŐĂŶŝǌĂƟŽŶ Data structure concerns the way data and its relationshipships are sstored. Algorithms require data for solving a given problem. The nature of data andnd theitheir orgorganizationPress can have impacts on the ef¿ciency of algorithms. Therefore, algorithmsms and data sstructures together often constitute an important aspect of problem solving. Data organization is something we arere familifamiliar withw in our daily life. Figure 1.4 shows an example of data organization calleded a queuqqueue. A gas station with one servicing point should have a queue, as shown inn the ¿gure,ggure to avoid chaos. A queue (or ¿rst come ¿rst serve—FCFS) is an organizationationtion wherewher theth processing (¿lling of gas) is done in one end and a vehicle is added at thehe other eendend. All vehicles have same priority in this case. Often a problem dictates thee choichoiceeeof Universityof ststructures. However, this structure may not be valid in cases of, for example,ple,, handlinghandl medical emergencies, where highest priority should be given to urgent cases. Thus, data organizationanization is a very important issue that inÀuences the effectiveness of algorithms. At some pointoint of probproproblem solving, careful consideration should be given to storing the data effectively.y. A popupopular statement in computer science is ‘algorithm + data structure = program’. A wrong selectionectiOxfordti of a data structure often proves fatal in the problem-solving process.

Fuel station

Queue of trucks

Fig. 1.4 Example of a queue

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 11

1.5.3 Designing an Algorithm Algorithm design is the next stage of problem solving. The following are the two primary issues of this stage: 1. How to design an algorithm? 2. How to express an algorithm? Algorithm design is a way of developing algorithmic solutions using an appropriate design strategy. Design strategies differ from problem to problem. Just imagine searching a name in a telephone directory. If anyone starts searching from page 1, it is termed as a brute force strategy. Since a telephone directory is indexed, it makes sense to open the book and use the index at the top to locate the name. This is a better strategy. One may come across many design paradigms in the algorithm study. Some of the important design paradigms are divide and conquer, and dynamic programming. Divide and conquer, for example, divides thee prpproblem into sub-problems and combines the results of the sub-problems to get the ¿nal solutionlution oof the given problem. Dynamic programming visualizes the problem as a sequencece of decisdecisions. Then it combines the optimal solutions of the sub-problems to get an optimalmal global ssolution. Many design variants such as greedy approach, backtracking, and branchch and boubound techniques have been dealt with in this book. The role of algorithm design is importantmportanporta tinti Pressin developing ef¿cient algorithms. Thus, one important skill required for problem solving is ththe selection and application of suitable design paradigms. A skilled algorithm designersigner is cacalled an ‘algorist’.

ůŐŽƌŝƚŚŵ^ƉĞĐŝĮĐĂƟŽŶ After the algorithm is designed, it should be communicated to a programmer so that the algorithm can be coded as a program.ogram. ThisTh sstage is called algorithm speci¿cation. Only three possibilities exist for communicatingunicating the idea of algorithms. One is to use natural languages such as English to communicatemunicunic teUniversity the th aalgorithm. This is preferable. However, the natural language has some disadvantagesdvantagesantages such as ambiguity and lack of precision. Hence, an algorithm is often written andnd communcommucommunicated through pseudocode. Pseudocode is a mix of the natural language and mathematicsmathematics.hematic Another way is to use a programming language. The problem with programmingng languaglanguage notation is that readers often get bogged down by the programming code details.ils.s. ThereTherefore, the pseudocode approach is preferable. Even though there are no speci¿c rulessfo forfOxford algorithm writing, certain guidelines can be followed to make algorithms more understandable. Some of these guidelines are presented in Chapter 2.

1.5.4 ValidaƟng and Verifying an Algorithm Algorithm validation and veri¿cation are the methods of checking algorithm correctness. An algorithm is expected to give correct outputs for all valid inputs. Sometimes, algorithms may not give correct outputs due to logical errors. Hence, validation of algorithms becomes a necessity. Algorithm validation is a process of checking whether the given algorithm gives correct outputs for valid inputs or not. This is done by comparing the output of the algorithm with the expected output. Once validation is over, algorithm veri¿cation or algorithm proving begins. Algorithm veri- ¿cation is a process of providing a mathematical proof that the given algorithm works correctly for all instances of data. One way of doing it is by creating a set of assertions that are expressed using mathematical logic. Assertions are statements that indicate the conditions of algorithm

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 12 Design and Analysis of Algorithms

variables at various points of the algorithm. Preconditions indicate the conditions or variables before the execution of an algorithm, and postconditions indicate the status of the variables at the end of the execution of an algorithm. Remember, still the algorithm has not been translated or converted into a code of any programming language. Hence, all these executions are ¿rst carried out theoretically on a paper and proof for the algorithm is determined. A proof of an algorithm is said to exist if the preconditions can be shown to imply the postconditions logically. A complete proof is to write each statement clearly for proving that the algorithm is right. In addition to assertions, mathematical proof techniques such as mathematical induction can be used for proving that algorithms do work correctly. Mathematical induction is one such useful proof technique that is often used to prove that the algorithm works for all possible instances. Mathematical proofs are rigorous and always better than algorithm validation. Program correctness itself is a major study and extensive research is done in thish ¿eld. 1.5.5 Analysing an Algorithm In algorithm study, complexity analysis is important as we aree mainly iinterested in ¿nding optimal algorithms that use fewer computer resources. Complexityexity theorytheorthe is a ¿eld of algorithms that deals with the analysis of a solution in terms of computationalmputatuta ionalionPress resources and their optimization. Humans are more complex than, say, amoeba. Soo what doesdo the word ‘complexity’ refer to here? Complexity is the degree of dif¿culty associatedtedd with a problemproprob and the algorithm. The complexity of an algorithm can be observed to be relatedated to iits iinput size. For example, an algorithm for sorting an array of 10 numbers is easy,y, but the ssamsame algorithm becomes dif¿cult for 1 million inputs. This shows the connection betweenetween cocompcomplexity and the size of the input. Thus, complexity analysis iss useful forfo theth following two purposes: 1. To decide the ef¿ciencyy of the givengiv algorithms 2. To compare algorithmshmss fofor decididecdecidingUniversity the effective solutions for a given problem Consider the followingollowingowing scescenario: for a problem =, two algorithms A and B exist. Find the optimal algorithm.hm.m. To solve this, there must be some measures based on which comparisons can be made. In general,neral, twtwo types of measures are available for comparing algorithms—subjective and objective.ctive.ive. SubjectiveSubje measures are factors such as ease of implementation, algorithm style, and readabilityility oof the algorithm. However, the problem with these subjective measures is that these factors cannotOxford be quanti¿ed. In addition, a measure such as the ease of implementation, style of the algorithm, or understandability of algorithms is a subjective measure that varies from person to person. Therefore, in algorithm study, comparisons are limited to some objective measures. Objective measures, unlike subjective measures, can be quanti¿ed. The advantages of objective measures are that they are measurable and quanti¿able, and can be used for predic- tions. Often time and space are used as objective measures for analysing algorithms. Time complexity means the time taken by an algorithm to execute for different increasing inputs (i.e., differently-scaled inputs). In algorithm study, two time factors are considered— execution time and run time. Execution time (or compile time) does not depend on problem instances. Additionally, a program may be compiled many times. Therefore, time in an algorithm context always refers to the run time as only this is characterized by instances. Another complexity is space complexity, which is the measurement of memory space requirements of an algorithm. Technically, an algorithm is ef¿cient if lesser resources are used. Chapter 3 focuses on the analysis of algorithms.

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 13

In algorithm study, time complexity is not measured in absolute terms. For example, one cannot say the algorithm takes 3.67 seconds. It is wrong as time complexity is always de- noted as a complexity function t(n) or T(N), where t(n) is a function of input size ‘n’ and is not an absolute value. The variables n and N are used interchangeably and always reserved to represent the input size of the algorithm. Recollect that input size is the number of binary bits used to represent the input instance. The logic here is that for larger inputs the algorithm would take more time. For example, sorting a list of 10 elements is easy, but sorting a list of 1 billion elements is dif¿cult. Generally, algorithms whose time complexity function is a polynomial, for example, say N or log N, can be solved easily, and but problems whose algorithms have exponential functions, say 2N, are dif¿cult to solve.

Example 1.1 Assume that there are two algorithms A and B for a given problem P. The time complexity functions of algorithms A and B are, respectively,ly,, 3n anand 2n. Which algorithm should be selected assuming that all other conditions remainn thehe same for both algorithms? 6oOution Assuming that all conditions remain the same for both algorithms, the best algorithm takes less time when the input is changedgedd to a lalarger value. Results obtained employing different values of n are shown in Tableble 1.31.3. Press

Table 1.3 Time complexitiesplexitiesexities of algorithmsalg A and B Input size (n) Algorithmhmm A T(n) rsity= 3n AlgorithmAlgorithm B T(n) = 2n ers 1 3ve 2 5 15 32 10 UniveUniU30 1024 100d 300 2100

It can bee observedfordfor that algorithm A performs better as n is increased; time complexity increases linearlynearly aandan gives lesser values for larger values of n. The second algorithm instead grows exponentially,entiOxfo and 2100 is a very large number. Therefore, algorithm A is better than algorithm B.

Example 1.21.2 Let us assume that, for a problem P, two algorithms are possible—algorithm A and algorithm B. Let us also assume that the time complexities of algorithms A and B are, respectively, 3n and 10n2 instructions. In other words, the instructions or steps of algorithms A and B are 3n and 10n2 respectively. Here, n is the input size of the problem. Let the input size n be 105 instructions. If the computer system executes 109 instructions per second, how much time do algorithms A and B take? 6oOution Here, n = 105, and the computer can execute 109 instructions per second. 3 × 105 3 Therefore, algorithm A (time complexity 3n) would take = = 0.0003 seconds. 109 104 10 × (105)2 10 × 1010 Algorithm B (time complexity 10n2) would take = = 100 seconds. 109 109

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 14 Design and Analysis of Algorithms

It can be observed that algorithm A would take very less time compared to algorithm B. Hence, it is natural that algorithm A is the preferred one. One may wonder whether this has got anything to do ef¿ciency. Imagine that we are now executing algorithm A on a slower machine—a machine that executes only 105instructions. What would be the scenario in this case? 3 × 105 Algorithm A would take 5 = 3 seconds. 10 × 5 2 We can see that algorithm A is still better compared to algorithm B that takes 10 (10 ) = 105 106 seconds. Therefore, the important point to be noted here is that speed of the machine does not affect selection of algorithm A as a better algorithm. Hence, while computer speed is crucial, the role of a better designed algorithm is still signi¿cant.

1.5.6 ImplemenƟng an Algorithm and Performing Empiricalcall AnalAnalyAnalysis After the algorithm is designed, it is expressed as a program andd a suitabsuitable programming language is chosen to implement the algorithm as a code. After thehe program is written, it must be tested. Even if the algorithm is correct, sometimes the programogram may not give expected results. This may be due to syntax errors in coding the algorithmithmthm in a programmingPress language or hardware fault. The error due to syntax problems of a languageanguagenguage is cacalled a logical error. The process of removing logical errors is called debuggingg. Iff the programproprogra leads to an error even in the absence of syntax errors, then one has to go backck to the dedesigdesign stage to correct the algorithmic errors. Now, complexity analysis can be performperformed for the developed program. The analysis that is carried out after the programgramam is ddevedeveloped is called empirical analysis (or a priori analysis or theoretical analysis).sis).s). This anaaanalysis is popular for larger programs; a new area, called experimental algorithmicshmics, whewwhere analysis is performed for larger programs using a dataset, is emerging. A datasetdatasatas t Universityisi a huge collection of valid input data of an algorithm; the standard dataset thathat iss often used for testing of algorithms is called a benchmark dataset. Interpretation off the resulresultresults of a program on a benchmark dataset provides vital statistical information aboutut the bbehaviour of the algorithm. This kind of analysis is called empirical analysis (alsoalso called a posteriori analysis). In addition, the term pro¿ling is often used to denote thee proceprocesprocess of running a program on a dataset and measuring the time/space require- ment of the proprogramOxford empirically. 1.5.7 Post ;or PostmortemͿ Analysis A problem-solving process ends with postmortem analysis. Any analysis should end with a valuable insight. The following are the questions that are often raised in this problem-solving process: 1. Is the problem solvable? 2. Are there any limits for this algorithm? Is there any theoretical limit for the ef¿ciency of this problem? 3. Is the algorithm ef¿cient, and are there any other algorithms that are faster than the current algorithm? Answers to these questions give a lot of insight into the algorithms that are necessary for re¿ning the design of algorithms to make them more ef¿cient. The best possible or optimal solution that is theoretically possible for a given problem speci¿es a lower bound of a given problem. The worst-case estimate of resources required by the algorithm is called an upper

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 15

bound. Theoretically, the best solution of a problem should be closer to its lower bound. The difference between the lower and upper bounds should be minimal for a better solution. The difference between the upper and the lower bounds is called an algorithmic gap. Technically, this should be zero. However, in practice, there may be vast difference between an upper and a lower bound. A problem-solving process tries to reduce this difference by focusing on better planning, design, and implementation on a continuous basis.

1.6 CLASSIFICATION OF ALGORITHMS As algorithms are important for us, let us classify them for further elaboration. No single criterion exists for classifying algorithms. Figure 1.5 shows some of the criteria used for classi¿cation of algorithms.

Algorithms

Classi¿cation Classi¿cation ClassiCla ¿PressPrPcation Classi¿cation based on based on based on area of based on implementation design specspecialization tractability

Fig. 1.5 Classissi¿cationcatiosity oof algorithms

1.6.1 Based on ImplementaƟon Based on implementation, onene can classiclclassify the algorithms as follows ZĞĐƵƌƐŝǀĞůŐŽƌŝƚŚŵƐǀƐEŽŶͲƌĞĐƵƌƐŝǀĞ;/ƚĞƌĂƟǀĞͿůŐŽƌŝƚŚŵƐǀƐEŽŶͲƌEŽŶͲƌĞĐƵƌƐUniversity Problem reduction is a scientscienti¿c principle for problem solving. Take a problem, reduce it, and repeat the reductioneductionduction prproprocess till the problem is reduced to a level where it can be solved directly. Usingng recursionrecursion, the problem is reduced to another problem with a decrease in input instance. Thehe transfotransformed problem is the same as the original one but with a different input, which is lessss than that of the original problem. The problem reduction process is continued till the given problemOxford is reduced to a smaller problem that can be solved directly. Then the results of the sub-problems are combined to get the result of the given problem. This strategy of problem solving is called recursion. Recursive algorithms use recursive functions for creating repetitions required for solving a given problem. A good example of recursion is computing the factorial of a number n. Factorial of a number can be computed using the recursive function as follows: 0 if n = 0 n! = ® ¯n × (n í 1)! for n 1 Thus, a recursive function has a base case and an inductive case. A base case is the simplest problem that can be solved directly. In this factorial example, the base case is ¿nding 0!, as 0! can be solved directly. An inductive case is a recursive de¿nition of the problem that captures the essence of a problem reduction process. It can be observed from Fig. 1.6, which shows the computation of 4!, that recursion works by the principle of work postponement or delaying the

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 16 Design and Analysis of Algorithms

Reduce work. For a given factorial program, the facto- 4! rial function calls itself by reducing its input by Reduce 1 till the program reaches 0! This is the base 3! case and the algorithm stops here; results of the 4 × 3 × 2 × 1 Reduce sub-problems are collected for computing the 2! ¿nal answer of the given problem. Reduce 3 × 2 × 1 The correctness of recursive algorithms is 1! due to its close relationship with the concept 2 × 1 0! of mathematical induction. In other words, recursion is a mirror of mathematical induction. 1 Non-recursive (or iterative) algorithms, on Fig. 1.6 Computation of 4! using recursion the other hand, aree deductive in nature. Non- recursive algorithmsithmsms do not use the recursion concept but, instead, rely on looping constructs, such as for or whilehile statemstatement, to create repetition of tasks. Non-recursive (or iterative) algorithms and theirr anaanalyseslyses aare discussed in Chapter 2, and recursive algorithms and their analyses in Chapter 3. EĂƚƵƌĞĂŶĚEƵŵďĞƌŽĨWƌŽĐĞƐƐŽƌƐ Press An algorithm that is designed for a single processorrocessorcessor is ccacalled a sequential algorithm. A parallel algorithm is designed for systemsystemstems thattha useu a set of processors. The concept of parallel processing and distributed processingocessingessing area interrelated. Parallel systems have multiple processors that are located closely.osely. DistribuDistrDistributed systems, on the contrary, have multiple processors that are located at differentifferentferent plplaceplaces separated by a vast distance geographically. Hence, distributed systems aree called llooloosely coupled systems, while parallel systems are called tightly coupled systems.ms. DistribuDistriDistributed algorithms are implemented in a distributed system environment. Parallel algorithalgorithmsorithmsUniversity arare discussed in Chapter 20. ǆĂĐƚǀƐƉƉƌŽǆŝŵĂƟŽŶůŐŽƌŝƚŚŵƐŵĂƟŽŶůŐŽ An exact algorithmorithmthm ¿ndsnd the exact solutions for a given problem. Some problems are so complex thathatat ¿nding ttheir exact solutions is dif¿cult. Approximation algorithms (discussed in Chapterer 19 of thisth book) ¿nd equivalent or approximate solutions for a given problem. ĞƚĞƌŵŝŶŝƐƟĐǀƐEŽŶͲĚĞƚĞƌŵŝŶŝƐƟĐůŐŽƌŝƚŚŵƐĐǀOxford Deterministic algorithms always provide ¿xed predictable results for a given input. In contrast, non-deterministic algorithms or randomized algorithms take a different approach. For deterministic algorithms, the output should always be true. On the other hand, randomized algorithms relax this condition. It is argued that outputs based on random decisions may not often result in correct answers or the algorithm may not terminate at all. Thus, the accuracy of an output is associated with a probability. In daily life, we often use random decisions, for example, in games such as dice. Industries conduct random quality checks on products. Randomized samples are used to predict poll results and so on. One can be very sceptical about randomized algorithms. However, randomized algorithms have been proved to be very effective. Randomized algorithms are discussed in Chapter 19. 1.6.2 Based on Design Based on design techniques, algorithms can be classi¿ed into various categories. As discussed earlier, every design technique uses a strategy for solving a problem. Algorithms can be ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 17

classi¿ed based on design as brute force, divide and conquer, dynamic programming, greedy approach, backtracking, and branch and bound algorithms. This textbook is organized based on this classi¿cation of algorithms only. 1.6.3 Based on Area of SpecializaƟon Algorithms can be classi¿ed based on the area of specialization. General algorithms such as searching and sorting, and order statistics such as ¿nding mean, median, rank, and so on are useful for all ¿elds of specialization. These algorithms can be considered as building blocks of any area of specialization. Other than these basic algorithms, some algorithms are specialized for a particular domain. String algorithms, graph algorithms, combinatorial algorithms, and geometric problems are some examples of specialized algorithms that are discussed in this book. Let us discuss some of these algorithms now. 'ĞŶĞƌĂůůŐŽƌŝƚŚŵƐ Sorting algorithm is a general algorithm that is useful in all areas oof specialization. Sorting problem is a structural problem. It involves structuring or rearrangrearranging the sequence in a speci¿c order. The importance of this problem arises fromomm the facfact that all modern applications require sorting. For example, an organization mayy requireequi sossortingPress of employee records based on employee identi¿cation number (called primarymary key). A library system may want books to be ordered based on titles or ISBN numbers. Sortingorting probprproblems for small instances are relatively easy. When sorting is required for a largergerr numbernumber, ssay a billion elements, designing sorting algorithms becomes more challenging.ng. ŽŵĂŝŶͲƐƉĞĐŝĮĐ^ƉĞĐŝĂůŝǌĞĚůŐŽƌŝƚŚŵƐŐŽƌŝƚŚŵƐƌŝƚŚŵƐ Let us discuss some of these domain-specdomain-spdomain-speci¿c algorithms now: String algorithms Stringringg alalgorithmsorithmsorithUniversity are used frequently in document editing, web searches, and pattern matching.g. A sequence off charactercharacters is called a string. The sequence can be text, bits, numbers, or gene sequenceses (A,A, C, G, or T). One problem that is of considerable interest is string matching, which takes a pattern,pattpat searches the text string, and reports whether a pattern is present or not. Stringringng algoalgorithms,algor which are discussed in Chapter 16, constitute a very important ¿eld of study.y Graph algorithms A graph is used for modelling complex systems by representing the rela- tionships among the subsystems. A graph represents a set of nodes and edge. The nodes are also called vertices. The nodes are connected by edges, which are also known as arcs. Consider the graph shown in Fig. 1.7. Here, the nodes are given as {A, B, C, D, E} and the edges as {(A,B), (A,C), (A,D), (B,A), (B,E), (B,C), (C,A), (C,B), (C,D), (C,E), (D,A), (D,C), (D,E), (E,B), (E,C), and (E,D)}. A Many interesting problems can be formulated using this graph structure. The following are some of the interesting graph algorithms: B C D 7raYeOOing saOesperson proEOeP As stated earlier, a salesperson starts from a node, visits all other nodes only once, and gets back to the starting node. This problem is reduced to a Hamiltonian cycle if the graph is undirected. In the E case of a weighted graph (where edges carry some weights), the TSP is about Fig. 1.7 Sample graph ¿nding a tour of minimum cost. ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 18 Design and Analysis of Algorithms

Chinese postman problem This problem is the same as a TSP, but instead of vertices, an edge should be visited only once. There is no restriction on vertices in this problem. Graph colouring problem This problem is about how to colour all the vertices distinctly using only a small number of colours such that no two neighbouring vertices share the same colour. Combinatorial algorithms The focus of combinatorial algorithms is to ¿nd a combinatorial object inherent in the problem such as permutations, combinations, or a subset that satis¿es some constraints and objective functions. These problems are dif¿cult to solve, and many problems do not have algorithmic solutions. TSPs and graph colouring problems are examples of combinatorial problems. Geometric algorithms Geometric algorithms deal with geometrical objects such as points, lines, and polygons. The following are some of the algorithms discussed in this textbook: Closest pair problem This problem deals with ¿nding the distancetance bbetween points in a 2D space and ¿nding a pair of points that are closest to each otherother.her. Convex hull problem This problem deals with ¿ndingding the smsmallest convex polygon that includes all points in a 2D space. Press 1.6.4 Based on Tractability Tractability means solvability of a givenn problem wiwithin a reasonable amount of time and space. An intractable problem is dif¿cultt to solvsolve wwithin the reasonable amount of computer resources. Based on tractability, thehee followingfollowin categories are possible: Easily solvable problems Thesehese probproblemproblems have polynomial time complexity or their upper bounds are characterized by a polynomiapolynompolynomial. These problems are solvable. For example, sorting is an example of solvableblee or polynompolynpolynomialUniversity problems. Unsolvable problemsems ProbProblems such as halting problems cannot be solved at all. These are called unsolvablevable or nonnonon-computable problems. Knowledge about the non-computability of these problemslemsms helps in project management. Intractableblele problemsproblem These problems are of two categories. One category comprises a set of problemsblemslems thattth have algorithmic solutions but require more computer resources. Hence, these solutionsOxford are practically non-implementable. In addition, these have been proved to be computationally hard. The other category consists of a set of problems that have been proved to be computationally hard. For example, a TSP is a proven computationally hard problem; it had already been discussed that time complexity of the algorithm increases exponentially with an increase in the number of cities.

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 19

Thus, algorithm study can be concluded as the central theme of computer science. An aspiring computer professional should be a better problem solver. The problem-solving process starts with the understanding of a given problem and ends with ¿nding an ef¿cient programming code for the given problem. Problem solving poses many challenges as it is a creative process. Chapter 2 introduces the basics of a problem-solving process and also introduces basic guidelines for writing algorithms.

Q SUMMARY Q

An algorithm is a step-by-step procedure for solving a Algorithm veri¿cation is a process of providing a math- given problem. ematical proof that the given algorithm works correctly A computational problem is characterized by two for all instances of data. factors—speci¿cation of valid input and output param- A proof of an algorithmm is said to exist if the precondi- eters of the algorithm, and speci¿cation of the relation- tions can be shown to implymply postconditionspo logically. ship between inputs and outputs. An estimation off thee time andaan space complexities of An algorithm that yields the correct output for a legal an algorithm for varying inputin sizes is called algorithm input is called an algorithmic solution. analysis. The art of designing, analysing, and implementing Time complexityompllexe ty refrerefers to the measurement of run time algorithms is called algorithmics. of an algorithm in terms of its input size, and space Algorithms can be contrasted with programs. A program complexitymplexity is thetth measurement of space required for is an expression of algorithm in a programming language. a given algoaalgorithm. A valid input is called an instance. The number of binary A datasedatadataset is a huge collection of valid input data bits necessary to encode inputs of an algorithm is calleded of ana algorithm. The standard dataset that is often the input size. ususede for testing of algorithms is called a benchmark An algorithm should have characteristics suchuch as a dataset. well-de¿ned order, inputs, outputs, ¿niteness,ess,ss, de¿nite-nite- The theoretically best possible or optimal solution for ness, ef¿ciency, and generality. a given problem speci¿es a lower bound of a given Problem solving starts with the understandingnderstanerstan ingi ofo a problem. The worst-case estimate of resources that can problem statement without any confusion.fusion. be required by an algorithm is called an upper bound. A computation or computationalnal model is an abstraction The difference between the upper and the lower bound of a real-world computer. is called an algorithmic gap. Algorithm design is a wayy of developingdevedev algorithmic Problem reduction is a scienti¿c principle for problem solutions using a suitableuitableitable coursecourscour of action called a solving. Take a problem, reduce it, and repeat the re- design strategy. Algorithmrithm specisp ¿cation is about com- duction process till the problem is reduced to a level municating the design strategytra to a programmer often where it can be solved directly. in the form of a pseudocode. Algorithms can be categorized based on their imple- Algorithmic validation means checking whether the mentation methods, design techniques, ¿eld of study, algorithm gives a correct result or not. and tractability.

Q GLOSSARY Q Agent A performer of an algorithm $OJRULWKPYHUL¿FDWLRQ The process of providing a mathemati- Algorist A person who is skilled in algorithm development cal proof that the algorithm works correctly for all valid inputs Algorithm A step-by-step procedure for solving a given Algorithm validation The process of checking the correct- problem ness of an algorithm, this is done by giving valid inputs to Algorithm gap The difference between lower and upper it and checking its results with expected values bounds Approximation algorithms Algorithms that provide ap- $OJRULWKPVSHFL¿FDWLRQ Formalization of an algorithm in proximate solutions for problems whose exact solutions a suitable form that can be conveyed to a programmer are dif¿cult to obtain

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ 20 Design and Analysis of Algorithms

Complexity theory $ ¿HOG RI DOJRULWKP VWXG\ WKDW GHDOV Non-recursive algorithm (iterative algorithm) $Q DOJRULWKP ZLWK WKH DQDO\VLV RI DOJRULWKPV WKDW LV GHYHORSHG XVLQJ LWHUDWLYH FRQVWUXFWV IRU FUHDWLQJ Computability theory $ ¿HOG RI VWXG\ WKDW GHDOV ZLWK WKH UHSHWLWLRQ RI WDVNV LQVWHDG RI XVLQJ UHFXUVLYH IXQFWLRQV WKHRU\ RI DOJRULWKPV DQG VROYDELOLW\ RI WKH DOJRULWKPV Parallel algorithm $Q DOJRULWKP GHVLJQHG IRU V\VWHPV WKDW Computational model $Q DEVWUDFWLRQ RI D UHDO-ZRUOG FRP- XVH PXOWLSOH SURFHVVRUV SXWHU V\VWHP 3UR¿OLQJ 7KH SURFHVV RI PHDVXULQJ WLPH DQG VSDFH FRP- Computational problem $ SUREOHP WKDW FDQ EH VROYHG E\ SOH[LW\ E\ UXQQLQJ D SURJUDP RQ D ODUJHU GDWDVHW FRPSXWHU V\VWHPV Proof 7KH ORJLFDO GHULYDWLRQ VKRZLQJ WKDW WKH SUHFRQGLWLRQV Deterministic algorithm $OJRULWKPV WKDW DOZD\V JLYH WKH RI DOJRULWKPV LPSO\ WKHLU SRVWFRQGLWLRQV VDPH UHVXOW IRU D ¿[HG LQSXW Recursive algorithm $Q DOJRULWKP WKDW XVHV D UHFXUVLYH Exact algorithm $Q DOJRULWKP WKDW LV GHVLJQHG WR JLYH DQ IXQFWLRQ H[DFW VROXWLRQ IRU D JLYHQ SUREOHP Sequential algorithm $Q DOJRULWKP WKDW LV GHVLJQHG IRU D Experimental algorithmics $ ¿HOG RI VWXG\ WKDW GHDOV ZLWK VLQJOH-SURFHVVRU V\VWHP WKH DQDO\VLV RI DOJRULWKPV H[SHULPHQWDOO\ XVLQJ D ODUJH Space complexity 0HDVXUHPHQW RI VSDFH UHTXLUHG IRU GDWDVHW DQ DOJRULWKP Intractable problems 3UREOHPV WKDW FDQQRW EH VROYHG ZLWKLQ Strategy $ ZD\ RI GHVLJQLQJLJQLQJ DOJRULWKPVD D UHDVRQDEOH DPRXQW RI FRPSXWHU UHVRXUFHV Time complexity 5XQ WLPH PHDVXUHPHQW RI WKH DOJRULWKP Lower bound $ WKHRUHWLFDO EHVW VROXWLRQ IRU D JLYHQ SUREOHP ZKHQ WKH LQSXWSXWS LV VFDOHGVFDOH WR D ODUJHU YDOXH Non-deterministic algorithm $Q DOJRULWKPV WKDW XVHV UDQ- Tractable problemsoblems 3UREOHPV3 IRU ZKLFK DOJRULWKPV WKDW GRPL]HG FKRLFHV WKDW GHWHUPLQH WKH FRXUVH RI H[HFXWLRQ ¿QG VROXWLRQVROXWLRWLR V ZLWKLQZLWKZ D UHDVRQDEOH DPRXQW RI FRPSXWHU RI LQVWUXFWLRQV KHQFH WKH RXWSXW RI DQ DOJRULWKP YDULHV UHVRXUFHVVRXUFHVRXUFHV H[LVWH[LH Press HYHQ IRU D ¿[HG LQSXW Upperpperer bounbound 77KH ZRUVW-FDVH FRPSOH[LW\ RI DQ DOJRULWKP

Q REVIEWW QUESTIONQUESTQUESTIONS Q 'H¿QHDQDOJRULWKP . :KDWLVDQDOJRULWKPVSHFL¿FDWLRQ":KDWLVDSVHXGR- :KDWDUHWKHFKDUDFWHULVWLFVRIDQDOJRULWKP"P" FRGHDQGKRZGRHVLWKHOSLQDOJRULWKPVSHFL¿FDWLRQ" 6XUYH\WKH,QWHUQHWDQGOLVWRXWDWOHDVW¿YHDOJRULWKPVYH DOJRULWKPVDOJRULWK :KDWLVWKHGLIIHUHQFHEHWZHHQDOJRULWKPYHUL¿FDWLRQ WKDWKDYHKXJHLPSDFWRQRXUGDLO\OLYHVLYHVYHV DQGDOJRULWKPYDOLGDWLRQ" :KDWDUHWKHVWDJHVRISUREOHPVROYLQJ"VROYLQJ"YLQJ" University +RZLVDQDOJRULWKPYDOLGDWHGDQGYHUL¿HG"([SODLQ :KDWLVPHDQWE\WLPHDQGVSDFHFRPSOH[LWLHV"SDFH FRPSOH[ ZLWKDQH[DPSOH 'H¿QHWKHWHUP²GHVLJQSDUDGLJPDUDGLJPUDGLJP. :KDWDUHWKHFULWHULDXVHGIRUFODVVL¿FDWLRQRIDOJRULWKPV"

Q EXERCISES Q $VVXPHWKDWWKHUHDUHWZRDOJRULWKPV$DQG%IRUDWZOxford FRPSOH[LWLHV RI$ % DQG & DUH n n DQG ORJ n JLYHQSUREOHP37KHWLPHFRPSOH[LW\IXQFWLRQVRIDOJR- UHVSHFWLYHO\$VVXPHWKDWWKHLQSXWLQVWDQFHnLV3. 9 ULWKPV$DQG%DUHUHVSHFWLYHO\nDQGORJ2n:KLFK $VVXPH WKDW WKH PDFKLQH H[HFXWHV LQVWUXFWLRQV DOJRULWKPVKRXOGEHVHOHFWHGDVVXPLQJWKDWDOORWKHU SHU VHFRQG. +RZ PXFK WLPH ZLOO DOJRULWKPV $ % DQG FRQGLWLRQVUHPDLQWKHVDPHIRUERWKWKHSUREOHPV" & WDNH" :KLFK DOJRULWKP ZLOO EH WKH EHVW" 2 /HWXVDVVXPHWKDWIRUDWHOHSKRQHGLUHFWRU\VHDUFK SUREOHP3WKUHHDOJRULWKPVH[LVW$%DQG&7LPH

Q ADDITIONAL PROBLEM Q . -RKQ 0DF&RUPLFN KDG ZULWWHQ D ERRN WLWOHG Nine UHFRJQLWLRQ GDWD FRPSUHVVLRQ GDWDEDVHV DQG GLJLWDO Algorithms That Changed the Future: The Ingenious VLJQDWXUHV WKDW FKDQJHG WKH ZRUOG. Ideas that drive Today’s Computers, 3ULQFHWRQ 8QLYHUVLW\ D :KDW DUH WKHVH DOJRULWKPV" 6HDUFK WKH ,QWHUQHW 3UHVV 3ULQFHWRQ WKDW KDG OLVWHG WKH QLQH ZRQGHUIXO DQG ¿QG ZKDW WKHVH DOJRULWKPV DUH IRU. DOJRULWKPV QDPHO\ VHDUFK HQJLQH LQGH[LQJ SDJH UDQN E ,GHQWLI\ RQH PRUH DOJRULWKP WKDW \RX IHHO FKDQJHV SXEOLF NH\ FU\SWRJUDSK\ HUURU FRUUHFWLQJ FRGHV SDWWHUQ WKH ZRUOG ZH OLYH LQ.

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘ Introduction to Algorithms 21

Q CROSSWORD Q

1 2 3 45

7 8 910 11 ssss reessees PressPrPPre 12 y ititytty 13 rsr ersityeer Across DownDowD 1. Algorithms that are solved within reasonableblee amount 1.1 Measurement of time of computable resources 2. Study of analysis of algorithms 4. Algorithms that use recursive functionsons University3. A step-by-step procedure . Who coined the word algorithm analysisysis 5. A person skilled in algorithms 7. An abstraction of a real world problem 9. Process of removing syntax errors 8. Statistical process of measuringsuringing resourcesresouresourc on a larger dataset 10. Process of checking algorithms 11. Theory of algorithms 12. Analytical skills Oxford 13. Attributed as Father of Modern Computing by many

Answers to the Crossword are available as online resources

ΞKǆĨŽƌĚhŶŝǀĞƌƐŝƚǇWƌĞƐƐ͘ůůƌŝŐŚƚƐƌĞƐĞƌǀĞĚ͘