Tutor:- Prof. Avadhoot S. Joshi Subject:- Design & Analysis of Algorithms Session II Unit I - Fundamentals • Algorithm Design – II • Algorithm as a Technology • What kinds of problems are solved by algorithms? • Evolution of Algorithms • Timeline of Algorithms Algorithm Design – II Algorithm Design (Cntd.) Figure briefly shows a sequence of steps one typically goes through in designing and analyzing an algorithm. Figure: Algorithm design and analysis process. Algorithm Design (Cntd.) Understand the problem: Before designing an algorithm is to understand completely the problem given. Read the problem’s description carefully and ask questions if you have any doubts about the problem, do a few small examples by hand, think about special cases, and ask questions again if needed. There are a few types of problems that arise in computing applications quite often. If the problem in question is one of them, you might be able to use a known algorithm for solving it. If you have to choose among several available algorithms, it helps to understand how such an algorithm works and to know its strengths and weaknesses. But often you will not find a readily available algorithm and will have to design your own. An input to an algorithm specifies an instance of the problem the algorithm solves. It is very important to specify exactly the set of instances the algorithm needs to handle. Algorithm Design (Cntd.) Ascertaining the Capabilities of the Computational Device: After understanding the problem you need to ascertain the capabilities of the computational device the algorithm is intended for. The vast majority of algorithms in use today are still destined to be programmed for a computer closely resembling the von Neumann machine—a computer architecture outlined by the prominent Hungarian-American mathematician John von Neumann (1903–1957), in collaboration with A. Burks and H. Goldstine, in 1946. The essence of this architecture is captured by the so- called random-access machine (RAM). Its central assumption is that instructions are executed one after another, one operation at a time. Accordingly, algorithms designed to be executed on such machines are called sequential algorithms. Algorithm Design (Cntd.) Ascertaining the Capabilities of the Computational Device (Cntd.): The central assumption of the RAM model does not hold for some newer computers that can execute operations concurrently, i.e., in parallel. Algorithms that take advantage of this capability are called parallel algorithms. Speed and memory of a computer also play pivotal role depending on area where algorithm is going to be designed. For scientific work researchers do not bother for particular computers speed and memory, but if algorithm is going to be developed as practical tool the answer may depend on a problem you need to solve. In many situations you need not worry about a computer being too slow for the task because, even the “slow” computers of today are almost unimaginably fast. There are important problems, however, that are very complex by their nature, or have to process huge volumes of data, or deal with applications where the time is critical. In such situations, it is imperative to be aware of the speed and memory available on a particular computer system. Algorithm Design (Cntd.) Choosing between Exact and Approximate Problem Solving: The next principal decision is to choose between solving the problem exactly or solving it approximately. In the former case, an algorithm is called an exact algorithm; in the latter case, an algorithm is called an approximation algorithm. Why would one opt for an approximation algorithm? First, there are important problems that simply cannot be solved exactly for most of their instances. Second, available algorithms for solving a problem exactly can be unacceptably slow because of the problem’s intrinsic complexity. Third, an approximation algorithm can be a part of a more sophisticated algorithm that solves a problem exactly. Algorithm Design (Cntd.) Algorithm Design Techniques: An algorithm design technique (or “strategy” or “paradigm”) is a general approach to solving problems algorithmically that is applicable to a variety of problems from different areas of computing. There are basically 5 fundamental techniques which are used to design an algorithm efficiently: Algorithm Design (Cntd.) Algorithm Design Techniques (Cntd.): Design Description Problems that follows strategy Divide & - Divide & conquer technique is a top-down Binary search conquer approach to solve a problem. The algorithm Multiplication of two n-bits which follows divide and conquer technique numbers involves 3 steps: Quick Sort 1. Divide the original problem into a set of sub Heap Sort problems. Merge Sort 2. Conquer (or Solve) every sub-problem individually, recursive. 3. Combine the solutions of these sub problems to get the solution of original problem. Algorithm Design (Cntd.) Algorithm Design Techniques (Cntd.): Design Description Problems that follows strategy Greedy - Greedy technique is used to solve an optimization problem. Knapsack (fractional) Method - An Optimization problem is one in which, we are given a set Problem of input values, which are required to be either maximized or minimized (known as objective function) w. r. t. some Minimum cost Spanning constraints or conditions. tree - Greedy algorithm always makes the choice (greedy criteria) . Kruskal’s algorithm that looks best at the moment, to optimize a given objective function. That is, it makes a locally optimal . Prim’s algorithm choice in the hope that this choice will lead to a overall Single source shortest path globally optimal solution. problem - The greedy algorithm does not always guaranteed the optimal solution but it generally produces solutions that . Dijkstra’s algorithm are very close in value to the optimal. Algorithm Design (Cntd.) Algorithm Design Techniques (Cntd.): Design Description Problems that follows strategy Dynamic - Dynamic programming technique is similar to divide All pair shortest path- programmi and conquer approach. Both solve a problem by Floyed algorithm breaking it down into a several sub problems that can ng Chain matrix multiplication be solved recursively. The difference between the two is that in dynamic programming approach, the results Longest common obtained from solving smaller sub problems are reused subsequence (LCS) (by maintaining a table of results) in the calculation of 0/1 Knapsack Problem larger sub problems. Thus dynamic programming is a Traveling salesmen Bottom-up approach that begins by solving the smaller problem (TSP) sub-problems, saving these partial results, and then reusing them to solve larger sub-problems until the solution to the original problem is obtained. Algorithm Design (Cntd.) Algorithm Design Techniques (Cntd.): Design Description Problems that follows strategy - Reusing the results of sub-problems (by maintaining a table of results) is the major advantage of dynamic programming because it avoids the re-computations (computing results twice or more) of the same problem. Thus Dynamic programming approach takes much less time than naïve or straightforward methods, such as divide-and-conquer approach which solves problems in top-down method and having lots of re-computations. - The dynamic programming approach always gives a guarantee to get a optimal solution. Algorithm Design (Cntd.) Algorithm Design Techniques (Cntd.): Design Description Problems that follows strategy Backtrackin - The term “backtrack” was coined by American N-queen’s problem g mathematician D.H. Lehmer in the 1950s. Sum-of subset - Backtracking can be applied only for problems which admit the concept of a “partial candidate solution” and relatively quick test of whether it can possibly be completed to a valid solution. Algorithm Design (Cntd.) Algorithm Design Techniques (Cntd.): Design Description Problems that follows strategy - Backtrack algorithms try each possibility until they find the right one. It is a depth-first-search of the set of possible solutions. During the search, if an alternative doesn’t work, the search backtracks to the choice point, the place which presented different alternatives, and tries the next alternative. When the alternatives are exhausted, the search returns to the previous choice point and try the next alternative there. If there are no more choice points, the search fails. Algorithm Design (Cntd.) Algorithm Design Techniques (Cntd.): Design Description Problems that follows strategy Branch & - Branch-and-Bound (B&B) is a rather general Assignment problem Bound optimization technique that applies where the greedy Traveling salesmen method and dynamic programming fail. problem (TSP) - B&B design strategy is very similar to backtracking in that a state-space-tree is used to solve a problem. Branch and bound is a systematic method for solving optimization problems. However, it is much slower. Indeed, it often leads to exponential time complexities in the worst case. On the other hand, if applied carefully, it can lead to algorithms that run reasonably fast on average. Algorithm Design (Cntd.) Algorithm Design Techniques (Cntd.): Design Description Problems that follows strategy - The general idea of B&B is a BFS-like search for the optimal solution, but not all nodes get expanded (i.e., their children generated). Rather, a carefully selected criterion determines which node to expand and when, and another criterion tells the algorithm when an optimal solution has been found. - Branch and Bound (B&B)