Course Notes – January 24, 2005 IEOR 251 – Facilities Design and Logistics Notes by: Justin Azadivar Computational Complexity Garey and Johnson (1978) Appendix in Ahuja, Magnanti, Orlin, Network Flows Definition: A theoretical way to compare the efficiency of algorithms and the difficulty of problems. Efficiency means speed. Difficulty means characteristics of the problem that limit the speed of algorithms. Problem: A generic model/problem such as TSP or Shortest Path. e.g. TSP: Given a complete graph G(N,A) where each arc has associated length cij, find a minimum total length tour. Instance: A problem with data specified for the problem parameters. Algorithm: A step-by-step procedure for solving a problem instance. Steps are classified: Assignment steps (Let ) Arithmetic steps (+,-,*,/) Logical steps (Comparisons) Total number of steps is the key determinant of an algorithm’s efficiency. Worst Case Analytical Approach We express an upper bound on the number of steps an algorithm takes as a function of problem size. This bound is called the (worst case) running time or complexity of an algorithm. Problem Size Problem size is a function of the space to store data making up the problem. For a number X, this takes ~log X storage space. For our purposes, parameters such as number of nodes (n), number of arcs (m), maximum costs, distance, demands, weights (c,d,w) are adequate to describe the problem. Running Time for a Particular Problem Size Actual time depends on computer speed, etc. We count the worst case number of basic operations the problem will take as a function of problem size n. If this is f(n), we say that the algorithm is O(f(n)) (Order f(n)) if there exists K such that the running time of the algorithm is less than Kf(n). 1 We say an algorithm is a(n) Polynomial algorithm if f(n) is a polynomial function of n. Exponential algorithm if f(n) grows exponentially with n. Examples: Polynomial: o(nm) o(n2+n) -> o(n2) o(n2logn) o(nlogC) Exponential: o(2n) o(nm) o(n!) Pseudopolynomial o(nC) (is exponential in size of data) Determining Complexity, an example Graph: G consists of a set N of nodes and a set A of arcs. Network: A graph with additional information associated with each arc. Path: A sequence of nodes (e.g. {1,3,4}) such that (i,j) exists in A for every consecutive pair i,j, with no nodes repeated. Cycle: A path that returns to its starting node with no nodes repeated. Dijkstra’s algorithm for shortest path in a network with nonnegative arc lengths: Given a network with non-negative arclengths, find the shortest path from node s to all other nodes. Observe that if (1,3,4,9,8) is the shortest path from 1 to 8, then (1,3,4,9) is the shortest path from 1 to 9, etc. A solution can consist of a set of node labels: pred(i) – predecessor node in shortest path from s to i d(i) – distance from s to i in the shortest path. 2 Action Number of computing steps Step I S={s} (S is the set of scanned nodes) o(1) Sc=N\s o(n) d(i)=∞, for all i in Sc f o(n) d(s)=0 o(1) pred(i)= empty, for all i in Sc o(n) Step II m=last scanned node o(1) For all (m,j) such that j is not in S o(n) tempj=d(m)+cmj o(1) If tempj<d(j) o(1) D(j)=tempj Pred(j)=m Step III i=argmin{d(j):j in S} o(n) S=S+{i} o(1) Sc= Sc-{i} o(1) Step IV If Sc is not empty, goto step II o(1) Algorithm Complexity: Each line is o(1) or o(n). (In step II, do o(1) things o(n) times, so also o(n)) Steps repeat n-1=o(n) times. Algorithm is o(n*n)=o(n2) 3.