Linear programming 1 Linear programming Linear programming (LP, or linear optimization) is a method to achieve the best outcome (such as maximum profit or lowest cost) in a mathematical model whose requirements are represented by linear relationships. Linear programming is a special case of mathematical programming (mathematical optimization). More formally, linear programming is a technique for the optimization of a linear objective function, subject to linear equality and linear inequality constraints. Its feasible region is a convex polyhedron, which is a set defined as the intersection of finitely many half spaces, each of which is defined by a linear inequality. Its objective function is a real-valued affine function defined on this polyhedron. A linear programming algorithm finds a point in the polyhedron where this A pictorial representation of a simple linear program with two variables and six inequalities. function has the smallest (or largest) value if such a point exists. The set of feasible solutions is depicted in light Linear programs are problems that can be expressed in canonical form: red and forms a polygon, a 2-dimensional polytope. The linear cost function is represented by the red line and the arrow: The red line is a level set of the cost function, and the arrow indicates the direction in which we are optimizing. A closed feasible region of a problem with three variables is a convex polyhedron. The surfaces giving a fixed value of the objective function are planes (not shown). The linear programming problem is to find a point on the polyhedron that is on the plane with the highest possible value. where x represents the vector of variables (to be determined), c and b are vectors of (known) coefficients, A is a (known) matrix of coefficients, and is the matrix transpose. The expression to be maximized or minimized is called the objective function (cTx in this case). The inequalities Ax ≤ b and x ≥ 0 are the constraints which specify a convex polytope over which the objective function is to be optimized. In this context, two vectors are comparable Linear programming 2 when they have the same dimensions. If every entry in the first is less-than or equal-to the corresponding entry in the second then we can say the first vector is less-than or equal-to the second vector. Linear programming can be applied to various fields of study. It is used in business and economics, but can also be utilized for some engineering problems. Industries that use linear programming models include transportation, energy, telecommunications, and manufacturing. It has proved useful in modeling diverse types of problems in planning, routing, scheduling, assignment, and design. History The problem of solving a system of linear inequalities dates back at least as far as Fourier, after whom the method of Fourier–Motzkin elimination is named. The linear programming method was first developed by Leonid Kantorovich in 1939.[1] Leonid Kantorovich developed the earliest linear programming problems in 1939 for use during World War II to plan expenditures and returns in order to reduce costs to the army and increase losses to the enemy. The method was kept secret until 1947 when George B. Dantzig published the simplex method and John von Neumann developed the theory of duality as a linear optimization solution, and applied it in the field of game theory. Postwar, many industries found its use in their daily planning. The linear-programming problem was first shown to be solvable in polynomial time by Leonid Khachiyan in 1979, but a larger theoretical and practical breakthrough in the field came in 1984 when Narendra Leonid Kantorovich Karmarkar introduced a new interior-point method for solving linear-programming problems. Dantzig's original example was to find the best assignment of 70 people to 70 jobs. The computing power required to test all the permutations to select the best assignment is vast; the number of possible configurations exceeds the number of particles in the observable universe. However, it takes only a moment to find the optimum solution by posing the problem as a linear program and applying the simplex algorithm. The theory behind linear programming drastically reduces the number of possible optimal solutions that must be checked. Uses Linear programming is a considerable field of optimization for several reasons. Many practical problems in operations research can be expressed as linear programming problems. Certain special cases of linear programming, such as network flow problems and multicommodity flow problems are considered important enough to have generated much research on specialized algorithms for their solution. A number of algorithms for other types of optimization problems work by solving LP problems as sub-problems. Historically, ideas from linear programming have inspired many of the central concepts of optimization theory, such as duality, decomposition, and the importance of convexity and its generalizations. Likewise, linear programming is heavily used in microeconomics and company management, such as planning, production, transportation, technology and other issues. Although the modern management issues are ever-changing, most companies would like to maximize profits or minimize costs with limited resources. Therefore, many issues can be characterized as linear programming problems. Linear programming 3 Standard form Standard form is the usual and most intuitive form of describing a linear programming problem. It consists of the following three parts: • A linear function to be maximized e.g. • Problem constraints of the following form e.g. • Non-negative variables e.g. The problem is usually expressed in matrix form, and then becomes: Other forms, such as minimization problems, problems with constraints on alternative forms, as well as problems involving negative variables can always be rewritten into an equivalent problem in standard form. Example Suppose that a farmer has a piece of farm land, say L km2, to be planted with either wheat or barley or some combination of the two. The farmer has a limited amount of fertilizer, F kilograms, and insecticide, P kilograms. Every square kilometer of wheat requires F kilograms of fertilizer, and P kilograms of insecticide, while every 1 1 square kilometer of barley requires F kilograms of fertilizer, and P kilograms of insecticide. Let S be the selling 2 2 1 price of wheat per square kilometer, and S be the selling price of barley. If we denote the area of land planted with 2 wheat and barley by x and x respectively, then profit can be maximized by choosing optimal values for x and x . 1 2 1 2 This problem can be expressed with the following linear programming problem in the standard form: Maximize: (maximize the revenue—revenue is the "objective function") Subject to: (limit on total area) (limit on fertilizer) (limit on insecticide) (cannot plant a negative area). Which in matrix form becomes: maximize subject to Linear programming 4 Augmented form (slack form) Linear programming problems must be converted into augmented form before being solved by the simplex algorithm. This form introduces non-negative slack variables to replace inequalities with equalities in the constraints. The problems can then be written in the following block matrix form: Maximize Z: x, x ≥ 0 s where x are the newly introduced slack variables, and Z is the variable to be maximized. s Example The example above is converted into the following augmented form: Maximize: (objective function) Subject to: (augmented constraint) (augmented constraint) (augmented constraint) . where are (non-negative) slack variables, representing in this example the unused area, the amount of unused fertilizer, and the amount of unused insecticide. In matrix form this becomes: Maximize Z: Duality Every linear programming problem, referred to as a primal problem, can be converted into a dual problem, which provides an upper bound to the optimal value of the primal problem. In matrix form, we can express the primal problem as: Maximize cTx subject to Ax ≤ b, x ≥ 0; with the corresponding symmetric dual problem, Minimize bTy subject to ATy ≥ c, y ≥ 0. An alternative primal formulation is: Maximize cTx subject to Ax ≤ b; with the corresponding asymmetric dual problem, Minimize bTy subject to ATy = c, y ≥ 0. Linear programming 5 There are two ideas fundamental to duality theory. One is the fact that (for the symmetric dual) the dual of a dual linear program is the original primal linear program. Additionally, every feasible solution for a linear program gives a bound on the optimal value of the objective function of its dual. The weak duality theorem states that the objective function value of the dual at any feasible solution is always greater than or equal to the objective function value of the primal at any feasible solution. The strong duality theorem states that if the primal has an optimal solution, x*, then the dual also has an optimal solution, y*, and cTx*=bTy*. A linear program can also be unbounded or infeasible. Duality theory tells us that if the primal is unbounded then the dual is infeasible by the weak duality theorem. Likewise, if the dual is unbounded, then the primal must be infeasible. However, it is possible for both the dual and the primal to be infeasible. As an example, consider the linear program: Maximize: Subject to: . Example Revisit the above example of the farmer who may grow wheat and barley with the set provision of some L land, F fertilizer and P pesticide. Assume now that y unit prices for each of these means of production (inputs) are set by a planning board. The planning board's job is to minimize the total cost of procuring the set amounts of inputs while providing the farmer with a floor on the unit price of each of his crops (outputs), S for wheat and S for barley.