Misplaced Pages

This article does not cite any sources. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Column generation" – news · newspapers · books · scholar · JSTOR (May 2024) (Learn how and when to remove this message)

Column generation or delayed column generation is an efficient algorithm for solving large linear programs.

The overarching idea is that many linear programs are too large to consider all the variables explicitly. The idea is thus to start by solving the considered program with only a subset of its variables. Then iteratively, variables that have the potential to improve the objective function are added to the program. Once it is possible to demonstrate that adding new variables would no longer improve the value of the objective function, the procedure stops. The hope when applying a column generation algorithm is that only a very small fraction of the variables will be generated. This hope is supported by the fact that in the optimal solution, most variables will be non-basic and assume a value of zero, so the optimal solution can be found without them.

In many cases, this method allows to solve large linear programs that would otherwise be intractable. The classical example of a problem where it is successfully used is the cutting stock problem. One particular technique in linear programming which uses this kind of approach is the Dantzig–Wolfe decomposition algorithm. Additionally, column generation has been applied to many problems such as crew scheduling, vehicle routing, and the capacitated p-median problem.

Algorithm

The algorithm considers two problems: the master problem and the subproblem. The master problem is the original problem with only a subset of variables being considered. The subproblem is a new problem created to identify an improving variable (i.e. which can improve the objective function of the master problem).

The algorithm then proceeds as follow:

1. Initialise the master problem and the subproblem
2. Solve the master problem
3. Search for an improving variable with the subproblem
4. If an improving variable is found: add it to the master problem then go to step 2
5. Else: The solution of the master problem is optimal. Stop.

Finding an improving variable

The most difficult part of this procedure is how to find a variable that can improve the objective function of the master problem. This can be done by finding the variable with the most negative reduced cost (assuming without loss of generality that the problem is a minimization problem). If no variable has a negative reduced cost, then the current solution of the master problem is optimal.

When the number of variables is very large, it is not possible to find an improving variable by calculating all the reduced cost and choosing a variable with a negative reduced cost. Thus, the idea is to compute only the variable having the minimum reduced cost. This can be done using an optimization problem called the pricing subproblem which strongly depends on the structure of the original problem. The objective function of the subproblem is the reduced cost of the searched variable with respect to the current dual variables, and the constraints require that the variable obeys the naturally occurring constraints. The column generation method is particularly efficient when this structure makes it possible to solve the sub-problem with an efficient algorithm, typically a dedicated combinatorial algorithm.

We now detail how and why to compute the reduced cost of the variables. Consider the following linear program in standard form:

${\displaystyle {\begin{aligned}&\min _{x}c^{T}x\\&{\text{subject to}}\\&Ax=b\\&x\in \mathbb {R} ^{+}\end{aligned}}}$

which we will call the primal problem as well as its dual linear program:

${\displaystyle {\begin{aligned}&\max _{u}u^{T}b\\&{\text{subject to}}\\&u^{T}A\leq c\\&u\in \mathbb {R} \end{aligned}}}$

Moreover, let ${\displaystyle x^{*}}$ and ${\displaystyle u^{*}}$ be optimal solutions for these two problems which can be provided by any linear solver. These solutions verify the constraints of their linear program and, by duality, have the same value of objective function (${\displaystyle c^{T}x^{*}=u^{*T}b}$) which we will call ${\displaystyle z^{*}}$. This optimal value is a function of the different coefficients of the primal problem: ${\displaystyle z^{*}=z^{*}(c,A,b)}$. Note that there exists a dual variable ${\displaystyle u_{i}^{*}}$ for each constraint of the primal linear model. It is possible to show that an optimal dual variable ${\displaystyle u_{i}^{*}}$ can be interpreted as the partial derivative of the optimal value ${\displaystyle z^{*}}$ of the objective function with respect to the coefficient ${\displaystyle b_{i}}$ of the right-hand side of the constraints:${\displaystyle u_{i}^{*}={\frac {\partial z^{*}}{\partial b_{i}}}}$ or otherwise ${\displaystyle u^{*}={\frac {\partial z^{*}}{\partial b}}}$. More simply put, ${\displaystyle u_{i}^{*}}$ indicates by how much increases locally the optimal value of the objective function when the coefficient ${\displaystyle b_{i}}$ increases by one unit.

Consider now that a variable ${\displaystyle y}$ was not considered until then in the primal problem. Note that this is equivalent to saying that the variable ${\displaystyle y}$ was present in the model but took a zero value. We will now observe the impact on the primal problem of changing the value of ${\displaystyle y}$ from ${\displaystyle 0}$ to ${\displaystyle {\hat {y}}}$. If ${\displaystyle c_{y}}$ and ${\displaystyle A_{y}}$ are respectively the coefficients associated with the variable ${\displaystyle y}$ in the objective function and in the constraints then the linear program is modified as follows:

${\displaystyle {\begin{aligned}&\min _{x}c^{T}x+c_{y}{\hat {y}}\\&{\text{subject to}}\\&Ax=b-A_{y}{\hat {y}}\\&x\in \mathbb {R} ^{+}\end{aligned}}}$

In order to know if it is interesting to add the variable ${\displaystyle y}$ to the problem (i.e to let it take a non-zero value), we want to know if the value ${\displaystyle z_{\hat {y}}^{*}}$ of the objective function of this new problem decreases as the value ${\displaystyle {\hat {y}}}$ of the variable ${\displaystyle y}$ increases. In other words, we want to know ${\displaystyle {\frac {\partial z_{\hat {y}}^{*}}{\partial {\hat {y}}}}}$. To do this, note that ${\displaystyle z_{\hat {y}}^{*}}$ can be expressed according to the value of the objective function of the initial primal problem: ${\displaystyle z_{\hat {y}}^{*}=c_{y}{\hat {y}}+z^{*}(c,A,b-A_{y}{\hat {y}})}$. We can then compute the derivative that interests us:

${\displaystyle {\begin{aligned}{\frac {\partial z_{\hat {y}}^{*}}{\partial {\hat {y}}}}&~=~&&c_{y}+{\frac {\partial z^{*}}{\partial {\hat {y}}}}\\&~=~&&c_{y}+{\frac {\partial z^{*}}{\partial c}}{\frac {dc}{d{\hat {y}}}}+{\frac {\partial z^{*}}{\partial A}}{\frac {dA}{d{\hat {y}}}}+{\frac {\partial z^{*}}{\partial b}}{\frac {db}{d{\hat {y}}}}\\&~=~&&c_{y}+{\frac {\partial z^{*}}{\partial b}}{\frac {db}{d{\hat {y}}}}\\&~=~&&c_{y}+u^{*}(-A_{y})\\&~=~&&c_{y}-u^{*}A_{y}\end{aligned}}}$

In other words, the impact of changing the value ${\displaystyle {\hat {y}}}$ on the value ${\displaystyle z_{\hat {y}}^{*}}$ translates into two terms. First, this change directly impacts the objective function and second, the right-hand side of the constraints is modified which has an impact on the optimal variables ${\displaystyle x^{*}}$ whose magnitude is measured using the dual variables ${\displaystyle u^{*}}$. The derivative ${\displaystyle {\frac {\partial z_{\hat {y}}^{*}}{\partial {\hat {y}}}}}$ is generally called the reduced cost of the variable ${\displaystyle y}$ and will be denoted by ${\displaystyle cr_{y}}$ in the following.

References

v t e Optimization: Algorithms, methods, and heuristics
Unconstrained nonlinear Functions Golden-section search Powell's method Line search Nelder–Mead method Successive parabolic interpolation Gradients Convergence Trust region Wolfe conditions Quasi–Newton Berndt–Hall–Hall–Hausman Broyden–Fletcher–Goldfarb–Shanno and L-BFGS Davidon–Fletcher–Powell Symmetric rank-one (SR1) Other methods Conjugate gradient Gauss–Newton Gradient Mirror Levenberg–Marquardt Powell's dog leg method Truncated Newton Hessians Newton's method
Constrained nonlinear General Barrier methods Penalty methods Differentiable Augmented Lagrangian methods Sequential quadratic programming Successive linear programming
Convex optimization Convex minimization Cutting-plane method Reduced gradient (Frank–Wolfe) Subgradient method Linear and quadratic Interior point Affine scaling Ellipsoid algorithm of Khachiyan Projective algorithm of Karmarkar Basis-exchange Simplex algorithm of Dantzig Revised simplex algorithm Criss-cross algorithm Principal pivoting algorithm of Lemke Active-set method
Combinatorial Paradigms Approximation algorithm Dynamic programming Greedy algorithm Integer programming Branch and bound/cut Graph algorithms Minimum spanning tree Borůvka Prim Kruskal Shortest path Bellman–Ford SPFA Dijkstra Floyd–Warshall Network flows Dinic Edmonds–Karp Ford–Fulkerson Push–relabel maximum flow
Metaheuristics Evolutionary algorithm Hill climbing Local search Parallel metaheuristics Simulated annealing Spiral optimization algorithm Tabu search
Software

This applied mathematics–related article is a stub. You can help Misplaced Pages by expanding it.

• v
• t
• e

• Optimization algorithms and methods
• Applied mathematics stubs