Misplaced Pages

The Klee–Minty cube or Klee–Minty polytope (named after Victor Klee and George J. Minty) is a unit hypercube of variable dimension whose corners have been perturbed. Klee and Minty demonstrated that George Dantzig's simplex algorithm has poor worst-case performance when initialized at one corner of their "squashed cube". On the three-dimensional version, the simplex algorithm and the criss-cross algorithm visit all 8 corners in the worst case.

In particular, many optimization algorithms for linear optimization exhibit poor performance when applied to the Klee–Minty cube. In 1973 Klee and Minty showed that Dantzig's simplex algorithm was not a polynomial-time algorithm when applied to their cube. Later, modifications of the Klee–Minty cube have shown poor behavior both for other basis-exchange pivoting algorithms and also for interior-point algorithms.

Description

The Klee–Minty cube was originally specified with a parameterized system of linear inequalities, with the dimension as the parameter. The cube in two-dimensional space is a squashed square, and the "cube" in three-dimensional space is a squashed cube. Illustrations of the "cube" have appeared besides algebraic descriptions. The Klee–Minty polytope is given by: $${\displaystyle {\begin{aligned}x_{1}&\leq 5\\4x_{1}+x_{2}&\leq 25\\8x_{1}+4x_{2}+x_{3}&\leq 125\\&\vdots \\2^{D}x_{1}+2^{D-1}x_{2}+\dots +4x_{D-1}+x_{D}&\leq 5^{D}\\x_{1}\geq 0,\,\,\dots ,\,\,x_{D}&\geq 0.\end{aligned}}}$$

This has ${\displaystyle D}$ variables, ${\displaystyle D}$ constraints other than the ${\displaystyle D}$ non-negativity constraints, and ${\displaystyle 2^{D}}$ vertices, just as a ${\displaystyle D}$-dimensional hypercube does. If the objective function to be maximized is $${\displaystyle 2^{D-1}x_{1}+2^{D-2}x_{2}+\dots +2x_{D-1}+x_{D},}$$ and if the initial vertex for the simplex algorithm is the origin, then the algorithm as formulated by Dantzig visits all ${\displaystyle 2^{D}}$ vertices, finally reaching the optimal vertex ${\displaystyle (0,0,\dots ,5^{D})}$.

Computational complexity

The Klee–Minty cube has been used to analyze the performance of many algorithms, both in the worst case and on average. The time complexity of an algorithm counts the number of arithmetic operations sufficient for the algorithm to solve the problem. For example, Gaussian elimination requires the order of ${\displaystyle D^{3}}$ operations, and so it is said to have polynomial time-complexity because its complexity is bounded by a cubic polynomial. There are examples of algorithms that do not have polynomial-time complexity. For example, a generalization of Gaussian elimination called Buchberger's algorithm has for its complexity an exponential function of the problem data (the degree of the polynomials and the number of variables of the multivariate polynomials). Because exponential functions eventually grow much faster than polynomial functions, an exponential complexity implies that an algorithm has slow performance on large problems.

Worst case

In mathematical optimization, the Klee–Minty cube is an example that shows the worst-case computational complexity of many algorithms of linear optimization. It is a deformed cube with exactly  2 corners in dimension ${\displaystyle D}$. Klee and Minty showed that Dantzig's simplex algorithm visits all corners of a (perturbed) cube in dimension ${\displaystyle D}$ in the worst case.

Modifications of the Klee–Minty construction showed similar exponential time complexity for other pivoting rules of simplex type, which maintain primal feasibility, such as Bland's rule. Another modification showed that the criss-cross algorithm, which does not maintain primal feasibility, also visits all the corners of a modified Klee–Minty cube. Like the simplex algorithm, the criss-cross algorithm visits all 8 corners of the three-dimensional cube in the worst case.

Path-following algorithms

Further modifications of the Klee–Minty cube have shown the poor performance of central-path–following algorithms for linear optimization, in that the central path comes arbitrarily close to each of the corners of a cube. This "vertex-stalking" performance is surprising because such path-following algorithms have polynomial-time complexity for linear optimization.

Average case

The Klee–Minty cube has also inspired research on average-case complexity. When eligible pivots are made randomly (and not by the rule of steepest descent), Dantzig's simplex algorithm needs on average quadratically many steps (on the order of ${\displaystyle O(D^{2})}$. Standard variants of the simplex algorithm take on average ${\displaystyle D}$ steps for a cube. However according to Fukuda & Namiki (1994), when it is initialized at a random corner of the cube, the criss-cross algorithm visits only ${\displaystyle D}$ additional corners. Both the simplex algorithm and the criss-cross algorithm visit exactly 3 additional corners of the three-dimensional cube on average.

See also

• Projective algorithm of Karmarkar
• Ellipsoidal algorithm of Khachiyan

References

Notes

1. More generally, for the simplex algorithm, the expected number of steps is proportional to ${\displaystyle D}$ for linear-programming problems that are randomly drawn from the Euclidean unit sphere, as proved by Borgwardt and by Smale.

Citations

1. Klee & Minty (1972).
2. Deza, Nematollahi & Terlaky (2008).
3. • Deza, Nematollahi & Terlaky (2008)
   • Gartner & Ziegler (1994)
4. Greenberg (1997).
5. • Klee & Minty (1972)
   • Murty (1983), 14.2 Worst-case computational complexity, pp. 434–437
   • Terlaky & Zhang (1993)
6. • Bland (1977) Murty (1983), Chapter 10.5, pp. 331–333; problem 14.8, p. 439 describes Bland's rule.
   • Murty (1983), Problem 14.3, p. 438; problem 14.8, p. 439 describes the worst-case complexity of Bland's rule.
7. • Terlaky & Zhang (1993)
   • Roos (1990)
   • Fukuda & Terlaky (1997)
8. • Deza, Nematollahi & Terlaky (2008)
   • Megiddo & Shub (1989)
9. Gartner & Ziegler (1994).
10. Borgwardt (1987).
11. • Fukuda & Namiki (1994), p. 367
    • Fukuda & Terlaky (1997), p. 385

Bibliography

• Bland, Robert G. (May 1977). "New finite pivoting rules for the simplex method". Mathematics of Operations Research. 2 (2): 103–107. doi:10.1287/moor.2.2.103. JSTOR 3689647. MR 0459599.
• Borgwardt, Karl-Heinz (1987). The simplex method: A probabilistic analysis. Algorithms and Combinatorics (Study and Research Texts). Vol. 1. Berlin: Springer-Verlag. ISBN 978-3-540-17096-9. MR 0868467.
• Deza, Antoine; Nematollahi, Eissa; Terlaky, Tamás (May 2008). "How good are interior point methods? Klee–Minty cubes tighten iteration-complexity bounds" (PDF). Mathematical Programming. 113 (1): 1–14. CiteSeerX 10.1.1.214.111. doi:10.1007/s10107-006-0044-x. MR 2367063. S2CID 156325.
• Fukuda, Komei; Namiki, Makoto (March 1994). "On extremal behaviors of Murty's least index method". Mathematical Programming. 64 (1): 365–370. doi:10.1007/BF01582581. MR 1286455. S2CID 21476636.
• Greenberg, Harvey J. (1997). "Klee-Minty Polytope Shows Exponential Time Complexity of Simplex Method" (PDF). University of Colorado at Denver.
• Fukuda, Komei; Terlaky, Tamás (1997). Liebling, Thomas M.; de Werra, Dominique (eds.). "Criss-cross methods: A fresh view on pivot algorithms". Mathematical Programming, Series B. 79 (Papers from the 16th International Symposium on Mathematical Programming held in Lausanne, 1997): 369–395. CiteSeerX 10.1.1.36.9373. doi:10.1007/BF02614325. MR 1464775. S2CID 2794181. Postscript preprint.
• Gartner, B.; Ziegler, G. M. (1994). "Randomized simplex algorithms on Klee-Minty cubes". Proceedings 35th Annual Symposium on Foundations of Computer Science. IEEE. pp. 502–510. CiteSeerX 10.1.1.24.1405. doi:10.1109/SFCS.1994.365741. ISBN 978-0-8186-6580-6. S2CID 8090478.
• Klee, Victor; Minty, George J. (1972). "How good is the simplex algorithm?". In Shisha, Oved (ed.). Inequalities III (Proceedings of the Third Symposium on Inequalities held at the University of California, Los Angeles, Calif., September 1–9, 1969, dedicated to the memory of Theodore S. Motzkin). New York-London: Academic Press. pp. 159–175. MR 0332165.
• Megiddo, Nimrod; Shub, Michael (February 1989). "Boundary Behavior of Interior Point Algorithms in Linear Programming". Mathematics of Operations Research. 14 (1): 97–146. CiteSeerX 10.1.1.80.184. doi:10.1287/moor.14.1.97. JSTOR 3689840. MR 0984561.
• Murty, Katta G. (1983). Linear programming. New York: John Wiley & Sons. pp. xix+482. ISBN 978-0-471-09725-9. MR 0720547.
• Roos, C. (1990). "An exponential example for Terlaky's pivoting rule for the criss-cross simplex method". Mathematical Programming. Series A. 46 (1): 79–84. doi:10.1007/BF01585729. MR 1045573. S2CID 33463483.
• Terlaky, Tamás; Zhang, Shu Zhong (1993). "Pivot rules for linear programming: A Survey on recent theoretical developments". Annals of Operations Research. 46 (Degeneracy in optimization problems, number 1): 203–233. CiteSeerX 10.1.1.36.7658. doi:10.1007/BF02096264. ISSN 0254-5330. MR 1260019. S2CID 6058077.

External links

Algebraic description with illustration

The first two links have both an algebraic construction and a picture of a three-dimensional Klee–Minty cube:

• Deza, Antoine; Nematollahi, Eissa; Terlaky, Tamás (May 2008). "How good are interior point methods? Klee–Minty cubes tighten iteration-complexity bounds" (PDF). Mathematical Programming. 113 (1): 1–14. CiteSeerX 10.1.1.214.111. doi:10.1007/s10107-006-0044-x. MR 2367063. S2CID 156325.
• Gartner, B.; Ziegler, G. M. (1994). "Randomized simplex algorithms on Klee-Minty cubes". Proceedings 35th Annual Symposium on Foundations of Computer Science. IEEE. pp. 502–510. CiteSeerX 10.1.1.24.1405. doi:10.1109/SFCS.1994.365741. ISBN 978-0-8186-6580-6. S2CID 8090478. IEEE mis-spells the name "Gärnter" as "Gartner" (sic.).

Pictures with no linear system

• Articles of E. Nematollahi, which discuss the Klee-Minty cube with illustrations.
• A picture of a Klee-Minty cube showing a simplex-algorithm path (automatic translation of German) by Günter Ziegler. The picture in the second half of the page.

v t e Optimization: Algorithms, methods, and heuristics
Unconstrained nonlinear Functions Golden-section search Powell's method Line search Nelder–Mead method Successive parabolic interpolation Gradients Convergence Trust region Wolfe conditions Quasi–Newton Berndt–Hall–Hall–Hausman Broyden–Fletcher–Goldfarb–Shanno and L-BFGS Davidon–Fletcher–Powell Symmetric rank-one (SR1) Other methods Conjugate gradient Gauss–Newton Gradient Mirror Levenberg–Marquardt Powell's dog leg method Truncated Newton Hessians Newton's method
Constrained nonlinear General Barrier methods Penalty methods Differentiable Augmented Lagrangian methods Sequential quadratic programming Successive linear programming
Convex optimization Convex minimization Cutting-plane method Reduced gradient (Frank–Wolfe) Subgradient method Linear and quadratic Interior point Affine scaling Ellipsoid algorithm of Khachiyan Projective algorithm of Karmarkar Basis-exchange Simplex algorithm of Dantzig Revised simplex algorithm Criss-cross algorithm Principal pivoting algorithm of Lemke Active-set method
Combinatorial Paradigms Approximation algorithm Dynamic programming Greedy algorithm Integer programming Branch and bound/cut Graph algorithms Minimum spanning tree Borůvka Prim Kruskal Shortest path Bellman–Ford SPFA Dijkstra Floyd–Warshall Network flows Dinic Edmonds–Karp Ford–Fulkerson Push–relabel maximum flow
Metaheuristics Evolutionary algorithm Hill climbing Local search Parallel metaheuristics Simulated annealing Spiral optimization algorithm Tabu search
Software

• Linear programming
• Analysis of algorithms
• Computational complexity theory
• Convex geometry
• Cubes