Exact algorithms for non-convex quadratic programming

This question is about quadratic programming problems with box constraints (box-QP), i.e., optimisation problems of the form

• minimise $f(\mathbf{x}) = \mathbf{x}^T A \mathbf{x} + \mathbf{c}^T \mathbf{x}$ subject to $\mathbf{x} \in [0,1]^n$.

If $A$ was positive semi-definite, then everything would be nice and convex and easy, and we could solve the problem in polynomial time.

On the other hand, if we had the integrality constraint $\mathbf{x} \in \{0,1\}^n$, we could easily solve the problem in time $O(2^n \cdot \mathrm{poly}(n))$ by brute force. For the purposes of this question, this is reasonably fast.

But what about the non-convex continuous case? What is the fastest known algorithm for general box-QPs?

For example, can we solve these in moderately exponential time, e.g., $O(3^n \cdot \mathrm{poly}(n))$, or is the worst-case complexity of the best known algorithms something much worse?

Background: I have some fairly small box-QPs that I would actually like to solve, and I was a bit surprised to see how poorly some commercial software packages perform, even for very small values of $n$. I started to wonder if there is a TCS explanation for this observation.

• Can you solve exactly even for PSD $A$? The solution can be irrational, no? If you are willing to lose additive $\epsilon$ perhaps one can get an exponential time algorithm by doing brute-force search over a sufficiently fine grid. Just a vague suggestion. – Chandra Chekuri Dec 5 '12 at 22:08
• Downside is that the "base" of the exponent would be something like $1/\epsilon$, but maybe clever grid engineering can help for "small" $n$ – Suresh Venkat Dec 5 '12 at 22:24
• @ChandraChekuri: Approximations are perfectly fine if you can achieve, e.g., $\epsilon = 10^{-9}$. However, brute-forcing over such a fine grid is not feasible. – Jukka Suomela Dec 5 '12 at 23:18
• By quantifier elimination on real closed fields, it is always possible to solve these systems exactly. – user6973 Dec 6 '12 at 0:29
• If $O(3^n)$ is allowed, you may optimize the function on each of the faces of the cube just by writing down the first-order optimality criteria. – Yoshio Okamoto Dec 8 '12 at 19:02

An optimal solution lies on some face. So, we can go through all the faces of the cube, and find all stationary points on each of the faces.

Here is a more concrete procedure. A face of the cube can be characterized by two disjoint index sets $I_0$ and $I_1$. For $i \in I_0$, we fix $x_i = 0$, and for $i \in I_1$ we fix $x_i = 1$. Let $\tilde{x}$ consist of the remaining unfixed entries of $x$. This fixing turns the objective function into the following form:

$$\tilde{x}^{\top} \tilde{A} \tilde{x} + \tilde{c}^{\top} \tilde{x} + d,$$

with appropriate $\tilde{A}$ and $\tilde{c}$, and some constant $d$, and we want to find the stationary points of this function under the condition that $0 < \tilde{x} < 1$.

To this end, we take the differentiation of the objective function to obtain

$$\frac{1}{2} \tilde{A} \tilde{x} + \tilde{c} = 0.$$

Solving this system of linear equations gives you the stationary points, the candidates for optimal solutions. We go through all of them, check the condition, and choose one with the minimum objective value.

The overall time complexity is something like $O(3^n \text{poly}(n))$ since the number of faces of the $n$-cube is $3^n$ and a system of linear equations can be solved in polynomial time. The space complexity is polynomial in $n$.

• Why does an optimal solution lie on some face with a non-convex $f$? – cody Apr 15 '15 at 20:43
• @cody: That's because every polytope is the disjoint union of its faces. – Yoshio Okamoto Apr 21 '15 at 4:45
• What if $f$ is cubic, or quartic? Does this property still hold? – cody Apr 21 '15 at 12:57
• @cody: The property still holds, but we need to solve an algebraic equation of degree more than one. I'm afraid this is not trivial for multivariate cases. – Yoshio Okamoto Apr 21 '15 at 14:25