13
$\begingroup$

This question is about quadratic programming problems with box constraints (box-QP), i.e., optimisation problems of the form

  • minimise $f(\mathbf{x}) = \mathbf{x}^T A \mathbf{x} + \mathbf{c}^T \mathbf{x}$ subject to $\mathbf{x} \in [0,1]^n$.

If $A$ was positive semi-definite, then everything would be nice and convex and easy, and we could solve the problem in polynomial time.

On the other hand, if we had the integrality constraint $\mathbf{x} \in \{0,1\}^n$, we could easily solve the problem in time $O(2^n \cdot \mathrm{poly}(n))$ by brute force. For the purposes of this question, this is reasonably fast.

But what about the non-convex continuous case? What is the fastest known algorithm for general box-QPs?

For example, can we solve these in moderately exponential time, e.g., $O(3^n \cdot \mathrm{poly}(n))$, or is the worst-case complexity of the best known algorithms something much worse?


Background: I have some fairly small box-QPs that I would actually like to solve, and I was a bit surprised to see how poorly some commercial software packages perform, even for very small values of $n$. I started to wonder if there is a TCS explanation for this observation.

$\endgroup$
8
  • 1
    $\begingroup$ Can you solve exactly even for PSD $A$? The solution can be irrational, no? If you are willing to lose additive $\epsilon$ perhaps one can get an exponential time algorithm by doing brute-force search over a sufficiently fine grid. Just a vague suggestion. $\endgroup$ Dec 5, 2012 at 22:08
  • $\begingroup$ Downside is that the "base" of the exponent would be something like $1/\epsilon$, but maybe clever grid engineering can help for "small" $n$ $\endgroup$ Dec 5, 2012 at 22:24
  • $\begingroup$ @ChandraChekuri: Approximations are perfectly fine if you can achieve, e.g., $\epsilon = 10^{-9}$. However, brute-forcing over such a fine grid is not feasible. $\endgroup$ Dec 5, 2012 at 23:18
  • $\begingroup$ By quantifier elimination on real closed fields, it is always possible to solve these systems exactly. $\endgroup$
    – user6973
    Dec 6, 2012 at 0:29
  • 2
    $\begingroup$ If $O(3^n)$ is allowed, you may optimize the function on each of the faces of the cube just by writing down the first-order optimality criteria. $\endgroup$ Dec 8, 2012 at 19:02

1 Answer 1

12
$\begingroup$

An optimal solution lies on some face. So, we can go through all the faces of the cube, and find all stationary points on each of the faces.

Here is a more concrete procedure. A face of the cube can be characterized by two disjoint index sets $I_0$ and $I_1$. For $i \in I_0$, we fix $x_i = 0$, and for $i \in I_1$ we fix $x_i = 1$. Let $\tilde{x}$ consist of the remaining unfixed entries of $x$. This fixing turns the objective function into the following form:

$$ \tilde{x}^{\top} \tilde{A} \tilde{x} + \tilde{c}^{\top} \tilde{x} + d, $$

with appropriate $\tilde{A}$ and $\tilde{c}$, and some constant $d$, and we want to find the stationary points of this function under the condition that $0 < \tilde{x} < 1$.

To this end, we take the differentiation of the objective function to obtain

$$ \frac{1}{2} \tilde{A} \tilde{x} + \tilde{c} = 0. $$

Solving this system of linear equations gives you the stationary points, the candidates for optimal solutions. We go through all of them, check the condition, and choose one with the minimum objective value.

The overall time complexity is something like $O(3^n \text{poly}(n))$ since the number of faces of the $n$-cube is $3^n$ and a system of linear equations can be solved in polynomial time. The space complexity is polynomial in $n$.

$\endgroup$
4
  • 1
    $\begingroup$ Why does an optimal solution lie on some face with a non-convex $f$? $\endgroup$
    – cody
    Apr 15, 2015 at 20:43
  • $\begingroup$ @cody: That's because every polytope is the disjoint union of its faces. $\endgroup$ Apr 21, 2015 at 4:45
  • $\begingroup$ What if $f$ is cubic, or quartic? Does this property still hold? $\endgroup$
    – cody
    Apr 21, 2015 at 12:57
  • $\begingroup$ @cody: The property still holds, but we need to solve an algebraic equation of degree more than one. I'm afraid this is not trivial for multivariate cases. $\endgroup$ Apr 21, 2015 at 14:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.