# Techniques for showing that problem is in hardness "limbo"

Given a new problem in $\mathsf{NP}$ whose true complexity is somewhere between $\mathsf{P}$ and being NP-complete, there are two methods that I know of that might be used to prove that resolving this is difficult:

1. Show that the problem is GI-complete (GI = Graph Isomorphism)
2. Show that the problem is in $\mathsf{co-AM}$. By known results, such a result implies that if the problem is NP-complete, then PH collapses to the second level. For example, the famous protocol for Graph Nonisomorphism does exactly this.

Are there any other methods (maybe with different "strengths of belief") that have been used ? For any answer, an example of where it has actually been used is required: obviously there are many ways one might try to show this, but examples make the argument more convincing.

• If a problem seems hard enough, but you are not able to prove that it is NPC, a quick check is to count the number of strings of length n in the language: if the set is sparse it is unlikely to be NPC (otherwise P=NP by Mahaney's theorem) ... so it's better to direct efforts towards proving that it is in P :-) :-) An example from Fortnow & Gasarch's blog: { (n,k) : there exists a way to partition {1,...,n} into at most k boxes so that no box has x,y,z with x+y=z } Commented Feb 5, 2014 at 22:17
• @MarzioDeBiasi sounds like an answer to me. Commented Feb 6, 2014 at 0:04
• There are two parts to such a demonstration: showing the difficulty of placing the problem in BPP, and showing the difficulty of placing the problem in the class NP-complete. $\:$ (Recall that GI-completeness just means "is in GI and is GI-hard".) $\;\;\;$
– user6973
Commented Feb 6, 2014 at 4:52
• +1 for Ricky Demer; we might want to have a list of methods for the first part. Commented Feb 6, 2014 at 10:08
• For problems in FNP without obvious decision versions in NP, PPAD is a useful (and growing) class to consider. PPAD-complete problems include many problems about finding fixed points, for instance Nash equilibria. Shiva's list is useful: cs.princeton.edu/~kintali/ppad.html Commented Feb 10, 2014 at 9:30

Showing that your problem is in coAM (or SZK) is indeed one of the main ways to adduce evidence for "hardness limbo." But besides that, there are several others:

• Show that your problem is in NP ∩ coNP. (Example: Factoring.)
• Show that your problem is solvable in quasipolynomial time. (Examples: VC dimension, approximating free games.)
• Show that your problem is no harder than inverting one-way functions or solving NP on average. (Examples: Lots of problems in cryptography.)
• Show that your problem reduces to (e.g.) Unique Games or Small-Set Expansion.
• Show that your problem is in BQP. (Example: Factoring, though of course that's also in NP ∩ coNP.)
• Rule out large classes of NP-completeness reductions. (Example: The Circuit Minimization Problem, studied by Kabanets and Cai.)

I'm sure there are others that I'm forgetting.

• That's an excellent list, Scott ! Commented Feb 6, 2014 at 7:28
• Just curious...which of these techniques show that the problem is unlikely to be solvable in polynomial time (or RP, or BPP)? I didn't see any that seemed to do this.
– user1338
Commented Feb 7, 2014 at 17:06
• Philip: You're right, they don't. For adducing evidence that a particular NP problem is not in P, it all boils down to (1) trying to put it in P and failing, and/or (2) reducing other problems that people failed to put in P to that problem. Commented Feb 8, 2014 at 1:02

From the comment above: if a problem seems hard enough, but you are not able to prove that it is NP-complete, a quick check is to count the number of strings of length $n$ in the language: if the set is sparse it is unlikely to be NPC, otherwise P=NP by Mahaney's theorem ... so it's better to direct efforts towards proving that it is in P :-) :-)

An example is the problem of partitioning numbers into k-boxes (from Fortnow & Gasarch's blog, original source: Doctor Ecco's Cyberpuzzles):

$\{ (n,k) \mid \text{ there exists a way to partition }$ $\{1,...,n\} \text{ into at most k boxes so that no box has } x,y,z \text{ with } x+y=z \}$

Here are three additions to Scott's list:

• Show your problem is in fewP. This means that the number of solutions is bounded by some polynomial. (Example: Turnpike problem). No NP-complete problem is known to be in fewP. (impossible unless fewP=NP).
• Show your problem is in $LOGNP$ or in $NP[log^2n]$ (Can be solved using limited number of nondeterministic bits, Example Dominating set problem in tournaments)
• Show that your problem has sub-exponential density (H. Buhrman and J. M. Hitchcock proved density lower bound ($2^{n^{\epsilon}}$), unless the polynomial hierarchy collapses. Therefore, any $NP$-complete set must have some $\epsilon \gt 0$ such that for infinitely-many integers $n\ge 0$, the set contains at least $2^{n^{\epsilon}}$ strings of length $n$. ). This is a much stronger than proving just sparsity (as stated in Marzio's answer).

H. Buhrman and J. M. Hitchcock, NP-Hard Sets are Exponentially Dense Unless $coNP ⊆ NP/poly$, In IEEE Conference on Computational Complexity, pages 1–7, 2008

• Or even in UP (not just FewP)! Commented Feb 7, 2014 at 0:29

Show your problem is in $$RP$$ ( hence unlikely to be $$NP$$-hard)

Example:

Exact Perfect Matching problem

Input: A graph G, with each edge colored red or blue, and integer k.

Problem: Decide whether there exists a perfect matching M in G with exactly k red edges

• This is subsumed by “show that your problem is in BQP” in the top answer. Commented Aug 18, 2022 at 18:04
• @EmilJeřábek $RP$ is a subset of $NP$. $BQP$ is not known to be in $NP$. The post about $NP$ problems Commented Aug 18, 2022 at 18:08