Common false beliefs in theoretical computer science

EDIT AT 10/12/08:

I'll try to modified the question so it may interest more people to share their opinions. We NEED your contributions!

This post is inspired by the one in MO: Examples of common false beliefs in mathematics. Big lists sometimes generate a massive number of answers the qualities of which are hard to control, but after the success of the related post on MO I am convinced that it would be helpful to list a bunch of common false beliefs in TCS.

Still, since the site is designed for answering research level questions, examples like $\mathsf{NP}$ stands for non-polynomial time should be not on the list. Meanwhile, we do want some examples that may not be hard, but without thinking in details it looks reasonable as well. We want the examples to be educational, and usually appears when studying the subject at the first time.

What are some (non-trivial) examples of common false beliefs in theoretical computer science, that appear to people who are studying in this area?

To be precise, we want examples different from surprising results and counterintuitive results in TCS; these kinds of results make people hard to believe, but they are TRUE. Here we are asking for surprising examples that people may think it is true at the first glance, but after a deeper thought the fault within is exposed.

As an example of proper answers on the list, this one comes from the field of algorithms and graph-theory:

For an $n$-node graph $G$, a $k$-edge separator $S$ is a subset of edges of size $k$, where the nodes of $G \setminus S$ can be partition into two non-adjacent parts, each consists of at most $3n/4$ nodes. We have the following "lemma":

A tree has a 1-edge separator.

Right?

-
The post has been flagged to be requested as a CW. –  Hsien-Chih Chang 張顯之 Dec 7 '10 at 9:12

This is one is common to computational geometry, but endemic elsewhere: Algorithms for the real RAM can be transferred to the integer RAM (for integer restrictions of the problem) with no loss of efficiency. A canonical example is the claim “Gaussian elimination runs in $O(n^3)$ time.” In fact, careless elimination orders can produce integers with exponentially many bits.

Even worse, but still unfortunately common: Algorithms for the real RAM with floor function can be transferred to the integer RAM with no loss of efficiency. In fact, a real-RAM+floor can solve any problem in PSPACE or in #P in a polynomial number of steps.

-
+1, never heard before! –  Hsien-Chih Chang 張顯之 Dec 9 '10 at 1:08
The Gaussian elimination misconception is very wide-spread. Perhaps part of the problem is that we often work on finite fields and, since there is no problem there, we forget. –  slimton Dec 11 '10 at 8:16

I've just got another myth busted, which is contributed by @XXYYXX's answer to this post:

• A problem X is $\mathsf{NP}$-hard if there is a polynomial time (or, logspace) reduction from all $\mathsf{NP}$ problems to X.
• Assume Exponential time hypothesis, 3-SAT does not have a sub-exponential time algorithm. Also, 3-SAT is in $\mathsf{NP}$.
• So no $\mathsf{NP}$-hard problems X have sub-exponential time algorithms. Otherwise a sub-exponential time algorithm for X + a polynomial time reduction = a sub-exponential time algorithm for 3-SAT.

But we do have sub-exponential time algorithms for some NP-hard problems.

-
I had the same impression. –  Mohammad Al-Turkistany Dec 7 '10 at 14:26
So what does this tell us about the Exponential Time Hypothesis? Or did I miss some flaw in this line of reasoning? –  Mikhail Glushenkov Dec 7 '10 at 23:07
There is a fault in point 3. That is exactly what I've misunderstood for a long time :) –  Hsien-Chih Chang 張顯之 Dec 8 '10 at 0:07
I am not sure if I cannot find the fault. Is it that since $P \neq NP$ , the reduction must not be necessarily polynomial but that it can be exponential in time , since both problems would be in EXPTIME (due to ETH?) –  chazisop Dec 8 '10 at 2:58
Polynomial-time reductions can change the input size by a polynomial amount. So if you reduce an instance of Q of size n to an instance of P of size n squared, a 2 to the root n algorithm for P only gives you a 2 to the n algorithm for Q. –  Russell Impagliazzo Dec 8 '10 at 4:05

A false belief that was popularized this year and is told many times when one tries to explain the whole $P \neq NP$ problem, since $P$ is explained as efficient :

"If $P=NP$ , then we can solve a vast number of problems efficiently. If not, we cannot"

If $3SAT$ can be solved in $O(n^{googolplex})$ then $P=NP$ . I don't think anyone would even think of running this algorithm.

If $P \neq NP$ , we can still have an algorithm for $TSP$ that runs in $n^{\log(\log n)}$ , which is smaller than $n^{5}$ for $n\leq2^{32}$ . Most people would be more than happy to be able to solve $TSP$ for 4 billion cities that fast.

-
The blog post by Lipton are nice: rjlipton.wordpress.com/2009/07/03/is-pnp-an-ill-posed-problem –  Hsien-Chih Chang 張顯之 Dec 8 '10 at 2:02
“For every polynomial-time algorithm you have, there is an exponential algorithm that I would rather run” -- Alan Perlis, via Gödel’s Lost Letter and P=NP. –  Pål GD Jan 22 '13 at 15:57

Distributed computing = distributed high-performance computing (clusters, grids, clouds, seti@home, ...).

Distributed algorithms = algorithms for these systems.

Spoiler: If this does not sound that much like a "false belief", I suggest that you have a look at conferences such as PODC and DISC, and see what kind of work people are really doing when they study theoretical aspects of distributed computing.

A typical problem setting is the following: We have a cycle with $n$ nodes; the nodes are labelled with unique identifiers from the set $\{1,2,...,\text{poly}(n)\}$; the nodes are deterministic and they exchange messages with each other in a synchronous manner. How many synchronous communication rounds (as a function of $n$) are needed to find a maximal independent set? How many rounds are needed to find an independent set with at least $n/1000$ nodes? [The answer to both of these questions is exactly $\Theta(\log^* n)$, discovered in 1986–2008.]

That is, people often study problems that are completely trivial from the perspective of centralised algorithms, and have very little in common with any kind of supercomputing or high-performance computing. The point certainly is not speeding up centralised computation by using more processors, or anything like that.

The goal is to build a complexity theory by classifying fundamental graph problems according to their computational complexity (e.g., how many synchronous rounds are needed; how many bits are transmitted). Problems like independent sets in cycles may seem pointless, but they serve a role similar to 3-SAT in centralised computing: a very useful starting point in reductions. For concrete real-world applications, it makes more sense to have a look at devices such as routers and switches in communication networks, instead of computers in grids and clusters.

This false belief is not entirely harmless. It actually makes it fairly difficult to sell work related to theory of distributed algorithms to the general TCS audience. I have received hilarious referee reports from TCS conferences...

-
Concerning computing, I wouldn't say this is not a false belief, but rather an outdated one. Other than multicore processors, small-scale distributed computing was a trivial case of the high-performance one (from what I know at least). With cores being "computers" but in such a small distance, with no network between them , new problems arise. I agree however that distributed algorithms should be used for m >= 2 nodes. –  chazisop Dec 8 '10 at 1:10
So you are just saying that people confuse parallel computing with distributed computing? –  Sasho Nikolov Jan 24 '13 at 13:56
I think your claim does not apply to theoretical computer scientists, though it may apply to pratictioners without a theoretical background. As pointed out by Sasho Nikolov, people working in the field know very well the differences between parallel and distributed computing. The problem arising in clusters, grids, clouds etc strictly depend on the context. For instance, we do not assume failures when using a cluster or a cloud, but we do for grids. And so on. –  Massimo Cafaro May 3 '13 at 6:59
Moreover, for this scientific community, distributed algorithms are algorithms for problems such as those commonly found in the books of Nancy Lynch, Hagit Attiya and Jennifer Welch, and Gerard Tel to name a few. And, as such, these algorithms are designed for a specific theoretical distributed computing model, and analyzed as required in terms of the resources used (time complexity, message complexity, bit complexity, number of rounds etc). –  Massimo Cafaro May 3 '13 at 7:02
@MassimoCafaro: Of course people who work in the field of distributed computing know what is distributed computing. However, my experience is that theoretical computer scientists in general do not know what is distributed computing. –  Jukka Suomela May 3 '13 at 8:56

This is really a false belief in math, but comes up often in TCS contexts: If random variables $X$ and $Y$ are independent, then conditioned on $Z$ they remain independent. (false even if $Z$ is independent of both $X$ and $Y$.)

-
Do you have a favorite simple example of this you would recommend, to help people quickly recognize why it is false? –  D.W. May 2 '13 at 21:22
Say $X$ and $Y$ are independent and uniformly random bits, and $Z = X+Y$ (mod 2). Then, $Z$ is independent of $X$ and is independent of $Y$, but conditioned on $Z$, knowing $X$ reveals $Y$ and vice versa. –  MCH May 3 '13 at 21:31

An elementary one, but common when we first dealing with asymptotic notations. Consider the following "proof" to the recurrence relation $T(n) = 2T(n/2) + O(n \log n)$ and $T(1) = 1$:

We prove by induction. For the base case it holds for $n=1$. Assume the relation holds for all numbers smaller than $n$,

\begin{align} T(n) &= 2 \cdot T(n/2) + O(n \log n) \\ &= 2 \cdot O(n/2 \log n/2) + O(n \log n) \\ &= O(n \log n/2) + O(n \log n) \\ &= O(n \log n) \\ \end{align}

Q.E.D. (is it?)

-
I've seen students do this. This is one of the pitfalls of abusing notation and writing $f(x) = O(g(x))$ rather than $f(x) \in O(g(x))$. –  Aaron Roth Dec 8 '10 at 14:35
I've seen researchers in theoretical computer science do variants of this error too ;) –  Jeremy May 18 '13 at 9:04

Here's my two cents:

The complexity class $\mathsf{RL}$, the randomized logspace, is defined as an analog of $\mathsf{RP}$, that is, the decision problems that can be solved by a non-deterministic logspace machine $M$, where

• for a positive instance, $M$ accepts with probability at least $1/2$;
• for a negative instance, $M$ rejects with probability $1$.

Furthermore, the machine always halt.

Is the definition correct? (No)

-

Until recently I thought that every multi-tape Turing machine $M$ that runs in time $T(n)$ can be simulated by a two-tape oblivious Turing machinne $M_o$ in time $O(T(n)\log T(n))$ in the following sense:

• the movement of $M_o$'s heads depends only on the input length
• for all inputs of the same length, $M_o$ stops at the same time (which is $\Theta(T(n)\log T(n))$).

(see this post of rjlipton for example)

Well, the second line does not hold in general, if $EXP-TIME\neq NEXP-TIME$. We need a fully time-constructible function of order $\Theta(T(n)\log T(n))$ (see this question for the definition of (fully) time-constructible functions) or we need to allow $M_o$ to run for infinite time (we allow $M_o$ to run after it reaches the accept state in $O(T(n)\log T(n))$ time). The problem is, that for general time-constructible $T:\mathbb{N}\rightarrow\mathbb{N}$ we are unable to "measure" $\Theta(T(n)\log T(n))$ steps in time $O(T(n)\log T(n))$ unless $EXP-TIME=NEXP-TIME$.

The proof of this claim is very similar to the proof in the answer of Q1 here, thus we will only give the key ideas.

Take an arbitrary problem $L\in NEXP-TIME$, $L\subseteq\{0,1\}^*$. Then there exists a $k\in\mathbb{N}$, s.t. $L$ can be solved by a NDTM $M$ in $2^{n^k}$ steps. We can assume that at each step $M$ goes in at most two different states for simplicity. Next define the function $$f(n)=\left\{\begin{array}{ll} (8n+2)^2 & \mbox{if }\left(\mbox{first } \lfloor\sqrt[k]{\lfloor\log n\rfloor+1}\rfloor\mbox{ bits of } bin(n)\right)\in L\\ 8n+1 & \mbox{else} \end{array} \right.$$ It can be proven that $f$ is time-construcible.

Now suppose that some oblivious Turing machine runs in time $g(n)=\Theta(f(n)\log f(n))$. Then $g$ is fully time-constructible.

Now the following algorithm solves $L$:

• on input $x$, let $n$ be the number with binary representation $x00\ldots 0$ ($|x|^{k-1}$ zeros). It follows that $x=\left(\textrm{first }\lfloor\sqrt[k]{\lfloor\log n\rfloor+1}\rfloor\textrm{ bits of }bin(n)\right)$.
• compute $g(n)$ in time $g(n)$. If $g(n)$ is large enough, then $x\in L$, else $x\not\in L$. (this only works for large enough $n$. How large depends on $g$.)

This algorithm runs in exponential time and solves $L$. Since $L\in NEXP-TIME$ was arbitrary, $EXP-TIME=NEXP-TIME$.

-

Let $f$ and $g$ be fully time-constructible functions (i.e. there exists a DTM that on input $1^n$ makes exactly $f(n)$ (resp. $g(n)$) steps) and let $f(n+1)=o(g(n))$.

The nondeterministic time-hierarchy is many times (superficially) stated as $NTIME(f(n))\subsetneq NTIME(g(n))$. (proof: ask Google for nondeterministic time hierarchy).

Well, the hierarchy actualy gives only $NTIME(g(n)) - NTIME(f(n))\neq\emptyset$. We would need e.g. $f(n)\leq g(n)$ for $NTIME(f(n))\subsetneq NTIME(g(n))$. For functions $f,g$ such that $f(n+1)=o(g(n))$, $f(n)\leq g(n)$ is very common. But strictly speaking, nondeterministic time hierarchy is many times stated superficially.

To show that $NTIME(f(n))\subseteq NTIME(g(n))$ does not hold for all fully time-constructible $f,g$ s.t. $f(n+1)=o(g(n))$, define $$f(n)=\left\{\begin{array}{ll} n+1 & n \mbox{ odd}\\ (n+1)^3 & \mbox{else} \end{array} \right.$$ and $g(n)=f(n+1)^2$. It is easy to see that $f$ and $g$ are fully time constructible and $f(n+1)=o(g(n))$. From nondeterministic time hierarchy we know that there is some language $L\in NTIME((n+1)^3)-NTIME((n+1)^2)$ over $\{0,1\}$. Define $$L_1=\{0x_10x_2\ldots 0x_n;\ \ x_1x_2\ldots x_n\in L\}.$$

It follows that $L_1\in NTIME(f(n))$. It is easy to see that from $L_1\in NTIME(g(n))$ follows $L\in NTIME((n+1)^2)$, which is not true. Hence, $L_1\in NTIME(f(n))-NTIME(g(n))$.

-

I've frequently heard it stated that Valiant-Vazirani says that $\mathsf{NP}$ randomly reduces to $\mathsf{UP}$, or that $\mathsf{NP} \subseteq \mathsf{RP}^{\mathsf{UP}}$, or that $\mathsf{NP} \subseteq \mathsf{R} \cdot \mathsf{UP}$. In particular, this would imply that if Valiant-Vazirani could be derandomized, then $\mathsf{NP}=\mathsf{UP}$. But in fact Valiant-Vazirani says that $\mathsf{NP} \subseteq \mathsf{RP}^{\mathsf{PromiseUP}}$.

Closely related false belief: $\mathsf{UP}$ is the class of languages $L$ with a nondeterministic poly time verifier such that $x \in L$ iff there is a unique witness. The correction is that the verifier must satisfy the semantic property that on all instances, there is at most one witness. The definition above, without the correction, is the definition of $\mathsf{US}$. But $\mathsf{US}$ is very different from $\mathsf{UP}$: for example, $\mathsf{coNP} \subseteq \mathsf{US}$.

-

If $\mu$ is the expected value of $X$, we expect that $\{X=\mu\}$ will actually happen.

-
This is certainly a common false belief among students of theoretical computer science, but it's not so common among theoretical computer science researchers. –  JɛﬀE May 17 '13 at 15:16

protected by KavehMay 17 '13 at 13:15

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?