I've been half-jokingly entertaining the idea of trying to blog about "n ways not to solve P vs NP" in the indefinite future, and in my search came across the How not to solve P=NP? question on cs.stackoverflow.

Clearly P vs NP is a great starting point, being the source of so much development in TCS. But this got me wondering if there are other problems where people have consistently published their failures, or just any thoughts on why some things don't work. Surely not in papers, unless a strong negative result came out, but probably in blog format (which is increasingly becoming its own medium for communicating science).

Of course, this doesn't seem very practical (unless you store all your progress in digital or easily-convertible-to-digital format), but maybe the benefits outweigh the costs. So the other natural question is, how useful do you think it would be if we started doing it as a community? How often is a failed attemp one (or several) steps away from a success?

  • 1
    $\begingroup$ Why don't people publish failures? $\endgroup$
    – Kaveh
    May 13 '16 at 17:59
  • $\begingroup$ thanks for the pointer. Which reminds me, in fields like theoretical computer science and math, the problem is somewhat different I think; it's not exactly the publication bias empirical sciences face, since for the most part the nature of theoretical research is not statistical. That's why I believe it is a question worthy of discussion in its own right. $\endgroup$
    – amakelov
    May 13 '16 at 19:28

I've idly entertained the idea of an online arxiv overlay "journal" of negative results. The idea would be to allow negative results or counterexamples that don't seem suitable for a full publication in traditional venues. It would be lightly reviewed, so that we can guarantee that every paper has at least been looked over and approved by an expert in the area. It wouldn't necessarily be archival or "official", but it would at least be citable.

I think such a resource could be very helpful for publicizing dead ends in some lines of inquiry as well as "folklore negative results" that may not be widely known. I can think of a few examples from my own experience. A main challenge would be in finding and connecting with reviewers, due to the variety of possible research areas involved.

If there seems to be enough interest in this idea from the cstheory community, particularly people saying they'd be willing to submit and to review, then I'd be willing to take a lead in organizing. I'm pretty sure I can find some colleagues willing to join and help organize for the first few years.

  • $\begingroup$ I think this is a great idea! $\endgroup$ Mar 15 '17 at 6:33

It would be great, not only for Maths and TCS, but also for natural sciences. There the problem is even bigger: by publishing only positive and "sexy" results and ignoring negative and boring ones, we can end up with a very wrong idea of what actually happens. This is because statistical noise can be presented as relevant, when only results above a certain threshold of "interest" are published. For instance if 5 studies try to answer the same question and only one finds a surprising result, it might be the only one getting published.

In Maths and TCS the picture is different, as the main interest is to help the community by putting a "dead-end" sign at the beginning of some roads, and save time for people who were going to explore them. Collecting counter-examples is another very interesting aspect of the project.

Bottomline I'm all for it, and would support initiatives like the one usul is proposing.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.