17
$\begingroup$

I have found that the most notable TCS conferences like FOCS, STOC, SODA, and ICALP are single-blind. That is the authors do not know the identity of reviewers; however, the reviewers know the identity of the authors.

I believe that the most ethical way of reviewing is the double-blind review, where neither the authors nor reviewers know the identity of each other. Then, why the top TCS conferences are just single-blind? What are the good reasons for doing it?

The only reason that I could think of is that a reputed author's name or institution could boost the confidence of the reviewer about the claims made in the paper. But again it can introduce bias.


Note: There are few conferences that do follow double-blind reviewing like ESA. Then, why this practice is not followed by the other top conferences?

$\endgroup$
15
  • 3
    $\begingroup$ SoCG is currently having a vote centred around this question. Will be interesting to see the results and what the CG community thinks about that. $\endgroup$
    – Tassle
    Jun 8, 2021 at 13:15
  • 6
    $\begingroup$ Also there is a problem with posting your paper on arxiv when the reviewing process is double blind. $\endgroup$ Jun 9, 2021 at 4:33
  • 5
    $\begingroup$ None of these arguments against double-blind work if we do not require the process to be perfect, but just to aim to reduce the initial bias. I think there are no downsides in double-blind if it amounts to just omitting names and affiliations in the submission to the conference server. This is how the lightweight double-blind works in e.g. ESA, STACS, LICS, and PODC. $\endgroup$
    – Laakeri
    Jun 9, 2021 at 13:37
  • 3
    $\begingroup$ STOC is supposedly going double blind (forced to by ACM). $\endgroup$ Jun 11, 2021 at 16:53
  • 5
    $\begingroup$ FWIW SODA this year (SODA 2022) is doing a "lightweight double-blind reviewing process" (siam.org/conferences/cm/submissions-and-deadlines/…). $\endgroup$
    – Neal Young
    Jun 14, 2021 at 13:56

1 Answer 1

12
$\begingroup$

This is not an answer to the question but too long for a comment by 1253 characters.

STACS 2021 used light double blind review. We used easychair for it, which provides some functionality for double blind review. You need the executive licence, which costs some money. Costs depend on the number of submissions, I think for STACS it was around 600 British pounds.

Overall, the process went quite smoothly. As a PC co-chair, you can of course see the authors. So I actively only handled very few papers, mostly those that no one wanted to have and there, I was rather far away from the topic. Instead I focused more on monitoring the process, stimulating discussion etc.

Authors can declare conflicts of interests when submitting. I had the feeling that this was used in an honest way. Furthermore, PC members get to see a list of all authors (without the paper titles) and can declare conflicts of interest against authors. This eliminates essentially all conflicts with PC members. (As a PC chair, you can see the list of all COIs. The authors' declarations and the PC's declarations matched with very little noise.)

More problematic are the subreferees. Easychair blocks referee request to authors of a paper automatically (if you use the email addresses from the list provided by easychair). Subreferees were asked to declare COIs if they were aware of it. In most cases, this worked well. There were a few cases (maybe 5 out of > 600) were I had the feeling that the subreferee correctly guessed the authors and wrote a biased (very positive) report. One could have a person (or committee) here, who sole purpose is to look for such conflicts but who is not involved in the PC work.

In a questionaire fill out after the notification, about 85% of the subreferees said that they did not actively searched for the authors. On the other hand, about 50% were able to guess the authors of at least some of papers they reviewed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.