16
$\begingroup$

Anonymous account for obvious reasons. I am a researcher in TCS. I have several publications in SODA/STOC/FOCS. I've recently been so disgruntled with the way these conferences are run, and wanted to voice my frustration into the void. It seems like the organizers collect "feedback" every year, and throw it into the bin. In particular,

  1. In what universe does the single chair model still make sense? How is the chair able to supervise the reviewing process for all PCs acurately? Every single discipline I know has switched to having several area chairs for different subdisciplines. Why are theory conferences lagging?

  2. What steps are being taken to prevent crap reviews? I've seen one of my students' SODA 2024 reviews, and they were a hot steaming pile of shit. The reviewers had "Expert" for confidence, when it was clear as day light the reviewers (external or PCs) had no idea what the paper is talking about. Some reviews were 2-3 lines, and I have no idea how that wasn't caught by a PC. What steps are being taken by the community to weed out low quality reviews and reviewers? Has any reviewer been banned from reviewing in (say) the last 5 years for the quality of their reviews?

  3. I am aware of a PC who has served several times on the committee in the last 9 years who notoriously reads the paper in their pile and writes reviews on the last day. Their reviews are notoriously utter garbage, and they get into flaming wars arguing for their BS reviews. I know they've sank several papers that deserved to get in. Why is this PC still being invited to serve on the commitee when they have shown such utter disregard to the seriousness of their task in objectively reviewing papers?

  4. I was optimistic when conferences started implementing double blind reviews, but I am now convinced that's complete and utter BS. I've recieved emails by known researchers in my area who publicized their paper after the SODA rebuttals period saying "they hoped to see it in SODA despite the poor scores". These papers, surprise surprise, got into this and last year's SODA. Not to mention a few bad actors that I am familiar with who search the name of the paper they are asked to review on arxiv, despite the instructions specifically asking not to do that. Do the conferences actually take double blind reviews seriously, or is this just lip service?

  5. Who is doing quality assurance on the papers that get in? There are for example 2 papers in last year's FOCS by 2 "superstars" where I was able to prove the statements in an hour. These are papers that claimed their main contribution was a "non-trivial" proof of such statements. Meanwhile, the papers I root for get weeded out and have to go through several rejection cycles.

  6. Why are poster sessions not a thing yet? Conferences complain that they can "only accept so many papers" because of physical constraints, but I believe this is utter BS. ML/Data mining conferences accept "good" papers, and categorize them as posters, oral, and spotlight papers. Poster papers are presented during a poster session, whereas the oral/spotlight papers get to present.

  7. What is with the obsession with acceptance rates? This is just driving people away from TCS.
The theory community keeps blabbering on wanting to be more inclusive to new researchers, but I know of at least 3 researches (including 2 of my own students) who are so sick of the BS happening in reviewing, and waiting several cycles of rejections for a paper to get in, that they switched to publishing almost solely in ML conferences. These same papers are still being cited by SODA/STOC/FOCS papers and are clearly relevant to the TCS community. Can we drop the facade of "acceptance rates" and just focus on accepting solid good papers?

As I near the end of my career, I'm filled with nothing but frustration with the way that TCS conferences are run. I actively encourage all my students to switch fields, including the really talented ones. Every time I discuss this with a TCS researcher, they share my frustration, but argue that "this is how it's always been".

$\endgroup$
12
  • 4
    $\begingroup$ About acceptance rates, they are much higher for top theory conferences than for top conferences in almost any other CS subfield. Also, coming from publishing in AAAI/IJCAI during my masters to publishing in theory in my phd, theory definitely compares positively on the quality of reviews and in the seriousness of the reviewing process in my experience. $\endgroup$
    – Laakeri
    Nov 11 at 5:32
  • 1
    $\begingroup$ @Laakeri that's not true. NeurIPS acceptance rates for 2022 was 25.6%, and 26.1% for 2023. Sigmod (for datamining) has an acceptance rate of 29.3% in 2022, and 41.8% in 2021 (I can't find figures for 2023). Compare this to for example SODA (28.1%, 30%), STOC (25.7%, 26.9%) and FOCS (37%, 34%). $\endgroup$ Nov 11 at 5:36
  • 4
    $\begingroup$ @Laakeri Also, I don't think it's constructive to compare acceptance rates across disciplines. The average ML submission is not the same as the average TCS submission. There is a ton of self-selection that happens for TCS. The current enforced acceptance rates are causing extremely solid and good papers to be rejected, when that should not be happening. $\endgroup$ Nov 11 at 5:39
  • 9
    $\begingroup$ As for crap reviews, this is inherent in the idea of the conference publication model. There is no consistent way to get quality reviews on a ridiculously short deadline such as a few weeks. Theoretical computer scientists should abandon this model and start publishing in journals, as all other scientists are doing. $\endgroup$ Nov 11 at 8:00
  • 7
    $\begingroup$ @DisgruntledTCSResearcher you state that we shouldn't compare acceptance rates between disciplines, but several of your points keep comparing TCS and ML conferences. I'm not saying all is rosy in TCS (it's not), but I haven't met a single researcher in ML (theory or not) who doesn't say the ML conference reviewing process is utterly broken. $\endgroup$
    – Clement C.
    Nov 11 at 8:54

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.