DISCLAIMER: This is an open ended question and stackexchange puritans would probably feel an extraordinary urge to vote it down to oblivion. However, I cannot think of any other forum more appropriate and promising for getting answer to this question.

While working on a research problem for course project I realize that if I model my problem in a particular way I could use techniques from a specialized track [for the recent case - PASCAL Recognizing Textual Entailment Challenge]. After the initial joy, I'm generally puzzled by the numerous papers with all of them silently shouting "Read me, else you'll miss the best way to solve the problem!". How does one learn to filter which paper to read and which one to discard in the face of deadline? Anything other than sort_by_citation count?

• This is indeed a broad question, but I like it, and the answers I see so far are very interesting! Thanks for asking that. – Michaël Cadilhac Apr 1 '11 at 2:08
• Yes, both the answers are brilliant ... and it was a hard decision problem to select one. – Tathagata Apr 1 '11 at 2:21
• It's a very nice question. I think you should wait some time to accept an answer. I also find the accepted answer brilliant, but you can give others the chance to think about your question and post also a good answer. In particular there are a lot of individual approaches to select a paper. If there is some interest, we can make a new community wiki question about this topic. – Marc Bury Apr 1 '11 at 22:29
• @Marc Gillé I felt I should wait for other answers, but @Sadeq Dousti answer was superb. But his point 3 is where people like me get trapped. That is where @MachineCharmer-s algorithm would help newbies without experience. There are many articles on how to read a paper, but there are hardly any that says how to select a paper - I am absolutely up for making it community wiki question. – Tathagata Apr 2 '11 at 2:19

1. If possible, find a survey, book, or a lecture note. This will help a lot to see the jungle through the trees! To find books, Google Books is your friend. Search the keywords in the Google Books, and try to find a relevant book on the subject.
2. Take the top-down / bottom-up approach, and alternate. That is, read the most recent papers first (the top research), and references thereof (bottom). Why? Because the top research is likely to be better, and has probably surveyed bottom works and improved them. However, to understand top, it is usually the case that you need to skim the bottom works. Important: Don't get stuck at the bottom. After reviewing older works, take the bottom-up approach. Finally, alternate between the top-down and bottom-up approaches.
3. Don't read a paper unless you are somehow sure that it is really relevant. The relevancy depends on several factors: The amount of citations, the fame of authors, the conference/journal it is published in, and so on.
4. Be sure to read the title and abstract carefully, and skim the sections. Decide if you should ever read this paper now, and to what extent.
5. If a paper has a conference version and a journal (full) version, select the first one. The conference version usually gives the general idea, and avoids cumbersome proofs. Read the full version only when you really need to learn the proof details/technique.

Edit 1: Some of my colleagues use reference manager tools like Zotero to help them organize their references. I'm not acquainted with such tools, but I think you should give it a go.

1. Take 100 papers.
2. Sort by number of citations.
3. Take top 50.
4. Go to the list references at the end of each paper. Make a big list using all those references.
5. Sort by number of times some paper appears in the big list.
6. Choose top 15 papers.
8. And now you will be able to tell papers you should read from those you shouldn't without much effort.
• ... and now I am tempted to write a script that does till 6. – Tathagata Mar 31 '11 at 17:40
• Try tex.SE they might have it ready. And if you do enough of 7 you can write script to do 8. I suspect some people on this site actually have such a step 8 script. – Pratik Deoghare Mar 31 '11 at 17:49

I can't comment on the answer of MachineCharmer. Citations have a bias to older papers. You can't learn the state of the art by those.

I would start by some recent prestigious conferences/journals (FOCS, STOC, or whatever is relevant) and I would try to find relevant papers. It also helps to take a look at the most recent papers of researchers in this area.

Choose one paper that is relevant. Print it. Take your pencil and go to an isolated environment. Start reading by noting everything that you think it's important. During the discussion of the subject, you will encounter references which explain some theorems/previous work/ideas/etc. Go back to the references section and note why this reference seems useful. When you have read the initial paper, make a list of what's next and why. Repeat as necessary.

That's what I do as a student. Most likely, a professor could also ask some of his/her colleagues (or here) in order to find an initial point.