DISCLAIMER: This is an open ended question and stackexchange puritans would probably feel an extraordinary urge to vote it down to oblivion. However, I cannot think of any other forum more appropriate and promising for getting answer to this question.

While working on a research problem for course project I realize that if I model my problem in a particular way I could use techniques from a specialized track [for the recent case - PASCAL Recognizing Textual Entailment Challenge]. After the initial joy, I'm generally puzzled by the numerous papers with all of them silently shouting "Read me, else you'll miss the best way to solve the problem!". How does one learn to filter which paper to read and which one to discard in the face of deadline? Anything other than sort_by_citation count?

• This is indeed a broad question, but I like it, and the answers I see so far are very interesting! Thanks for asking that. Apr 1 '11 at 2:08
• Yes, both the answers are brilliant ... and it was a hard decision problem to select one. Apr 1 '11 at 2:21
• It's a very nice question. I think you should wait some time to accept an answer. I also find the accepted answer brilliant, but you can give others the chance to think about your question and post also a good answer. In particular there are a lot of individual approaches to select a paper. If there is some interest, we can make a new community wiki question about this topic. Apr 1 '11 at 22:29
• @Marc Gillé I felt I should wait for other answers, but @Sadeq Dousti answer was superb. But his point 3 is where people like me get trapped. That is where @MachineCharmer-s algorithm would help newbies without experience. There are many articles on how to read a paper, but there are hardly any that says how to select a paper - I am absolutely up for making it community wiki question. Apr 2 '11 at 2:19

1. If possible, find a survey, book, or a lecture note. This will help a lot to see the jungle through the trees! To find books, Google Books is your friend. Search the keywords in the Google Books, and try to find a relevant book on the subject.
2. Take the top-down / bottom-up approach, and alternate. That is, read the most recent papers first (the top research), and references thereof (bottom). Why? Because the top research is likely to be better, and has probably surveyed bottom works and improved them. However, to understand top, it is usually the case that you need to skim the bottom works. Important: Don't get stuck at the bottom. After reviewing older works, take the bottom-up approach. Finally, alternate between the top-down and bottom-up approaches.
3. Don't read a paper unless you are somehow sure that it is really relevant. The relevancy depends on several factors: The amount of citations, the fame of authors, the conference/journal it is published in, and so on.
4. Be sure to read the title and abstract carefully, and skim the sections. Decide if you should ever read this paper now, and to what extent.
5. If a paper has a conference version and a journal (full) version, select the first one. The conference version usually gives the general idea, and avoids cumbersome proofs. Read the full version only when you really need to learn the proof details/technique.

Edit 1: Some of my colleagues use reference manager tools like Zotero to help them organize their references. I'm not acquainted with such tools, but I think you should give it a go.

1. Take 100 papers.
2. Sort by number of citations.
3. Take top 50.
4. Go to the list references at the end of each paper. Make a big list using all those references.
5. Sort by number of times some paper appears in the big list.
6. Choose top 15 papers.