How to find interesting research problems

Question

Despite several years of classes, I'm still at a loss when it comes to choosing a research topic. I've been looking over papers from different areas and spoken with professors, and I'm beginning to think this is the wrong approach.

I've read that it helps to find an interesting problem (nevermind the area) and to then work on that. Textbooks mention famous unsolved ones but I wouldn't want to tackle them directly. Research papers only mentioned positive results, not failed attempts.

How can I find interesting research problems? How do you find interesting research problems? Is there a list somewhere?

How do you decide if it is worth to work on a particular problem?

Answer 1

I strongly disagree with the "find a list of open problems" approach. Usually open problems are quite hard to make progress on, and I'm thoroughly unconvinced that good research is done by tackling some hard but uninteresting problem in a technical area.

That being said, of course solving an open problem is really good for academic credentials. But that's not what you are asking.

Research is a process designed to generate understanding at a high level. Solving technical problems is a means to that end: often the problem and its solution illuminate the structure or behavior of some scientific phenomenon (a mathematical structure, a programing language practice, etc).

So my first suggestion is: find a problem that you want to understand. Research is fundamentally about confusion. Are there some specific topics you are interested in, but that you feel you have a fundamentally incomplete comprehension of, or that seem technically clear, but that you lack any good intuition for? Those are good starting points. Follow Terry Tao's advice ask yourself dumb questions! A lot of good research comes out of these considerations. In fact, this whole page contains a lot of good advice. Note that if you are looking at a well-explored problem or field, it's unlikely you'll get original insights right away, so it's important to read up on literature concurrently with your own explorations.

Second, don't discount communicating with your Professors. Ask them about their own research, not necessarily about projects they want to give you. Engage in a conversation! This helps you find out what you are interested in, but also what the research landscape looks like in their field. Research doesn't happen in a vacuum, so you should speak to your fellow students, PhDs in your department, go to talks and workshops at your university, etc. You'll find that being immersed in a research environment helps you do research a lot more than finding a list or specific problem and locking yourself in your office.

Finally, I would suggest working on something small. Research is bottom-up much more than it is top down, and it's rare that a very simple task (writing a proof or a program) turns out to be as simple as you expected it to. Doing several small projects that are not research-scale (expanding on homework, writing up an explanation of something you learned) often build up into genuine research level stuff. It's common to try to "go big" at the beginning, but that's just now how our brains work.

Answer 2

David Hilbert is a renowned mathematician. He put forth a list of 23 unsolved problems at the International Congress of Mathematicians in Paris in 1900.
I just want to quote part of Yuri Manin interview entitled "Good Proofs are Proofs that Make us Wiser" about Hilbert and his list:

This year’s International Congress is the last ICM in this century. Do you think a Hilbert is still possible? Are there any contemporary problems corresponding to Hilbert’s Problems?
I don’t actually believe that Hilbert’s list had a great role in the mathematics of this century. It certainly was psychologically important for many mathematicians. For example Arnold told that while being a young graduate student he had copied the list of Hilbert problems in his notebook and always kept it with him. But when Gelfand learnt about that, he actually mocked Arnold on this. Arnold saw problem solving as an essential part of great mathematical achievements. For me it’s different. I see the process of mathematical creations as a kind of recognizing a preexisting pattern. When you study something — topology, probability, number theory, whatever — first you acquire a general vision of the vast territory, then you focus on a part of it. Later you try to recognize ”what is there?” and ”what has already been seen by other people?”. So you can read other papers and finally start discerning something nobody has seen before you.
Is the emphasis on problems solving a kind of romantic view: a great hero who conquers the mountain?
Yes, somehow a kind of sportive view. I don’t say it is irrelevant. It is quite important for young persons, as a psychological device to lure young people to create some social recognition for great achievements. A good problem is an embodiment of a vision of a great mathematical mind, which could not see the ways leading to some height but which recognized that there is a mountain. But it is no way to see mathematics, nor the way to present mathematics to a general public. And it is not the essence. Especially when such problems are put in the list, it is something like a list of capitals of great countries of the world: it conveys the minimal possible information at all. I do not actually believe that Hilbert thought this is the way organize mathematics.

Answer 3

this is ultimately a subjective and personal question and "over the long run" what problems are considered important to some degree go in and out of scientific fashion, but there can be some rough common guidelines that many would agree with, and also top experts have considered the question. problems are quite ubiquitous and its more a process of narrowing it down.

• #1 on the list is almost always, talk to your advisor! it is part of their job & if s/he is not forthcoming with any ideas than maybe thats not a great sign & consider you might benefit from or need another.

• what are many people in your university working on? each university typically has particular specializations and there will be enthusiasm or even excitement for particular areas/ problems.

• look at awards in the field to see what areas they study, or prizes. in TCS its Turing award, Godel prize, Nevanlinna Prize, Millenium prizes. obviously these are for very top/breakthrough work but by nature they all encompass large areas where there is incremental work.

• top TCS blogs are a great source of taking the pulse of the community interest in various problems.

also to answer this question it may be insightful to "go back to the roots" in the following sense. one of the legendary masters in this area among the greatest track record possible is Hilbert the mathematician, and many of his fundamental ideas on problem selection apply & are worth reviewing/ studying. many of his open problems that drove math at the turn of the 20th century turned out to have amazing/ deep connections to algorithmic theory wrt eg undecidability, eg Godel's thm, the Halting problem, and the pivotal 10th problem. his views are summarized by Lagarias, sec 9 in evaluating the Collatz conjecture as a "good problem":

It is difficult and often impossible to judge the value of a problem correctly in advance; for the final award depends upon the gain which science obtains from the problem. Nevertheless we can ask whether there are general criteria which mark a good mathematical problem. An old French mathematician said: “A mathematical theory is not to be considered complete until you have made it so clear that you can explain it to the first man that you meet on the street.” This clearness and ease of comprehension, here insisted on for a mathematical theory, I should still more demand for a mathematical problem if it is to be perfect; for what is clear and easily comprehended attracts, the complicated repels us. Moreover a mathematical problem should be difficult in order to entice us, but not completely inaccessible, lest it mock at our efforts. It should be to us a guide post on the mazy paths to hidden truths, and ultimately a reminder of our pleasure in its successful solution.

Lagarias summarizes these elements as:

1. Is the problem clear, and simply stated problem?
2. Is it a difficult problem?
3. Does it seem accessible and not "mock our efforts to solve it"?

unfortunately many open problems fail on #3 but as mentioned, there are always nearby problems and relaxations that are considered more accessible, and even just formulating these relaxations can be considered part of valid research.