# Convergence of Q-learning with non-linear function approximation

Q-learning is a well-known algorithm in Reinforcement learning which enjoys great empirical success but with insufficient theoretical understanding. In the tabular setting, it is known that if each state-action pair $(s, a), \forall s \in \mathcal{S}, \forall a \in \mathcal{A}$ is visited for infinite number of times.

In the function approximation setting, however, the algorithm is not known to be convergent. There are some counterexamples where it diverges, under various assumptions. I was wondering if Q-learning with nonlinear function approximation could converge if each state-action pair is visited for infinite times. If not, I wonder if there is any convergent variation of Q-learning algorithm that uses nonlinear function approximation.