What's the difference between

off-policy reinforcement learning algorithms


offline policy reinforcement learning algorithms ?

Or do they mean the same thing ? thanks

  • 4
    $\begingroup$ Please make your question self-contained and provide background and motivation for your question. See How to ask a good question. $\endgroup$ – Kaveh Dec 1 '10 at 14:38
  • 2
    $\begingroup$ to be more concrete: provide a short summary or link to description of RL, and also link to or explain/summarize the context of the two different kinds of RL you're asking about. $\endgroup$ – Suresh Venkat Dec 1 '10 at 16:17

I assume that you know what "policy evaluation" means. Briefly speaking, it refers to the task of estimating the value of a given policy.

In the RL literature, the off-policy scenario refers to the situation that the policy you want to evaluate is different from the data generating policy. In other words, you behave in one way, but you want to evaluate the value of another policy. This should be contrasted with the on-policy scenario that the policy you want to evaluate is the same policy that generates (or generated) data.

The offline sampling scenario (and not "offline policy") is the scenario that you already have some samples and now you want to perform tasks like policy evaluation. In this scenario, the agent cannot have any further interaction with the environment. In other words, the data are given and fixed. This should be contrasted with the online sampling scenario that the agent actually interacts with the environment.

These two are orthogonal. So it is possible to have a on-policy/off-policy offline/online scenario.

Hope my short discussion has helped. You may also take a look at Sutton and Barto's textbook.

Amir massoud

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.