What's the difference between
off-policy reinforcement learning algorithms
offline policy reinforcement learning algorithms ?
Or do they mean the same thing ? thanks
Theoretical Computer Science Stack Exchange is a question and answer site for theoretical computer scientists and researchers in related fields. It only takes a minute to sign up.Sign up to join this community
I assume that you know what "policy evaluation" means. Briefly speaking, it refers to the task of estimating the value of a given policy.
In the RL literature, the off-policy scenario refers to the situation that the policy you want to evaluate is different from the data generating policy. In other words, you behave in one way, but you want to evaluate the value of another policy. This should be contrasted with the on-policy scenario that the policy you want to evaluate is the same policy that generates (or generated) data.
The offline sampling scenario (and not "offline policy") is the scenario that you already have some samples and now you want to perform tasks like policy evaluation. In this scenario, the agent cannot have any further interaction with the environment. In other words, the data are given and fixed. This should be contrasted with the online sampling scenario that the agent actually interacts with the environment.
These two are orthogonal. So it is possible to have a on-policy/off-policy offline/online scenario.
Hope my short discussion has helped. You may also take a look at Sutton and Barto's textbook.