I am studying online convex optimization, and it is stated that when we make a decision, we observe loss corresponding to our decision. In some problems like multi-armed bandit problems, we know the structure of the problem and we learn the parameters of the model. Or, in customer choice models, we know that customers will click on an ad with a probability that is determined based on multi-nomial logit. In these examples, we know the format of the objective function, but we don't know the true value of the parameters, and learn the value of parameters. I am wondering how we can consider parameter learning in OCO? Does OCO mean that we just observe the outcome if a customer clicks on a specific link or not, and there is no parameter learning part?

Based on my understaing, online convex optimization is used for policy learning not parameter learning. In RL, decisions depend on the state of the problem. However, there is no explicit dependency between decisions and state of the problem. I am wondering how we can consider dependency to states like remained budget?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.