If one is trying to find the global minima of a convex function using gradient descent then one will get a run-time which is a function of $\epsilon >0$ where $\epsilon$ measures the accuracy of the solution. Here apart from the error coming due to the finiteness of the run-time we are also getting an error from the machine precision.
But I guess this is not the situation when say one is using an LP solver to solve a LP which we can consider to be giving us an ``exact" answer in polynomial time. Such solvers would typically use some pivoting rule based simplex method. Here the only source of $\epsilon$ is I guess the machine precision and not algorithmic.
What is the mathematically rigorous way to distinguish between the above two situations?
If we were to say use gradient descent to solve a LP then can there be a sense in which we could say that it might work "faster" than calling a LP solver?