(1) Lance Fortnow and Scott Aaronson (Section 1.3) give good discussions about the role of oracles/relativization in general, and I believe most, if not all, of their comments remain valid whether or not the oracle is computable.
On the other hand, thinking of oracle separations as query complexity separations (one of the views in Aaronson's paper), a non-computable oracle gives a query complexity separation where the function being queried is non-computable, which means the function being queried probably didn't arise in the real world. Nonetheless, it still seems like a potentially good guide to research.
(2) From the point of view of the Arora, Impagliazzo, and Vazirani paper, I'd say that non-computable oracles are far far outside the realm of the "real world" of computation.
(3) If we think of the "computational world relative to an oracle" as simply another model of computation, then relative to a computable oracle the computable sets in this model are of course the same as the standard model, whereas relative to a non-computable oracle, there are "computable" sets that are not computable in the standard model. (This is completely trivial -- it's mostly a philosophical viewpoint.)
(4) In a similar manner to random oracles, generic oracles tend not to be computable, though I imagine many (but probably not all) generic oracle constructions can be adapted with very little work to get computable oracles from them. (There is in fact a notion of genericity $\mathcal{R}$ such that $\mathcal{R}$-generic oracle results are the same thing as random oracle results, so this is really a generalization of the observation about randoms.)